AI is built to sound right, which is not the same as being right
Dan Klein, a Berkeley professor who builds AI systems, explains that today's AI is trained to sound convincing rather than to be correct. That is why it can hand you a confident answer that is quietly wrong, and why smooth writing is no longer proof that the facts under it hold. The skill that keeps its value is not producing the words, which the AI now does well, but being the person who checks whether they are true before they go out.
Source: The AI Report ↗