Many media executives are replacing journalists with AI to cut costs and capitalize on hype, producing poor-quality, hallucinatory content that angers readers and reporters. AI hallucinations appear across online platforms, including flawed Google Overviews, gambling content in newspapers, and content farms that copy journalists. Google's use of AI diverts readers and reduces membership and ad revenue for publications. Reporters find AI tools inadequate for daily reporting tasks. An NYU-led team led by Hilke Schellmann devised a test measuring accuracy and truth and found current models can make short summaries with few hallucinations but underperform on accurate long summaries of around 500 words.
The result of these efforts so far has left a lot to be desired. We've come across countless examples of publications inadvertently publishing garbled AI slop, infuriating readers and journalists alike. AI's persistent hallucinations are already infecting large swathes of our online lives, from Google's hilariously terrible AI Overviews mangling trustworthy information to brainrot gambling content appearing in newspapers to entire AI slop farms that blatantly rip off real journalists' work.
Worse yet, Google's embrace of the tech is actively hurting the bottom lines of publications by keeping readers - and with them, much-needed membership and display ad revenue - away from the content their AI is monetizing. Meanwhile, journalists themselves are finding out the hard way that AI is woefully inadequate at meaningfully helping them out in their day-to-day work.
Schellman and her colleagues created a new test to evaluate the "journalistic values of accuracy and truth," finding that while most currently available AI models, including Google's Gemini 2.5 Pro and OpenAI's GPT-4o - which is still avialable to paying customers following the release of GPT-5 after OpenAI scrapped plans to pull it down - successfully generated short summaries of meeting transcripts and minutes from local government meetings with "almost no hallucinations."
Collection
[
|
...
]