Study reveals half of AI assistants produce significant factual errors
(MENAFN) A recent study involving a collaboration of public media organizations has revealed that AI assistants powered by large language models, such as ChatGPT, Copilot, Perplexity, and Gemini, frequently produce errors in news-related queries.
According to reports, 45% of all responses contained at least one significant mistake. The most common problems were incorrect or missing citations, affecting 31% of answers, while 20% included factual inaccuracies such as incorrect dates, names, or event details. In certain instances, the AI tools even generated fake news links, mimicking real URLs but leading to error pages instead of actual articles.
“The answers we received are worrying and have not made us feel more confident about loosening control,” said Pal Nedregotten, technology director at the Norwegian broadcaster involved in the study. “It is therefore not an option for NRK to open up for scraping permanently.”
For the investigation, the broadcaster temporarily allowed the AI companies to scrape content from its website to observe how editorial material might be utilized or represented in language model services. Nedregotten noted that the organization has since blocked all scraping to safeguard its journalistic and copyrighted material.
The test was modeled after a previous experiment conducted by a major UK broadcaster. In this version, 22 public media outlets asked the four AI assistants 30 standardized questions each. Journalists then evaluated the responses based on five criteria: accuracy, sourcing, separation of fact and opinion, neutrality, and contextual relevance. Questions ranged from topics such as “What is Nvidia known for?” to “Why can’t Ukraine join NATO?”
Legal Disclaimer:
MENAFN provides the
information “as is” without warranty of any kind. We do not accept
any responsibility or liability for the accuracy, content, images,
videos, licenses, completeness, legality, or reliability of the information
contained in this article. If you have any complaints or copyright
issues related to this article, kindly contact the provider above.
Legal Disclaimer:
EIN Presswire provides this news content "as is" without warranty of any kind. We do not accept any responsibility or liability for the accuracy, content, images, videos, licenses, completeness, legality, or reliability of the information contained in this article. If you have any complaints or copyright issues related to this article, kindly contact the author above.
