AI accuracy is Google’s problem—until it becomes a publisher’s

It’s hardly a revelation to say that Google’s AI Overviews sometimes get things wrong. The Gemini-written summaries at the top of search results have been misfiring on and off since they debuted in mid 2024. It feels like Google will never fully live down the infamous “glue on pizza” moment, and the errors come often enough that they always carry the warning, “AI can make mistakes, so double-check responses.”

Nonetheless, AI Overviews are now the reality for anyone (read: everyone) who uses Google. At some point, publishers have to stop treating each new mistake as a curiosity and start treating the system that produced it as their working environment.

This spring, The New York Times commissioned AI startup Oumi to measure the problem. The ultimate finding: The latest version of AI Overviews was accurate 91% of the time. That looks respectable until you run the math against Google’s billions of daily queries. A single-digit error rate at that scale produces millions of bad summaries every hour.

The Times drove the point home by citing BBC tech reporter Thomas Germain, who ran an experiment. He published a fake blog post crowning himself the world’s best hot dog eating tech journalist. Within a day, AI Overviews were repeating the claim, apparently without checking.

The stunt looks silly because the query was silly. But the underlying mechanism isn’t. Germain succeeded largely because he owned the only page anyone had ever written on that subject. It was an information vacuum. For a well-covered topic, a lone rogue post would barely register.

The lens publishers can’t remove

The hot dog stunt is only one failure mode; it turns out AI answer engines can go wrong in several ways. And the stakes for publishers keep rising: AI Overviews now appear in most searches. An April report from AI-visibility startup QuickSEO put their prevalence at 60.23%, and that was before Google’s May I/O conference tightened the loop between AI Overviews and AI Mode, letting users slide from a summary into a conversational follow up without leaving the results page.

Chatbots aren’t the biggest surface here. Google is. People can opt in to ChatGPT or Claude, but they get served AI Overviews whether they want them or not. That default status is what makes accuracy such a load-bearing question. Publishers can’t set the terms of the lens their work passes through, but they still have skin in the game once it does.

Ubiquity isn’t the same as blind acceptance. Trust in AI answers scales with the stakes of the question. A roast chicken recipe gets less scrutiny than a cancer treatment query, even if the entry point is identical in both cases.

By the time a reader decides to double check an answer, the framing has already landed. The summary supplies the vocabulary, sets up the follow up questions and points to what feels worth investigating next. If a publisher the reader trusts is cited in the summary, confidence rises even when the citation is never clicked. I’ve made the case before that citation is a form of value for publishers, but that value depends on the reporting being accurately represented.

Three ways the machine gets it wrong

To map how AI Overviews fail, I spoke to Isis Blachez, the AI lead at Newsguard who runs the organization’s AI False Claims Monitor. She sorts the failures into three buckets, and each one shows up in the Times study.

1. Weak or irrelevant material rises to the top. This is the glue-on-pizza scenario. That recommendation came from a Reddit post written as a joke (we hope), which made it irrelevant to a serious cooking query. The catch is that the post did answer the question head on, and direct answers rank well in AI discoverability. Journalistic content generally performs better in AI engines when it’s optimized for machines. When it isn’t, or when it’s blocked outright, thinner material can grab an outsize share of the response.
   
   “We do [reliability] ratings of news sites,” explains Blachez. “And we saw that for most of the highly ranked sites, they were blocking a lot of the AI bots, and then most of the low-quality sources were giving full access to AI web crawlers.”
2. The AI finds the right source and misreads it. This is the quietest failure mode and possibly the most consequential. Blachez points to a case where multiple chatbots cited Snopes to confirm a false claim that Iran had attacked a Pakistani flagged oil tanker. The Snopes piece was actually the debunking. The machine flipped it.
   
   “Sometimes, even if it’s citing a credible source, it can be incapable of citing it well or retrieving the information correctly,” Blachez says.
   
   The reporting itself is fine in these cases. The machine is the point of failure. This version of the problem is the one that often features in lawsuits against AI companies.
3. The information pool has been poisoned on purpose. The hot dog story is the innocent version of this. The pro-Kremlin Pravda network is the malicious one. It flooded the web with millions of articles across sites designed to look like news outlets, pushing Russian narratives at industrial scale. Coordinated actors publishing similar sounding claims across many domains can manufacture the appearance of consensus and crowd out honest reporting in retrieval systems.
   
   “So what we’ve observed that worked with Pravda is flooding search results,” says Blachez. “It’s like putting the same information with practically the same language, many domains, many times and just dominating narrative on that specific topic.”

• Subscribe to our newsletter

  How AI is changing media, journalism, and content creation.

  Learn More

Building the machine readability pass

So the answer layer can go sideways because access is blocked, the material is manipulated, or the content itself invites misreads. The AI operator has an obvious duty to raise the floor on quality. What about the publisher?

A lot of newsroom people have quietly written this problem off as somebody else’s, on the grounds that AI systems are a black box. That framing is understandable and mostly wrong. Publishers can influence all three failure modes. Being in the mix means not being blocked. Discouraging misreads means writing for machine comprehension as well as human. Beating manipulation means publishing your own answers to the queries you want to own.

Blocking crawlers is a legitimate choice. Copyright and the absence of any compensation model are real reasons to shut the door. And when journalism is blocked, Google and every other AI company still owe their users a duty of care with the material they do use. But when journalism is available to the AI, publishers have levers to make sure it’s represented correctly.

Every newsroom already runs an SEO pass on its work. The most effective way to shape what AI Overviews and chatbots surface is to run a machine readability pass alongside it. This isn’t just standard GEO hygiene like matching titles to common queries. It means writing so that the tricky parts of a story remain unambiguous to a machine reader, even when they’re already obvious to a human.

In practice, that means saying the quiet part out loud. A human understands that “alleged” applies to a whole run of paragraphs even when the word only appears once. A machine may not carry the qualifier forward.

A short set of questions to run through the pass:

• Are dates explicitly tied to the correct events?
• Is it clear whether an allegation is being reported, verified or debunked?
• Is the primary conclusion stated plainly rather than left entirely to implication?
• Are corrections and updates obvious?
• Does the article distinguish the original source from later repetition?
• Does the headline create ambiguity that the body later resolves?

As with SEO, editing for machine clarity tends to sharpen the human read too. The trade off is that the pass improves the odds. It does not guarantee anything. The goal isn’t “AI proof” journalism. The goal is to strip out avoidable ambiguity and give accurate reporting a better shot at surviving the answer layer.

Publishers can’t dictate what Google says about their work, and they shouldn’t be expected to patch the flaws in someone else’s product. But as AI settles in as a default filter between journalism and its audience, treating that as a reason to disengage stops being a strategy. Newsrooms can still make the truth easier to find, harder to misread and much harder to replace.

A version of this column appears in Fast Company.