AI has mostly stopped inventing sources, but it still sends you to closed restaurants
A Stanford study finds that today's frontier models barely invent citations anymore. On everyday questions, though, they still get things wrong a lot. Several tests from this fall turned up wrong health insurance premiums, restaurants that had closed and shops wrongly reported as shut down. We compared the tests. The pattern: the problem today is rarely invention. It's mostly outdated or mixed-up data.

What this is about
Made-up citations were long the textbook example of AI hallucinations, meaning invented claims that sound convincing. That weakness is now disappearing. A Stanford study by Owen Queen and James Zou, which Nature also covers today, lists it among the "expired" findings. But several tests from this fall paint a different picture once the questions touch the real world. Is that restaurant still open? How much will my health insurance cost? Where is this gadget cheapest on Black Friday? Chatbots still get questions like these wrong a lot. They rarely make things up. Mostly they repeat outdated data or mix things up.
We put the tests side by side to answer three questions. How high is the error rate with and without web search? What kinds of errors show up? And who pays for them? The timing matters: in Switzerland, the deadline for switching health insurers is November 30, and Black Friday is next week.
The good news: invented sources have almost disappeared
Queen and Zou re-tested 50 widely cited research findings about language models on 20 models from eight vendors. The models were released between July 2024 and September 2026. The result: 44 percent of the findings no longer hold, 34 percent have weakened considerably, 16 percent still hold, and 6 percent have actually gotten stronger. Of the 22 findings with an estimated expiry date, most lasted only 15.7 months on average.
The clearest change is in invented sources. In a 2023 study by Walters and Wilder, GPT-3.5 made up 55 percent of its citations. With the same prompts, GPT-6.1 Sol now scores 0 percent and Claude Sonnet 5.5 scores 1 percent, according to The Decoder. The progress is uneven, though. The small Qwen3.7-Flash, also a 2026 model, invents 69 percent of its citations. And not everything is solved. On two-hop questions, where you have to connect two facts, Sonnet 5.5 still fails in about 10 percent of cases. Contradictory answers to time-related questions have become rarer, but they still happen. One caveat: the study is a preprint, so it hasn't been peer-reviewed yet.
The tests at a glance
| Test | Tested by | What was tested | Web search | Result |
|---|---|---|---|---|
| Shelf Life (Stanford) | Queen/Zou | invented citations | – | 0–1 % for frontier models, 69 % for Qwen3.7-Flash |
| KI-Kassencheck 2027 | The Zone Media (GEO agency in Lucerne) | 1,363 premium figures, 26 cantons, October 2026 | yes | 939 wrong (69 %) |
| PlaceBench | Voygr (sells a place-verification API) | 6 assistants, 150 place questions, 121 cities, November 11–14 | not stated | 66 of 900 answers wrong (7 %), 12 of them pointing to closed or relocated places |
| Local Search Benchmark | Voygr | 2,415 answers, February/March 2026 | on and off | 8 % serious errors (OpenAI with search), about 20 % (Claude without search) |
| Invisible to the Machine (Bali) | Norly Research | 4,776 venues, 2,208 answers | yes | 93 recommendations of closed venues, 0.08 % invented |
| Portsmouth test | Koozai (marketing agency) | 100 restaurant recommendations | n/a | closed restaurants in ChatGPT and Google AI Mode |
| Fake shop study | Podqi (brand protection) | 6,795 shopping answers, September | yes | 9 % with risky shops, 367 of 718 domains dead |
| Price comparison | ShopSavvy | 270 products, AI Mode vs. Shopping tab | – | 80 prices (30 %) don't match |
We calculated the percentages in parentheses ourselves from the published figures.
With or without web search: looking things up isn't enough
You might think web search fixes the problem. The tests show a mixed picture. In the Voygr benchmark, search adds 10 to 21 points on questions like "Find me a café nearby." On business and booking tasks, though, Claude (89.6 without, 84.1 with search) and Gemini (84.0 without, 78.7 with search) did worse with search. Only OpenAI improved slightly, by 1.1 points. Without search, about one in five addresses Claude recommended had a serious error: a place that doesn't exist, is closed or sits in the wrong neighborhood. For OpenAI with search, it was 8 percent. Keep in mind that the test used older models like gpt-5.2 and claude-sonnet-4-5, and Gemini did the grading.
The Swiss analysis is the most striking. Every system was allowed to search the web, and according to Swiss IT Magazine almost every answer cited a source. Still, 939 of 1,363 premium figures were wrong. Answer quality depended heavily on how the models were accessed:
| Access | wrong / total | Rate |
|---|---|---|
| APIs (gpt-5.4-mini, Claude Haiku 5.5, Gemini 3.6 Flash, Perplexity Sonar) | 705 / 861 | 82 % |
| Google AI Mode | 87 / 120 | 73 % |
| Gemini app | 113 / 163 | 69 % |
| ChatGPT app | 34 / 219 | 16 % |
In an earlier measurement on October 15, only 33 of 258 figures were correct. That's where the "one in eight figures" number in the coverage comes from. When asked for the cheapest insurer in a given town, the bots were right in 101 of 312 cases, or about a third. The Zone suspects that more thorough searching would improve the results. Our take: web search just moves the problem somewhere else. The bot is only as good as the page it finds, and it can grab the wrong page.
Invented, outdated, mixed up: the three error types
Invented answers are now rare. In the Bali audit, only one single name survived a strict check as probably made up: 10 mentions, or 0.08 percent.
Outdated answers are the typical problem. In Bali, ChatGPT, Claude, Gemini and Perplexity recommended venues that were verifiably closed for good 93 times. By our math that's about 0.75 percent of all 12,439 valid mentions, roughly nine times as many as the inventions. The author sums it up:
"The risk isn't that AI invents restaurants. It's that AI remembers restaurants that no longer exist."
There were similar cases in Portsmouth. Google's AI Mode recommended the seafood restaurant Samphire, which has closed permanently. The recommendations also included a seafront spot that had shut on June 30, almost four months before the test. At Podqi, just over half of the risky shop domains (367 of 718) were simply dead. And in its Black Friday test, t3n found that prices were "not always really up to date."
The cause usually lies in the directories the bots draw from. According to a BrightLocal analysis of 60,970 queries, Yelp shows up as a source in 80 percent of local ChatGPT answers. For Google's AI Overviews, our calculation from the tables puts about two-thirds of the citations on Google Business Profiles. Those listings aren't always current either. Voygr checked 86 confirmed closures: Google Maps still showed 23 of them as open or didn't list them at all. That's 27 percent. Voygr's founders estimate that 25 to 30 percent of all places close, move or change every year. Both figures come from a vendor that sells a verification product for exactly this problem.
Mix-ups are common too. Google's AI Overview announced that Anna Mae's Bakery in Millbank (Ontario) would close on July 31. The bakery actually closing was one with the same name in Illinois. In Switzerland, the figure was correct in 30 cases but came from a different canton. Other answers referred to a different type of insurance. t3n found prices for phones with a contract that were presented as purchase prices. For Viridian Bookshop in Ohio, the AI gave the wrong address and later claimed the shop had never existed.
Who pays for the errors
Users: According to the Swiss analysis, if you trust a wrong standard premium, you're off by a median of 550 to 650 francs a year. If you follow the wrong "cheapest insurer," you pay a median of 145 francs more, according to Infosperber. When shopping, the bots recommended a risky shop in 57 percent of the cases where one showed up at all (Podqi). According to ShopSavvy, when prices didn't match, Google AI Mode showed the higher price 60 percent of the time, 20.2 percent higher on average.
Businesses: Viridian owner Chanel Bush says one customer didn't come at all because of the false closure report. She links this to a drop in sales. The correction took weeks, and the false report kept popping up again in the meantime. Anna Mae's spent days answering calls and emails. Google promised a fix but gave neither a ticket number nor a timeline. Google told CBC that the "vast majority" of AI Overviews are accurate, and that errors happen where there's little high-quality information on a topic online. A quieter cost is invisibility. In Bali, 85.6 percent of all venues didn't appear in a single recommendation. Even among venues with at least 50 reviews, the figure was 72.6 percent.
And legally? On May 28, 2026, Munich Regional Court I ruled that Google is itself liable for false statements in its AI Overviews (case no. 26 O 869/26). The AI had linked two Munich publishers to fraud allegations against other companies. The ruling isn't final. Berlin Regional Court II had ruled the opposite way in a trademark dispute.
An important caveat: almost all of the field tests come from companies with a stake in the outcome. Voygr sells a place-verification API, Norly sells AI visibility tools, and The Zone is a GEO agency that helps companies show up in AI answers. Koozai and PinMeTo make their money from local marketing, Podqi from brand protection. To their credit, most of them disclose their methods. The PlaceBench data is available under a CC-BY license on GitHub. However, a Claude model graded the answers there, and one of the tested assistants (Claude Duet) is itself built on Claude.
Checklist: when you shouldn't trust the AI's answer
- "Is X open right now?" or "Does X still exist?" Check the business's website, call them or look at recent social media posts.
- Prices and deals with a date (Black Friday, promotions): Check the price directly with the retailer and factor in shipping.
- Amounts that depend on where you live (premiums, rates, fees): In Switzerland, double-check with the federal premium calculator: priminfo.admin.ch.
- Shop links, especially supposedly "official" regional sites: Type the address in yourself.
- Businesses or places with the same name: Compare the address.
- A citation isn't proof. Click the source and check whether it actually answers your question.
Explaining terms, rules and deadlines, on the other hand, works well. All 104 answers about the cancellation deadline correctly named November 30. And according to t3n, AI is more useful for product advice than for price comparisons.
The bigger picture
The Stanford study and the field tests don't contradict each other. Together they show that the problem has shifted. The models have learned to stop making things up. Their weak spot now is the data they pull from: directories nobody maintains and search results that don't match the question. Our take: a low "hallucination rate" tells you very little about whether an answer on opening hours or prices is correct.
For users in Germany, this is getting closer to home right now. Google's Ask Maps launched in Germany this week and did worst in the PlaceBench test: 49 points and 30 wrong answers out of 150. On the plus side, it was by far the fastest at 8 seconds.
A lot remains open. There's no independent, representative study. The tests are snapshots with changing model versions. And according to Queen and Zou, findings about AI have an expiry date too. That applies to these tests as well.
Sources
- Stanford Zou Group: The shelf life of LLM findings zou-group.github.io
- Nature: AI is ditching hallucinated references nature.com
- The Decoder: Rapid LLM development is outrunning the research the-decoder.com
- Paired Ends (Stephen Turner): The Shelf Life of LLM findings blog.stephenturner.us
- The Zone Media: KI-Kassencheck 2027 thezone.ch
- Bilanz: Krankenkassenwechsel – besser nicht die KI fragen bilanz.ch
- Infosperber: KI – keine Ahnung von der günstigsten Krankenkasse infosperber.ch
- Netzwoche: KI-Systeme sind keine guten Versicherungsberater netzwoche.ch
- Swiss IT Magazine: KI-Chatbots scheitern beim Check der Krankenkassenprämie itmagazine.ch
- dev.to (Voygr): Do AI assistants recommend closed places? dev.to
- Voygr: PlaceBench voygr.tech
- GitHub: voygr-tech/placebench github.com
- Voygr: LLM Local Search Benchmark Report voygr-tech.github.io
- Hacker News: Launch HN Voygr (Spiegel) hn.nuxt.dev
- arXiv: Invisible to the Machine (Norly Research) arxiv.org
- PPC Land: ChatGPT and Gemini miss 85.6% of Bali's food venues ppc.land
- Portsmouth News: ChatGPT and Google AI recommend closed Portsmouth restaurants portsmouth.co.uk
- PinMeTo: Google AI Overviews are inventing fake store closures pinmeto.com
- PPC Land: Google AI Overview forces Anna Mae's Bakery to deny fake closing ppc.land
- Cleveland Magazine: Willoughby bookstore confronts Google AI error clevelandmagazine.com
- KTTC: Uni Uni says Google's AI feature wrongly listed it as permanently closed kttc.com
- t3n: Schnäppchenjagd mit ChatGPT & Gemini t3n.de
- Let's Data Science: Podqi study finds AI chatbots recommend fake retail websites letsdatascience.com
- ShopSavvy: Same product, two prices shopsavvy.com
- BrightLocal: Where to get local citations for AI search brightlocal.com
- WBS Legal: LG München I – Google haftet für falsche Antworten in der KI-Suche wbs.legal
Ask the AI assistant
Questions about this article, installation or background? Our assistant knows this page and runs on our own machine, not in someone else's cloud.
The Assistant is an AI (Gemma 4) and can make mistakes – please check important details against the source. Questions are used only to generate an answer and are deleted after one hour. Please don't enter any personal data.
Comments
What do you think? Already tried it? Write it here, no sign-up needed.
Comments are public. Links aren't allowed, and we delete abusive posts. You can delete your own comment at any time in this browser. Learn more in the Privacy Policy.