It's not length that makes AI forgetful, it's contradiction

The rule of thumb: the more text an AI reads at once, the worse it answers. A new paper with 126,473 openly published runs points to a different main culprit: competing statements about the same question. T-Mobile ran into exactly that with its support agents.

Analysis Nov 21, 2026 10 min read Original analysis by our editors
Illustrative image, AI-generated

What this is about

The more text you give a language model at once, the worse it answers. That's the common rule of thumb, and the phenomenon has a name: context rot. Even so, vendors keep advertising ever-larger context windows. The context window is the amount of text a model can take in at once. Tencent's new open-weights model Hy4 handles 1,048,576 tokens, and Google's Gemini 4 also handles a million. With Gemini 4, though, input, reasoning steps and output all share that budget, according to Google's documentation.

A paper from early November contradicts the rule of thumb on one crucial point: length alone barely hurts lookups. Things get expensive when the context contains two different answers to the same question. Almost at the same time, T-Mobile described how exactly this derailed its support agents. We explain the mechanism, show its limits and pull together what has been shown to help in practice.

The experiment: one piece of evidence, lots of filler

The paper is called "Length Is Cheap, Competition Is Not" (arXiv 2611.04169). Yinuo Liu wrote it alone and submitted it on November 3, 2026. It's a preprint and hasn't been peer-reviewed yet. The setup is simple. The context contains exactly one piece of evidence linking an everyday word to a four-digit number, along the lines of: "The value for river is 4711." At the end, the text asks for that value. What's measured is the loss, meaning how uncertain the model is about the correct number. The higher the value, the more uncertain.

Liu pads the text around the evidence in three ways:

  • neutral: random numbers, random words or the same sentence repeated
  • competing: sentences in the same format as the evidence, using keys drawn from 51 nouns. Some of them name the same key with a different value. These "rivals" are the heart of the study.
  • masked: the filler is there, but it's made invisible to the model's attention.

Five open models were tested: Qwen2.5-0.5B and -1.5B, SmolLM2-1.7B, OLMo-2-1B and Llama-3.2-3B. Qwen2.5-7B was added for a side experiment. The code (MIT license) and all measurements, including random seeds, are openly available on GitHub. So anyone can rerun the experiments.

The results: one contradiction weighs more than 30,000 tokens

Condition (Qwen2.5-1.5B, 8K to 32K tokens) Accuracy
neutral filler, visible 0.98 to 1.00
same filler, masked (32K) 0.52
sentences in evidence format 0.11 to 0.21

The paper's abstract gives 0.16 to 0.33 for sentences in evidence format, depending on the analysis. So treat these numbers as ballpark figures. The rivals are what tip the scale. Without a matching key, the loss sits at 0.02 to 0.34; with at least one rival, it's 1.38 to 3.24, measured in nats, a unit of uncertainty. That's a factor of 9 to 110. Each additional rival makes the result worse again. We roughly converted this into probabilities: without rivals, the model assigns the correct number about 71 to 98 percent; with rivals, only 4 to 25 percent.

To explain this, Liu measures attention. That's the mechanism a model uses at each step to weigh which parts of the text matter right now. Think of it as a limited budget of spotlight. How much light still falls on the correct evidence explains 35 to 58 percent of the error variance, depending on the model. Text length explains only 2 to 8 percent. Qwen2.5-1.5B has a tipping point: once the share drops below about 0.35, errors climb steeply. The intuition: neutral filler doesn't look like an answer and doesn't draw light away. A rival with the same key, by contrast, looks exactly like the evidence.

Liu names the limitations himself. The relationship is a correlation, not proof of cause. The models top out at 3 billion parameters, and everything rests on a single synthetic task. Lengths up to 32K were measured on only two models. The split by rivals was done after the fact. The steepness of the slope varies by model. So the author explicitly doesn't claim a general law.

Why earlier studies sounded different, and where the thesis ends

The opposing view comes mainly from Du et al. ("Context Length Alone Hurts LLM Performance Despite Perfect Retrieval," EMNLP Findings 2025). They measured drops of 13.9 to 85 percent, even when the filler was masked. Liu repeats that experiment. On MMLU, Qwen2.5-7B does fall from 65.8 to 48.1 percent with 30,000 masked tokens. On lookup, however, masking costs about 46 points, while the same text left visible costs nothing. Liu concludes that masking punches holes in the position information and therefore doesn't cleanly measure length.

That doesn't fully settle the debate. Du et al. also tested visible whitespace as filler, and scores dropped there too. At 30,000 tokens, Llama-3.1-8B lost 20.2 points on MMLU, and Mistral-7B lost 30 points on math problems (GSM8K). Those are reasoning tasks, though, not lookups. GPT-4o was barely affected. Adobe's NoLiMa benchmark (ICML 2025) shows a second limit. When the question and the evidence share no words, GPT-4o collapses from 99.3 to 69.7 percent at 32K. In Liu's task, by contrast, the key appears verbatim in both question and evidence. Chroma's Context Rot report from July 2025, covering 18 models, found both effects: even a single distractor hurts performance, and more so the longer the text. Chroma itself doesn't quantify how much is due to length and how much to distractors.

Our take: "length is cheap" holds up for verbatim lookup in small models. When reasoning over scattered evidence, length still costs you. In practice, that changes little, because contradictions are the more expensive mistake, and you can clean them up in advance.

How this differs: haystacks, poisoning, memory

Needle-in-a-haystack tests hide a fact in neutral filler. That's exactly the condition the paper says costs almost nothing. So high scores on such tests say little about company documents full of similar statements. Google itself concedes in its documentation that accuracy drops as soon as multiple needles are involved. Hy4's Model Card lists no long-context scores at all. On llm-stats' MRCR leaderboard, an eight-needle test, all 26 entries are vendor-reported, and not one has been independently verified. Gemini 4 and Hy4 aren't listed at all. We found no independent measurements for the newest models.

Context poisoning means something else. The term comes from a failure taxonomy by Drew Breunig. It describes an error, often the model's own hallucination, that lands in the context and then keeps getting picked up again. Breunig's example is a Gemini agent playing Pokémon that fixated on impossible goals. With competition, by contrast, the model initially makes no mistake at all, because the contradiction is already in the sources. Closer is Breunig's "context clash": in a study by Microsoft and Salesforce, scores fell by 39 percent on average when information arrived spread across multiple conversation turns. OpenAI's o3 went from 98.1 to 64.1.

Memory solutions only help if they replace old facts instead of stacking them. Database vendor Memgraph describes context rot exactly this way:

"Old information is never removed; new information is simply piled on top."

So if you just keep piling up notes, you're creating the next rivals.

The T-Mobile case

At a VentureBeat summit in October, T-Mobile CIO Jeff Simon described three attempts. First, all support documents went into one large context window. But contradictory instructions and ambiguous customer phrasing sent the agents down the wrong paths. "That didn't work at all," Simon said. The second attempt, a knowledge graph, meaning a network of linked facts, was too slow for live conversations, and the agents got lost in it. Today, a lookup table maps common customer phrasings to the next step. A routing agent figures out what the customer wants and hands them off to the agent with the right knowledge. Simon calls it "actually very simple, but fast." Irrelevant material is stripped out before a request ever reaches the model. On top of that, a small model acts as an "LLM judge" and reviews every conversation. It discovered, for example, that the customer-facing model was making up hold music that T-Mobile doesn't even use. Engineers spent two weeks hunting for the cause.

The VB Pulse survey from October 2026 shows T-Mobile isn't alone. 78 percent of 97 organizations traced at least one confidently wrong answer back to missing or contradictory business context. In the worst case, the wrong information reached customers or partners at 21 percent of them, and at 12 percent it ended up in financial, regulatory or board reports. Adobe offers an example in the same article: an EU campaign shouldn't have US consent rules in its context. Another VentureBeat piece about agents that "forget" compliance rules, however, is a guest post with a made-up scenario and no measured data.

What has been shown to help in practice

  1. Remove duplicates and old versions. Every rival increases the loss (Liu). In the VersionRAG study, a standard document search answered 58 percent of version-dependent questions correctly, while a version-aware one got 90 percent. The HoH benchmark shows that outdated passages lower accuracy even when the current version is provided too.
  2. Resolve contradictions up front. In the ConflictRAG approach, turning off conflict detection cost 16.6 points. For temporal conflicts, the newest source wins there.
  3. Route deliberately instead of loading everything. T-Mobile shows this, as does Anthropic's guide with "just-in-time" retrieval and sub-agents that get a clean context. At Uber, an agent with a context graph found a table in 38 seconds. Without the graph, it took over 20 minutes and reached the wrong conclusion.
  4. Have the model quote the evidence first. Du et al. have the model repeat the evidence before answering. For GPT-4o, that added up to 4 points on the RULER test.
  5. Check automatically. In VB Pulse, 57 percent of respondents automatically check against trusted sources.

One point is fuzzy: placement. OpenAI advises putting instructions at both the beginning and the end of the context. Google recommends putting the question at the end.

You can recreate the core finding yourself; we didn't run our own experiment for this. Take a local model, give it a pricing sheet with about 30,000 tokens of neutral filler and ask for the price. Then replace the filler with a second, older version of the same pricing sheet with a different price. According to the paper, the second run should come out noticeably worse.

The bigger picture

If you feed agents company knowledge, the message is practical. A bigger context window is no substitute for a clean knowledge base. Vendor claims like "one million tokens" only tell you how much fits, not how well the model handles contradictory content. It remains open whether large frontier models cope better with rivals. The paper only tested small models, and it's not proven that attention is actually the cause. Our take: the thesis is a useful diagnosis, not a law of nature. But it fits remarkably well with what T-Mobile, Adobe and the survey report from the field.

#context rot#long context#agents#rag#research#context engineering

Ask the AI assistant

Questions about this article, installation or background? Our assistant knows this page and runs on our own machine, not in someone else's cloud.

Connecting …

The Assistant is an AI (Gemma 4) and can make mistakes – please check important details against the source. Questions are used only to generate an answer and are deleted after one hour. Please don't enter any personal data.

Comments

What do you think? Already tried it? Write it here, no sign-up needed.

  1. Loading comments …

Comments are public. Links aren't allowed, and we delete abusive posts. You can delete your own comment at any time in this browser. Learn more in the Privacy Policy.