Context rot makes bigger windows less reliable.
A window size on a model card measures capacity, not usable attention.
Chroma's Context Rot report came out in July 2025 with tests across 18 models, among them GPT-4.1, Claude 4, Gemini 2.5 and Qwen3.
The design matters more than the model list. Each task stayed the same size while the surrounding text grew, across eight input lengths and 11 positions for the sentence holding the answer.
Performance fell anyway, and it fell unevenly.
Context is a budget, and every token you spend comes out of an attention account you cannot top up.
Why context rot hides in benchmarks
The long context numbers that arrive with a launch come mostly from Needle in a Haystack, a task where the model retrieves a sentence it has effectively been told to retrieve.
That measures word matching. It says little about the thing people actually ask for, which is finding an answer to a loosely worded question inside a pile of related text.
Chroma loosened the link between question and answer. Pairs with lower similarity degraded sooner and further as the input grew.
Distractors did the rest. A single plausible but wrong passage cut accuracy against the baseline, and four of them cut it further.
The failures were not evenly spread. In one haystack and needle pairing, two of the four distractors accounted for most of the hallucinated answers.
Model families behaved differently too. Claude models abstained more often when they could not find an answer, while GPT models produced confident wrong ones.
Then came the result I did not expect: coherent text hurt. Across all 18 models, removing the logical flow by shuffling sentences improved retrieval.
So a tidy retrieval test with well ordered text and a generous question reports a ceiling that production never reaches, which is roughly what a flattering benchmark looks like from the inside.
What to change
Three changes survive contact with a real system.
1. Measure at the length you actually send. Build a small set of real inputs at your upper percentile token count and score answers there, because a model that is dependable at 8,000 tokens can be unreliable at 80,000 on an identical task. A prompt evaluation check is a fair way to find out where your own line sits.
2. Cut near misses before you add context. The distractors that hurt most were the ones closest to the right answer, so a retriever that returns twenty chunks hands the model twenty chances to choose wrong. Tighter reranking, or dropping weak chunks, buys more reliability than a wider window. Anyone who has tuned a pipeline knows how much of this sits in retrieval quality rather than in the model.
3. Curate the window, and let the agent fetch the rest. Anthropic's guidance for context engineering puts the target at the smallest set of high signal tokens, with file paths, queries and links kept in context as pointers while the data loads only when a tool call needs it.
For work that runs for hours, compaction, notes written to disk, and subagents that return short summaries keep the window small enough to stay sharp. That is a different skill from prompt writing, closer to agent tooling and to information architecture than to phrasing.
Two assumptions do not survive
The old assumption was that a bigger window is more memory. The new one is that it is more surface for noise, and recall thins out as you fill it.
That changes the work: the interesting engineering is deciding what never gets in, not how much can fit.
Where did your last agent failure turn out to be a context problem rather than a reasoning problem?