Context Rot: Why Longer AI Agent Sessions Get Worse

Context rot is not "the model got confused" as a vibe — it is a specific, tested claim: as input length grows, model accuracy degrades on tasks it can solve easily at short lengths, even when the added tokens are relevant and coherent. Chroma's 2025 research tested 18 frontier models on controlled long-context tasks and found every one of them got measurably worse as input length increased — including simple retrieval and text-replication tasks that have nothing to do with reasoning difficulty. A model advertised with a 1M-token window does not reliably use all 1M tokens; some models showed serious accuracy loss well before their stated limit.

This matters for agents specifically because an agent session is not one call — it is dozens or hundreds of turns, each appending tool output, plans, and partial results to a transcript that the next call re-reads in full (or in a lightly trimmed form). A single well-packed prompt can still rot by turn 40, because the window is no longer one curated payload — it is an accumulated log nobody re-curated. Context rot is what context engineering becomes when you stretch it across a whole session instead of one call.

Four mechanisms compound across a session. Attention dilution: every additional token competes for the same fixed attention budget inside the model's layers, so any single fact's effective "share" of attention shrinks as more tokens surround it — this happens even if nothing you add is wrong. Lost-in-the-middle / position bias: facts near the very start or very end of the window are recalled more reliably than facts buried in the middle, so a constraint stated at turn 1 and never touched again drifts toward the least-reliable position as new turns get appended after it.

Distractor buildup is the one people underestimate: near-duplicate tool outputs (three similar diffs, five similar test-failure logs) interfere with retrieval more than an equal amount of random noise would, because the model has to discriminate between plausible-looking near-misses. Chroma's research adds a counterintuitive fourth finding: well-structured, coherent input can cost accuracy at scale more than shuffled input does — a clean, readable transcript is not automatically a safe one. None of these four require a wrong fact anywhere in the window; they are properties of length and structure, not correctness.

Running example. Subject: repo-agent, an autonomous coding agent. Input at turn 1: task RENAME-118 — "Rename calculate_tax to compute_tax everywhere; do not touch anything under legacy/." Success: every non-legacy call site renamed, full test suite green, legacy/ untouched. Failure: at turn 34, the agent renames the function inside legacy/billing_v1.py, and the pinned legacy-compatibility test fails.

Turns 2–20 accumulate real, useful work: diffs across a dozen files, test-runner output, a short replanning note after one file had two call sites instead of one. Turns 21–33 add a second replanning pass after the agent discovers a circular import, plus more test logs. None of these turns is wrong on its own — each is a faithful record of real progress. But the turn-1 constraint was never re-stated, never pinned, and by turn 34 it is deep in the middle of a transcript dominated by recent, structurally similar diff-and-log pairs. The model does not "decide" to violate the constraint; it simply never surfaces in the set of tokens that most strongly shape the next action.

You cannot eyeball a 40-turn transcript and know whether a constraint survived — you have to test it. The pattern from long-context evaluation research (needle-in-a-haystack style checks) applies directly to agent sessions: periodically ask "is fact X still recoverable from this window?" and log the answer, rather than assuming length alone is safe because the model's advertised window is bigger than your transcript.

The two snippets below simulate the same shape of failure as RENAME-118 with plain string budgets, so you can reproduce and measure it without an API call. naiveWindow keeps only the most recent turns that fit a character budget — no compaction, no pinning. pinnedWindow reserves budget for a fixed set of must-survive facts and fills the rest with recent turns. Run both against the same 40-turn history and compare.

Pick the fix that matches what actually failed, the same way you'd pick between prompt, context, and harness fixes in Prompt vs Context vs Harness. If constraints and decisions are getting buried, compaction — periodically summarizing the transcript into a short decision log and re-pinning must-keep facts — is the cheapest fix and the one pinnedWindow demonstrates above. If a subtask is long and mostly self-contained (a refactor pass, a research dive), sub-agent isolation gives it a fresh, short window and returns only the result to the parent, which never accumulates that subtask's noise at all.

When a fact needs to outlive the session entirely — not just one long conversation — move it out of the transcript into a persistent scratchpad or memory store that gets re-read every turn instead of re-derived from history; see AI Agent Memory for the storage patterns. And when the risk of a silent violation is high enough that you want a human in the loop before it compounds, cap the turn budget and checkpoint: stop the agent, summarize, and get a review before it resumes — the same checkpoint discipline covered in Human-in-the-Loop AI Agent Patterns. None of these are mutually exclusive; production agent harnesses typically run compaction on a schedule and still cap total turns as a backstop.