Accumulated agentic context: no clean degradation curve is published, and the nearest evidence says the damage is about what is in the history, not how many tokens it holds
The question
Verbatim: "Is there any published measurement of a degradation curve for ACCUMULATED multi-turn agentic context (tool results, file reads, subagent handbacks piling up over dozens of turns), and does the single-pass long-input evidence transfer to it at all?"
Context: follow-up #1 from [[2026-09-27-context-rot-mechanism]]. That brief proposed amending AGENTS.md hard rule #4 with a cumulative context budget as the primary trigger, and flagged that the choice rested on extrapolation from single-pass benchmarks.
Short answer. Within tonight's search cap I found no published curve of task success against accumulated context tokens for an agent working its own trajectory. What exists is four kinds of adjacent evidence: one controlled ablation of unbounded versus bounded accumulation (it finds a cost difference and no reliable quality difference), one multi-turn study (degradation is large, starts at turn two, and is not a length effect), one web-agent benchmark with injected history (steep decline, but the history is synthetic distractor), and vendor claims without curves. The single-pass evidence transfers in part. The parts that transfer are the weaker ones.
What we already know (from the vault)
- [[2026-09-27-context-rot-mechanism]] ranks the single-pass mechanisms as length (floor), retrieval difficulty (amplifier), position (modulator), with measured onset at 16K-32K tokens. It states that "the penalty compounds with accumulated length" and that "N short contexts genuinely beat one long one." Neither sentence has a measurement behind it for agentic context. Its own follow-up #1 says so.
- [[2026-04-15-thariq-claude-code-session-management-1m-context]] is first-party operating guidance, not data. Its strongest habit is event-based, not size-based: rewind to before a failed attempt instead of correcting on top of it. It also notes compaction happens at the model's "least intelligent" moment.
- [[2026-09-21-indy-dev-dan-self-compact-pi-agent-zero-hype-devlog]] gives practitioner thresholds (soft 225k tokens, forced 270k). Those are operating settings, not measured degradation points.
- [[2026-05-12-every-fallacy-16-hour-agent]] carries the METR time-horizon reading: success falls as task length grows (16 hours at 50% reliability, about 3 hours at 80%). That curve is against task difficulty in human-time, so it confounds context growth with harder tasks and compounding per-step error. It is not a context curve.
- A qmd sweep found no vault note covering multi-turn degradation studies, observation masking, or compaction ablations. [[context-rot]] has no agentic-accumulation evidence.
What the web says
- The one controlled accumulation ablation finds a cost effect, not a quality effect. "The Complexity Trap" (Lindenbauer et al., JetBrains Research and TUM, arXiv:2508.21433, listed on the NeurIPS 2025 program) runs SWE-agent on SWE-bench Verified with a 250-turn limit under three regimes: Raw Agent (context grows unchecked), Observation Masking (tool outputs older than 10 turns replaced by a placeholder), and LLM-Summary. Read directly from the paper's Table 1: Raw versus Masking solve rates were 17.0 vs 15.0 (Qwen3-32B), 23.0 vs 24.6 (Qwen3-32B thinking), 53.4 vs 54.8 (Qwen3-Coder 480B), 32.8 vs 35.6 (Gemini 2.5 Flash), and 40.4 vs 36.4 (Gemini 2.5 Flash thinking). With 95% intervals of roughly plus or minus 4 points, only the last difference is significant, and it favors the unbounded agent. Cost dropped about 50% under management in most setups. The authors write that "more context becomes a liability rather than an asset," but their own table supports that for cost only. (arxiv.org/abs/2508.21433)
- Same paper, three facts about the shape of agentic context. Tool observations are about 84% of the tokens in an average SWE-agent turn. Raw context reaches about 30K tokens by turn 21, and mean trajectories ran 44-50 turns in the setup where the paper reports them. And "the most recent context is often sufficient": a 10-turn window of observations held solve rate. The paper also documents a side effect of compaction, "trajectory elongation": summarized runs went 13-15% longer because summaries "mask failure signals that would otherwise prompt earlier termination." The masking window is scaffold-specific; the SWE-agent setting degraded "drastically" on OpenHands until retuned.
- Multi-turn degradation is large, immediate, and not a token-count effect. "LLMs Get Lost In Multi-Turn Conversation" (Laban et al., Microsoft Research and Salesforce, arXiv:2505.06120) simulates 200,000+ conversations across 15 models and six generation tasks. The same instruction delivered in pieces across turns scores about 65% against 90% delivered at once, a 39% average relative drop. The control that matters here is CONCAT: all the pieces in one turn recover 95.1% of full performance, so the information and the tokens are not the problem; the turn structure is. The drop appears "even in two-turn conversations." It decomposes into a small aptitude loss (about 16%) and a 112% rise in unreliability. Named causes: models "make assumptions in early turns and prematurely attempt to generate final solutions, on which they overly rely," plus answer bloat and a loss-of-middle-turns pattern. Restating all prior information each turn (the "snowball" condition) recovers only 15-20% of the gap. (arxiv.org/abs/2505.06120)
- Injected irrelevant history hurts agents badly. "Evaluating Long-Context Reasoning in LLM-Based WebAgents" (arXiv:2512.04307) is the closest thing to a curve in agentic format: success falls from roughly 40-50% to under 10% as the task is embedded in longer, noisier history. Search summary only; primary not read tonight. The history is injected irrelevant trajectory, so this measures distractor load in agent format, not an agent's own on-task accumulation.
- Several 2026 preprints attribute long-horizon failure to things other than visible context length. All four are search summary only, primary not read, so treat the characterizations as leads. arXiv:2604.11978 reports failures shifting to planning and memory errors as horizon grows, with per-step error compounding. arXiv:2603.24631 finds one SWE-bench failure type ("coherence collapse") rising with trajectory length and another not. arXiv:2603.06870 describes early wrong state propagating without recovery. arXiv:2607.27283 argues long-horizon results need matched short-task baselines and explicitly separates "trajectory-induced degradation" from "context rot." If that last paper reports what its summary implies, it is the controlled measurement this question asks for, and it is the first thing to read next.
- Vendor evidence is directional, with no curves. Anthropic's context-management announcement (September 2025) reported an internal agentic-search eval where context editing plus the memory tool beat baseline by 39%, and a 100-turn web-search eval where context editing cut token use 84% and let runs finish that otherwise exhausted the window. Figures recalled from the announcement, not re-fetched tonight; internal evals, no methodology published. Chroma's LongMemEval condition (cited in the prior brief: about 300 focused tokens versus about 113K of conversation history) is conversation-shaped but is replayed to the model in one pass, so it is single-pass evidence wearing multi-turn clothing.
Convergences and contradictions
- Convergence: vault and web agree that long sessions fail more and that isolating work in fresh contexts helps. Every source that proposes a fix proposes some form of bounding or restarting.
- Contradiction with the prior brief's central assumption: [[2026-09-27-context-rot-mechanism]] assumes the single-pass length penalty compounds across accumulated context. The only controlled test of unbounded versus bounded accumulation shows no reliable solve-rate penalty across five model setups up to 250 turns, and one significant result in the opposite direction. The accumulated-context failure the vault worries about is real, but the measured driver is not cumulative tokens.
- Contradiction inside the literature: the WebAgents benchmark shows steep decline with longer history while the Complexity Trap shows none. They are reconcilable: one fills history with off-task material, the other with the agent's own on-task observations. That reconciliation is Ray's inference, not a published comparison, and it lines up with the prior brief's finding that topical-but-irrelevant content is the malignant case.
Synthesis for RDCO
Is there a published curve? Not a clean one. No source read tonight plots success against accumulated tokens for an agent on its own trajectory with task difficulty held fixed. The benchmarks that report success against horizon (METR-style) confound length with difficulty and error compounding. The benchmark that reports success against history length (WebAgents) uses injected distractor. The one study that holds the task fixed and varies only whether context accumulates finds the quality effect is inside the noise. So the honest status is: degradation of long agent sessions is well attested, its curve against cumulative tokens is unmeasured, and the best available control suggests cumulative tokens are the wrong x-axis. Limits on that statement: the controlled study used Qwen and Gemini Flash on coding tasks whose trajectories averaged about 50 turns. Ray runs Claude-family models in sessions of hundreds of thousands of tokens over a day or more, with mixed tasks in one context. That regime is outside anything measured.
Does the single-pass evidence transfer? Partly, and unevenly. What plausibly transfers: (1) distractor interference, supported by the WebAgents result, since off-task history in agent format behaves like a high-similarity haystack; (2) position effects, supported weakly by the loss-of-middle-turns pattern in Laban et al., the conversational analogue of lost-in-the-middle; (3) the length floor in principle, since arXiv:2510.05381 shows it is content-independent, though at the trajectory sizes tested in agents it has not been large enough to show in solve rates. What does not transfer, because single-pass benchmarks cannot contain it: (1) self-generated content. The biggest multi-turn effect comes from the model anchoring on its own earlier assumptions and failed attempts, and it shows up at turn two with trivial token counts. No haystack benchmark has the model's own prior mistakes in the haystack. (2) Redundancy. In single-pass work extra tokens hurt; in multi-turn work restating earlier material helps (snowball recovers 15-20%), which matches the recitation fix in arXiv:2510.05381. Redundant re-reads in a transcript may be mildly protective, not harmful. (3) Recency structure. An agent's needed information is mostly in the last few turns and the rest is re-fetchable from the environment, which is why a 10-turn observation window loses nothing. A single-pass benchmark puts the needle anywhere and offers no re-read. (4) Behavioral side effects of the fix. Compaction changes what the agent does next (trajectory elongation, lost failure signals). Single-pass work has no analogue.
What this implies for the hard rule #4 amendment (recommendation for the founder, not a decision). The September brief proposed a cumulative context budget, 25-30% of the window, as the primary trigger. On tonight's evidence that trigger is defensible as a cost and headroom control and unsupported as a quality control. A turn-count trigger is also unsupported: degradation in Laban et al. is not a function of turn count beyond "more than one," and the masking window that works is scaffold-specific. What the evidence supports instead is content- and event-based triggers, and the labels below are Ray's inference from the cited results, not published prescriptions:
- Failed-attempt trigger. When a wrong approach or wrong assumption is sitting in context, rewind or restart from a clean handoff instead of correcting on top. This is the measured dominant multi-turn failure (Laban et al.) and matches Thariq's rewind habit.
- Task-boundary trigger. New task, fresh context. Off-task history is the condition under which agent benchmarks show steep decline.
- Adjacent-distractor trigger. Keep the September brief's point (3): delegate reads that are topically near the live task but are not the task.
- Staleness over size for tool output. Old tool results are safe to drop after a recency window, and dropping beats summarizing on cost with no measured quality loss. That argues for clearing stale observations more than for preventing reads.
- Cumulative tokens demoted to a secondary budget. Keep a budget for cost, cache, and compaction-avoidance reasons, and say so, without claiming a measured quality cliff at 25-30%.
The practical consequence is that the amendment would shift from "delegate when the session is getting big" toward "delegate or restart when the session contains the wrong things," with size kept as a cost guard. This also slightly raises the bar for mandatory delegation: if on-task accumulated tool output is close to harmless in the measured range, the unpriced extraction-fidelity cost of handbacks (covered tonight in [[2026-11-12-extraction-faithfulness-vs-source-length]]) weighs more heavily in the trade.
Why this is in the vault
It settles the open evidentiary question under the pending AGENTS.md hard rule #4 amendment from [[2026-09-27-context-rot-mechanism]]: the proposed primary trigger (cumulative context budget) has no quality measurement behind it, and the founder should see the alternative trigger set before greenlighting any wording.
Open follow-ups
- Does arXiv:2607.27283 ("Benchmarking the Residual") actually report a matched-baseline measurement that isolates visible-context growth from trajectory effects in agents, and if so what is the residual's size and onset for frontier models? This is the one lead that could overturn tonight's "no clean curve" finding.
- Is there any measurement of on-task context accumulation in the 200K-1M token range on Claude-family models (the regime Ray actually runs in), including Anthropic's own compaction or context-editing evals with published methodology?
- Is the "poisoned context" effect measured in agentic settings: how much does a failed attempt left in the transcript lower later success compared with a rewind to a clean state, holding tokens equal?
Related
- [[2026-09-27-context-rot-mechanism]] - parent brief; its cumulative-budget trigger is what this brief tests
- [[2026-11-12-extraction-faithfulness-vs-source-length]] - sibling brief tonight on the handback-fidelity cost side of the same rule
- [[context-rot]] - concept page; has no agentic-accumulation evidence yet
- [[2026-04-15-thariq-claude-code-session-management-1m-context]] - rewind-over-correction and compaction guidance; matches the failed-attempt trigger
- [[2026-09-21-indy-dev-dan-self-compact-pi-agent-zero-hype-devlog]] - practitioner compaction thresholds
- [[2026-05-12-every-fallacy-16-hour-agent]] - METR time-horizon reading; success versus task length, confounded with difficulty
- [[2026-07-24-thariq-context-engineering-claude-5-rules]] - argues against mechanical over-constraint
Sources
Vault
- [[2026-09-27-context-rot-mechanism]]
- [[context-rot]]
- [[2026-04-15-thariq-claude-code-session-management-1m-context]]
- [[2026-09-21-indy-dev-dan-self-compact-pi-agent-zero-hype-devlog]]
- [[2026-05-12-every-fallacy-16-hour-agent]]
Web, read in primary
- Lindenbauer, Slinko, Felder, Bogomolov, Zharov, "The Complexity Trap: Simple Observation Masking Is as Efficient as LLM Summarization for Agent Context Management," arXiv:2508.21433v3 - https://arxiv.org/abs/2508.21433 (Table 1 and sections 4-6 read from the PDF)
- Laban, Hayashi, Zhou, Neville, "LLMs Get Lost In Multi-Turn Conversation," arXiv:2505.06120 - https://arxiv.org/abs/2505.06120 (first 100K characters of the HTML version read; appendix not read)
Web, search summary only (not read in primary; verify before citing onward)
- "Evaluating Long-Context Reasoning in LLM-Based WebAgents," arXiv:2512.04307 - https://arxiv.org/abs/2512.04307
- "The Long-Horizon Task Mirage? Diagnosing Where and Why Agentic Systems Break," arXiv:2604.11978 - https://arxiv.org/abs/2604.11978
- "Coherence Collapse: Diagnosing Why Code Agents Fail After Reaching the Right Code," arXiv:2603.24631 - https://arxiv.org/abs/2603.24631
- "Benchmarking the Residual: What Long-Horizon Evaluations Add Beyond Matched Short-Task Performance," arXiv:2607.27283 - https://arxiv.org/abs/2607.27283
- arXiv:2603.06870 (no-recovery bottleneck), cited via a survey, arXiv:2607.05775
Recalled, not re-fetched tonight
- Anthropic, context management announcement (context editing and memory tool), September 2025 - https://www.anthropic.com/news/context-management
- Chroma, "Context Rot" technical report, LongMemEval condition - https://www.trychroma.com/research/context-rot (as cited in the parent brief)