06-reference/research

accumulated agentic context degradation curve

2026-11-12·research-brief·source: deep-research·by Ray Data Co (deep-research synthesis)

Accumulated agentic context: no clean degradation curve is published, and the nearest evidence says the damage is about what is in the history, not how many tokens it holds

The question

Verbatim: "Is there any published measurement of a degradation curve for ACCUMULATED multi-turn agentic context (tool results, file reads, subagent handbacks piling up over dozens of turns), and does the single-pass long-input evidence transfer to it at all?"

Context: follow-up #1 from [[2026-09-27-context-rot-mechanism]]. That brief proposed amending AGENTS.md hard rule #4 with a cumulative context budget as the primary trigger, and flagged that the choice rested on extrapolation from single-pass benchmarks.

Short answer. Within tonight's search cap I found no published curve of task success against accumulated context tokens for an agent working its own trajectory. What exists is four kinds of adjacent evidence: one controlled ablation of unbounded versus bounded accumulation (it finds a cost difference and no reliable quality difference), one multi-turn study (degradation is large, starts at turn two, and is not a length effect), one web-agent benchmark with injected history (steep decline, but the history is synthetic distractor), and vendor claims without curves. The single-pass evidence transfers in part. The parts that transfer are the weaker ones.

What we already know (from the vault)

What the web says

Convergences and contradictions

Synthesis for RDCO

Is there a published curve? Not a clean one. No source read tonight plots success against accumulated tokens for an agent on its own trajectory with task difficulty held fixed. The benchmarks that report success against horizon (METR-style) confound length with difficulty and error compounding. The benchmark that reports success against history length (WebAgents) uses injected distractor. The one study that holds the task fixed and varies only whether context accumulates finds the quality effect is inside the noise. So the honest status is: degradation of long agent sessions is well attested, its curve against cumulative tokens is unmeasured, and the best available control suggests cumulative tokens are the wrong x-axis. Limits on that statement: the controlled study used Qwen and Gemini Flash on coding tasks whose trajectories averaged about 50 turns. Ray runs Claude-family models in sessions of hundreds of thousands of tokens over a day or more, with mixed tasks in one context. That regime is outside anything measured.

Does the single-pass evidence transfer? Partly, and unevenly. What plausibly transfers: (1) distractor interference, supported by the WebAgents result, since off-task history in agent format behaves like a high-similarity haystack; (2) position effects, supported weakly by the loss-of-middle-turns pattern in Laban et al., the conversational analogue of lost-in-the-middle; (3) the length floor in principle, since arXiv:2510.05381 shows it is content-independent, though at the trajectory sizes tested in agents it has not been large enough to show in solve rates. What does not transfer, because single-pass benchmarks cannot contain it: (1) self-generated content. The biggest multi-turn effect comes from the model anchoring on its own earlier assumptions and failed attempts, and it shows up at turn two with trivial token counts. No haystack benchmark has the model's own prior mistakes in the haystack. (2) Redundancy. In single-pass work extra tokens hurt; in multi-turn work restating earlier material helps (snowball recovers 15-20%), which matches the recitation fix in arXiv:2510.05381. Redundant re-reads in a transcript may be mildly protective, not harmful. (3) Recency structure. An agent's needed information is mostly in the last few turns and the rest is re-fetchable from the environment, which is why a 10-turn observation window loses nothing. A single-pass benchmark puts the needle anywhere and offers no re-read. (4) Behavioral side effects of the fix. Compaction changes what the agent does next (trajectory elongation, lost failure signals). Single-pass work has no analogue.

What this implies for the hard rule #4 amendment (recommendation for the founder, not a decision). The September brief proposed a cumulative context budget, 25-30% of the window, as the primary trigger. On tonight's evidence that trigger is defensible as a cost and headroom control and unsupported as a quality control. A turn-count trigger is also unsupported: degradation in Laban et al. is not a function of turn count beyond "more than one," and the masking window that works is scaffold-specific. What the evidence supports instead is content- and event-based triggers, and the labels below are Ray's inference from the cited results, not published prescriptions:

  1. Failed-attempt trigger. When a wrong approach or wrong assumption is sitting in context, rewind or restart from a clean handoff instead of correcting on top. This is the measured dominant multi-turn failure (Laban et al.) and matches Thariq's rewind habit.
  2. Task-boundary trigger. New task, fresh context. Off-task history is the condition under which agent benchmarks show steep decline.
  3. Adjacent-distractor trigger. Keep the September brief's point (3): delegate reads that are topically near the live task but are not the task.
  4. Staleness over size for tool output. Old tool results are safe to drop after a recency window, and dropping beats summarizing on cost with no measured quality loss. That argues for clearing stale observations more than for preventing reads.
  5. Cumulative tokens demoted to a secondary budget. Keep a budget for cost, cache, and compaction-avoidance reasons, and say so, without claiming a measured quality cliff at 25-30%.

The practical consequence is that the amendment would shift from "delegate when the session is getting big" toward "delegate or restart when the session contains the wrong things," with size kept as a cost guard. This also slightly raises the bar for mandatory delegation: if on-task accumulated tool output is close to harmless in the measured range, the unpriced extraction-fidelity cost of handbacks (covered tonight in [[2026-11-12-extraction-faithfulness-vs-source-length]]) weighs more heavily in the trade.

Why this is in the vault

It settles the open evidentiary question under the pending AGENTS.md hard rule #4 amendment from [[2026-09-27-context-rot-mechanism]]: the proposed primary trigger (cumulative context budget) has no quality measurement behind it, and the founder should see the alternative trigger set before greenlighting any wording.

Open follow-ups

Related

Sources

Vault

Web, read in primary

Web, search summary only (not read in primary; verify before citing onward)

Recalled, not re-fetched tonight