06-reference

every context engineering when to start over

2026-10-16·reference·source: Every·by Mike Taylor
context-engineeringcontext-rotai-chat-hygieneagent-architecturememory-design

Context Engineering Is Knowing When to Start Over — Mike Taylor (Every)

Why this is in the vault

A named, concrete taxonomy of seven ways "context rot" actually degrades a chat session (distraction, confusion, neglect, drift, clash, poisoning, underspecification), each with a watch-for signal and a fix — not just the one-line warning RDCO's own AGENTS.md already carries.

Mike Taylor, Every's head of evals and author of the forthcoming O'Reilly book Context Engineering with DSPy, keeps both ChatGPT's and Claude's memory off and starts a fresh thread for every new task, even mid-project. He adapts seven failure modes from Drew Breunig's "How Long Contexts Fail" framework for everyday chat users: (1) distraction — early examples become implicit rules the model won't deviate from (a Steve Jobs product-naming demo goes from 21 percent to 100 percent "i-" names once a handful of examples are added, killing creative variety); (2) confusion — irrelevant details get treated as requirements, citing colleague Katie Parrott's style guide bloating from 759 to 3,855 words and one essay taking 127 drafts before the guide was cut back to 513 words; (3) neglect — Stanford's "Lost in the Middle" research shows a document at position six of nine gets ignored in favor of the first and last; (4) drift — a "follow PEP-8 strictly" instruction quietly stops being honored 50 messages later; (5) clash — Microsoft/Salesforce research found breaking one prompt into sequential messages produced 39 percent worse results, because the model locks in an early assumption that then collides with later clarification; (6) poisoning — a fabricated fact or Gemini's real "TEA item" Pokémon-playing dead end persists because the model treats its own prior output as settled, and arguing with it in-thread doesn't remove it — editing the earlier message to erase the mistake does; (7) underspecification — missing context forces the model to default to generic internet-average advice, fixable by asking "what would have eliminated several steps or mistakes" before closing a thread and carrying the answer into the next one.

His standing fix: keep durable context in local files/folders the agent re-reads on demand, rather than hoping a long thread or persistent memory preserves it faithfully.

Mapping against Ray Data Co

This directly sharpens AGENTS.md hard rule 4 ("route long artifacts through subagents... context rot"), which cites Thariq's [[2026-04-15-thariq-claude-code-session-management-1m-context]] note for the same term but only as a one-line soft-cost warning. Taylor's taxonomy gives that warning mechanism: rule 4 is really a defense against distraction and neglect (a 100KB email body read raw would both anchor the model on irrelevant boilerplate and bury the one fact that matters in the middle), while the PR-only workflow and fresh-eyes critic fleet (verify-vault-write, verify-dispatch, etc.) are independent defenses against poisoning and drift — catching a compounding error or a quietly-abandoned instruction before it reaches the founder.

The sharper finding is a real tension, not a validation: Taylor's core practice is keeping chat memory off and distrusting anything that persists across sessions by default, while this entire operator runs on MEMORY.md, a ~90-entry file of exactly the kind of cross-session persistent memory he argues degrades output. RDCO's mitigation — the precedence chain's "memory conflict → newer wins, and log it" rule, and the age-gated, append-only discipline — is a real answer to his poisoning and drift failure modes, but it's a bet that governance can substitute for his blunter fix (don't let it persist at all). Worth flagging rather than resolving here.

His poisoning fix — edit the earlier message to erase the mistake rather than correct it in-thread — is a concrete technique RDCO doesn't currently use anywhere in its skill/session design; sessions here run forward-only (compaction, working-context.md handoffs), never backward-edited. And his underspecification fix (ask "what would have eliminated mistakes" before closing a thread, carry the answer forward) is close kin to the implementation-notes sub-agent pattern already in use, but applied at the level of an individual chat rather than a sub-agent dispatch.

Related

⚠️ Sponsorship

House self-promo only: standard Every footer pitching the "All Access" bundle (Sparkle, Cora, Spiral, Monologue) plus a subscribe CTA. No third-party paid sponsor in this issue. Bias implication is low — the essay's argument and examples (Taylor's own workshops, Parrott's style guide, Daigle's AI-twin interview, Gemini's Pokémon run, Taylor's agency-pricing anecdote) are personal and independently sourced, unrelated to the footer's product pitch.