06-reference

every efficient agent token caching

2026-10-07·reference·source: Every·by Laura Entis

Building a More Efficient Agent -- Laura Entis (Every)

Every's Context Window issue combines three strands: (1) a podcast recap of Dan Shipper's conversation with Willie Williams on why Every retired personal agent-pets in favor of one shared company agent, now shipped as the Every Agent; (2) an engineering case study on how staff engineer Paridhi Agarwal cut Every Agent token costs on 11 frequent tasks by over 80 percent, including a prompt-caching bug nobody knew they had; and (3) the issue's recurring sections -- this week's model picks across the team, seven new Thesis Statements, and a Jagged Frontier essay on AI-first engineering management.

The core case study: an early attempt to route all Every Agent tasks through a cheap coordinator model (Sonnet) that escalated complex jobs to Opus backfired -- quality and cost both got worse, because many requests ended up processed by both models. What actually worked: (a) upgrading Opus 5 to Opus 5.5, which improved response quality and cut token costs within hours of release; (b) rewriting tool responses (GitHub, Linear) to return shorter, labeled results and loading full tool descriptions only on demand (~10% reduction); and (c) discovering that Claude auto-injects each user's Slack ID into memory-folder names inside the system prompt, which silently defeated prompt-cache reuse across users with zero error signal -- "the team assumed caching was working. It wasn't." Fixing it (generic folder names, Slack IDs kept out of the visible prompt) cut costs by 74 percent in Paridhi's best-case test, stacked on the other two fixes.

Why this is in the vault

Direct, measured engineering lessons (a coordinator-routing failure mode, a specific prompt-caching bug pattern, a tool-response-size fix) for any shop running Claude-based agents in production, plus a continuation of the personal-vs-shared-agent thesis this vault is already tracking.

Mapping against Ray Data Co

Strong mapping. The most concrete transferable finding is the caching bug: Claude silently injects user-specific identifiers (Slack IDs via memory-folder names) into the system prompt, which breaks prompt-cache reuse with no visible error -- just a quietly inflated bill. That is a directly checkable risk for RDCO's own Claude-based agents, starting with the always-on channels-agent ([[project_channels_agent_setup]]): does its system prompt or memory-folder naming vary per-conversation or per-chat-id in a way that defeats caching the same way? A 70%+ silent token leak wouldn't show up in cron logs or output quality, only in the bill -- worth a direct audit rather than assuming caching "just works." Second, the coordinator-routing failure (a cheap model told to self-triage and escalate hard work made both quality and cost worse, because requests got double-processed) is a sharp cautionary data point against any naive "router decides, bigger model executes" pattern, directly relevant to how RDCO already pairs model and effort explicitly across sub-agent dispatch rather than trusting a model to judge its own limitations (feedback_delegation_model_effort_pairing). Third, Willie Williams's personal-agent rule of thumb from the podcast half -- a transactional task gets a disposable Codex-style thread, a relationship-building task gets its own named persona -- is a usable design heuristic for Ray's own mascot/persona work (feedback_ray_mascot_instantiation_pattern), and it reinforces, with a new mechanism (prompt caching + tool-response cost, not just adoption virality), the shared-vs-personal-agent thesis the 10-06 GA-launch note already banked for RDCO's agent-deployer positioning.

Curation section

Related

⚠️ Sponsorship

House self-promotion throughout: the piece exists to sell subscriptions bundling access to the Every Agent (pass-through token pricing, "no markup" framing repeated twice in the issue), and the footer cross-promotes Every's own product suite (Sparkle, Cora, Spiral, Monologue). The token-efficiency case study itself reads as genuine engineering detail rather than puffery, but the "cut costs 80 percent" framing is still a vendor demonstrating its own product is worth the subscription price, not an independent benchmark.