06-reference

every opus5 compound engineering breakage

2026-07-25·reference·source: Every (X posts + EveryInc/compound-engineering-plugin PRs)·by Kieran Klaassen and Dan Shipper (Every)

Opus 5 broke Every's Compound Engineering plugin — the phantom hand-off, and whether RDCO shares it

Why this is in the vault

The founder switched RDCO's daily driver to Opus 5 on 2026-07-24 and rewrote the seven daily-firing skills on 2026-07-25; this note is the primary-source account of the one documented case of a model upgrade breaking a packaged skill system, and it names a specific mechanism precise enough to grep our own skills for. The mechanism turns out to be narrower and more interesting than "new model, new prompt style," and it interacts directly with the fact that RDCO's cron fleet runs headless.

Source note. The Every article https://every.to/vibe-check/opus-5 (Shipper + Parrott, 2026-07-24, sponsored by Scribe Optimize) is hard-paywalled — the email is a teaser and is filed separately. Everything below comes from sources that could actually be read: Dan Shipper's public X post (2080700057892815114), Kieran Klaassen's public X post (2080712817926443486) and his follow-up replies, and the two public pull requests Klaassen opened the same day — PR 1250 and PR 1252. The PRs are the good stuff: Klaassen writes the diagnosis down in engineering prose rather than newsletter prose.

What broke, and what compound engineering is

Compound engineering is Every's name for running software development as a multi-stage agent loop — ideate, plan, work, review, polish — where each stage's output is written down so the next stage and the next project inherit it. The system compounds because the agent's accumulated instructions, learnings, and review criteria are treated as the durable asset, not the code. It ships as EveryInc/compound-engineering-plugin, a public Claude Code plugin with dozens of skills and subagents, driven by an autonomous /lfg entry point.

That plugin is what Opus 5 broke. Shipper: the model "didn't play well with our existing skills and plugins like Compound Engineering," and "It breaks backward compatibility." The observable symptom, three ways: it argued with instructions, it stopped before the work was finished, and it fought the scaffolding. Klaassen, who maintains the plugin, was blunter and more specific: "it kept returning control to the user. Even though it's an autonomous flow."

The mechanism: prose describing a runtime that no longer exists

This is the part worth internalizing, because it is not "models changed" and it is not "prompts were too long."

PR 1250 is titled, verbatim, fix(skills): stop headless runs halting at phase boundaries in Opus 5. Klaassen's diagnosis: the plugin's skills were full of sentences addressed to an orchestrator that isn't there. Language like return control to the caller, hand back to LFG, the caller owns apply/fix policy was written when phase transitions were imagined as subprocess dispatches. They aren't. A Skill invocation loads instructions inline — same model, same conversation, same turn. So that prose is, in his words, "addressed to a caller that does not exist."

The kill shot is the interaction with headless execution: under claude -p, an assistant message with no tool call ends the session. So "return a summary to the caller" has exactly one expressible form — end the turn. A model that follows that instruction faithfully terminates the run at the phase seam and looks like it finished. PR 1250 fixes eight distinct halt sites and reduces them to a single root cause: "All eight reduce to one error: prose written as if a second party were waiting."

So the causal chain is: latent fiction in the prompt + a model that complies with it more literally + headless execution where compliance is indistinguishable from completion. Note which hypotheses this rules out. It is not prompt-cache or context-window behavior. It is not tool-definition drift — the PR changes zero allowed-tools declarations. It is not simply over-constraint, though over-constraint is adjacent: the fix is not shorter prompts but removing an imaginary call stack. Klaassen's own instruction on that point: "Prefer removing the fake call stack over rewording it," because rewording preserves the structure that caused the problem.

Two honest complications. First, this is a latent plugin bug that a model change exposed, not purely a model regression — repo issue #134, filed 2026-01-29, reports the identical symptom six months earlier with no model attribution. Second, Every's own measurement work found the halt behavior in four of eight models tested (struggling models died at one hand-off boundary in 17–38% of runs, healthy ones 81–100% clean), and reports model identity as roughly 3x the effect of thinking effort. It is a class of behavior at a fictional seam, not one release's defect. Klaassen's public read of himself is the same: "still thinking it's a skill issue (me and my skills/harness)."

What Every is actually doing about it

Not pinning to an older model, and not waiting for an Anthropic fix. Four things:

  1. Deleting, not rewriting. Shipper: "Then we deleted our existing skills and started from scratch," and without the old workflows the model "got dramatically better." Klaassen's advice to others: "Just remember to let go of you skills and big mega prompts."
  2. Rewriting the plugin. PR 1250 — 195 files, +2,429/−10,206, a 21% corpus reduction. The substitutions are mechanical: return X to the caller becomes record X; return control is deleted or becomes continue to the next stage; GATE: STOP becomes CHECKPOINT — verify, then continue; CRITICAL: You MUST execute every step IN ORDER becomes Run the steps below in order. Status as of writing: open and still a draft, zero reviews, not merged, not in any release.
  3. Shipping a migration tool. PR 1252 is merged (in eleven minutes) and adds /ce-retune, a skill whose whole job is retuning an existing corpus for a newer model. Its thesis: a corpus that degrades on a new model is a measurement problem before it is a writing problem, and word reduction is an explicit non-goal. It carries a ten-class halt taxonomy with greps for each, and a set of screens for stops that must survive.
  4. Turning thinking effort down. Both authors independently landed on medium or low effort for Opus 5. More thinking time correlated with more of the annoying behaviors.

The two most transferable rules from /ce-retune, both aimed at the failure the cure itself causes: "keep the requirement, drop the seam" — a dispatch mandate carries two payloads, the fictional hand-off and the requirement that the phase run at all, and deleting the first deletes the second. And verify a phase ran by artifact existence, never by the final message, because when Every over-cut, a run skipped two whole phases, shipped inline, and reported success with no plan file. Their note on it: "The transcript looked clean." Also worth keeping: "Removing a 'you must' does not remove the decision" — it hands the decision to the model.

Calibration check on the founder's framing

His summary was "the Every team say that they have a love hate with opus 5 and that it has broken their compound-engineering packaged skills/plugin." That is accurate, and closer to verbatim than he probably realized. Klaassen's post literally reads: "it broke Compound Engineering, but I also like it. Love it and hate it." No correction needed on the claim.

Two refinements rather than corrections. (a) The load-bearing detail is how it broke — premature turn-end at phase seams caused by hand-off prose, not general instruction-following decay — and that detail is what makes it testable against our own skills. (b) The remediation is stronger than "they're dealing with it": they deleted the scaffolding first and are now rebuilding it measurement-first, and the rewrite is still an unmerged draft. Anyone running the released plugin today has neither fix.

Mapping against Ray Data Co

The precondition is present and the trigger is not — yet. RDCO's cron fleet executes skills through ~/.claude/scripts/cron-job-runner.sh, which runs claude -p "$FULL_PROMPT" --model "$MODEL" --output-format json. That is the exact headless mode where a no-tool-call message ends the session, so a phantom hand-off in an RDCO skill would produce a silent early exit with exit=0 and a plausible summary. It would never appear in ~/.claude/state/cron-failures.log, which only records non-zero exits and is_error — the log is structurally blind to this failure mode. What is not present is Opus 5: every cron run since Jul 23 used claude-sonnet-4-6, claude-sonnet-5, claude-haiku-4-5, or claude-opus-4-8. Opus 5 is the founder's interactive driver only. So the seven rewritten skills are not currently executed by the model that provoked Every's breakage.

Grepping our corpus against Every's class-2 taxonomy (return control|hand back|the caller|caller owns) comes back nearly clean, and today's rewrite is why. The v1 backups carried the two clearest instances in the daily fleet — deep-research.SKILL.md.v1:146 opened an output block with the bare envelope label Return:, and open-threads-check.SKILL.md.v1:79 began "Return one line per channel checked." Both are gone from the live ~/.claude/skills/deep-research/SKILL.md and ~/.claude/skills/open-threads-check/SKILL.md. Every's class-5 diagnosis of exactly that pattern is that the verb, and only the verb, is the defect — the field list is fine. The rewrite removed the verb. Same story for class 1: deep-research.SKILL.md.v1:150 ("Wait for all sub-agents") and process-newsletter.SKILL.md.v1:54 ("wait for all sub-agents") are both gone.

So today's rewrite moved us toward safety on the mechanism Every actually hit — but it also walked straight into Every's class 10, the over-correction. Mandate density across the seven skills went from 17 occurrences of MANDATORY/CRITICAL/MUST/ALWAYS/REQUIRED to zero: open-threads-check 5→0, process-newsletter 8→0, deep-research 4→0. And no rewritten skill contains an artifact-existence check. ~/.claude/skills/deep-research/SKILL.md step 3 now reads "Fan out — one general-purpose sub-agent per question" as plain description, step 6 writes the overnight file that /morning-prep consumes, and step 7 reports "<N> briefs filed overnight." Nothing makes the fan-out required and nothing verifies a brief file exists before the count is asserted. That is precisely the shape of Every's regression: phases skipped, success reported, transcript clean. Their prescription applies verbatim — keep the requirement, drop the seam, and verify by artifact.

Three residual items, in priority order. (1) ~/.claude/skills/station-critic/SKILL.md:37 still instructs "Wait for all subagents to return." This one is probably legitimate — station-critic performs real Agent dispatches, and Every's class 9 warns explicitly that hand-off language inside real subagent prompts is the one boundary in a corpus that is not fiction, so a blanket sweep would break it. Read it, don't batch-edit it. (2) investing-label-historical-phases/SKILL.md:117 and investing-backtest-thesis/SKILL.md:195 both end with "One-line summary to the caller" — a real seam when dispatched as a subagent, a phantom one when invoked in-session via the Skill tool. Dual-mode skills are the ambiguous case and need the caller identity stated, as the investing one already half-does. (3) Cron model aliases are unpinned: check-board ran on claude-sonnet-4-6 at 08:00 today and claude-sonnet-5 at 13:00. Given Every's finding that model identity outweighs effort roughly 3:1 on this failure, an alias drifting under a fixed skill corpus is an uncontrolled model migration we would not currently detect.

The cheapest instrumentation we're missing. Every's headline metric is that tokens per minute separates working from stalling — healthy runs 10–12k/min, halting runs 1.5–3.8k/min — and that failing runs end early, averaging 48 minutes against 88 for successes. RDCO already logs duration_ms, num_turns, and full usage per cron run in ~/.claude/logs/cron-runs/*.log via --output-format json. The derived metric is free. One caution learned the hard way while researching this: the top-level num_turns in those logs can belong to a trailing task-notification record rather than the main run — read modelUsage.outputTokens for real work, or a healthy run will read as a 1-turn halt. Nothing in the current logs indicates an actual halt since the Jul 24 switch; the two exit=137 entries are watchdog timeouts, which is the opposite signature.

Nothing here argues for reversing today's rewrite. The direction was right and it removed real instances of the exact defect. What it argues for is finishing the job Every is only now doing measurement-first: re-add requirement-without-seam language to the fan-out steps, add artifact-existence gates before any "N filed" claim, pin cron model aliases, and derive tokens-per-minute from logs we already write.

Related