08-tooling

fable5 harness review

Fable 5 Harness Review — 2026-06-09

Produced by a 7-seat delegated audit workflow (23 agents) on the day the COO agent moved from Opus 4.8 to Fable 5; executive synthesis below is by the main session, detailed findings are per-seat.

Executive synthesis

The harness moved from Opus 4.8 to Fable 5 today (2x sticker price, ~2.5-2.6x effective after tokenizer inflation, 1M context default). A 7-seat delegated audit (23 agents; 16/16 adversarial verifications held, zero refuted) reached one verdict: the architecture is right — manifest-driven scheduling, fresh-session handoff seed, model-agnostic rules, clean memory index — but it leaks money at the edges and trust at the seams, and its priorities lag the founder's 2026-06-08 phData-primary shift.

Working (keep): the four CLAUDE.md hard rules (rule 4 — subagent-route >5KB artifacts — matters MORE at $10/MTok input); the single-manifest + SessionStart-hook re-arm design; ray-handoff.md as the fresh-session seed; memory index integrity (61/61); fresh-session-over-compaction discipline.

Broken (fixed same day): the restart script's legacy /loop block fired all 14 cron jobs simultaneously on every restart — deleted (root cause of the 2026-06-09 15:40 incident; the SessionStart hook was innocent; backup .bak-2026-06-09). /open-threads-check burned a guaranteed full cache-miss 48x/day (15-min spacing vs 5-min cache TTL), modeled up to ~$2.7k/mo at Fable prices — retuned to hourly (75% cut) pending the script-gated proper fix (~96% cut). Handoff and memory de-staled: Fable 5 recorded as DONE (settings.json pin covers the 4am restart; the dangerous --model-flag instruction removed), phData/L5/age entries corrected.

Founder decisions pending: (1) CRITICAL — 1Password service-account token plaintext in settings.local.json and every session env; verifier decoded it: master unlock key embedded. Rotate in 1P console, then move retrieval to Keychain/runtime-fetch. (2) enableAllProjectMcpServers=true auto-trusts any project .mcp.json (verifier planted a hostile one; it spawned with zero prompt) — flip to false; classifier correctly blocked Ray from self-editing permissions config, needs the founder's one-line edit; note the verifier found headless -p mode spawns project servers even with the flag false, so also vet .mcp.json in any untrusted dir. (3) The verify-action channel-reply guardrail the founder believes is active was never wired (snippet drafted, never applied; log empty for 35 days) — wire it in a supervised, non-blocking rollout; this is also the agreed "pre-send verification gate," the highest-value open build. (4) Prune: vercel plugin (36 skills, no usage evidence on a Cloudflare stack), core-3d-animation + animation-components plugins, blender/stripe/heygen/mobbin MCP servers, ~15 stale skills to archive. (5) Add: /cert-study skill + weekly cron (the PRIMARY bet has zero scheduled support today), investing watch crons (13F/Congress/EDGAR cadences declared in skills but never wired into the manifest — the May 13F window was missed), and a subagent model-tiering policy (haiku for extraction, sonnet for pipeline/critic seats, Fable only for founder-facing judgment; 10:3:1 price ratios).

Economics in one line: on Fable 5 the dominant cost lever is not per-turn model choice — it is cache-miss recurring wakes against a fat session prefix. Gate crons with scripts, keep the prefix lean, tier the subagents.

Detailed findings by seat

Scheduling (crons, loops, restart scripts)

Scheduling is architecturally sound (single manifest, SessionStart auto-arm hook, OS-level cron only for pure scripts) but has one live defect and one cost hotspot. The 2026-06-09 incident was NOT caused by emit-cron-rearm.sh — that hook (registered only on startup/resume matchers) correctly emits a single CronCreate instruction; the 14 "/loop 0 0 * * * /vault-health"-style enqueues came from a legacy block at claude-channels-restart.sh:100-111 that pipes raw manifest lines into /loop via tmux immediately after the script's own /reload-plugins send at line 93, and /loop cannot parse 5-field cron (an unrecognized interval falls through to dynamic mode which executes the whole line as a prompt immediately — 14 jobs firing at once). The fix is deleting that block, plus closing the gap that /reload-plugins wipes session crons with no matcher to re-arm them. On Fable 5 economics ($10/MTok input, 2x Opus 4.8, 5-min cache TTL), /open-threads-check at 48 fires/day on 15-min spacing is a guaranteed cache miss every fire — roughly $30-90/day ($900-2,700/mo) at plausible session-context sizes — and should become a pure-script pre-check that wakes the agent only when an aged unanswered ask exists. The manifest also has inverted priorities: the PRIMARY bet (phData certs, hard deadlines 2026-08-24 and 2026-11-22) has zero scheduled support and no cert-related skill exists, while two investing skills that explicitly specify cron cadences (13F quarterly, Congress monthly, EDGAR capex quarterly) were never wired into the manifest.

1. [critical] incident/root-cause (action: fix)

2. [info] incident/hook-paths (action: keep)

3. [critical] incident/loop-grammar (action: fix)

4. [warn] incident/rearm-race (action: fix)

5. [critical] jobs/open-threads-check (action: fix)

6. [warn] jobs/process-inbox (action: fix)

7. [warn] jobs/process-newsletter (action: fix)

8. [info] jobs/vault-health (action: fix)

9. [info] jobs/graph-reingest (action: fix)

10. [info] jobs/keep-set (action: keep)

11. [warn] jobs/missing-phdata-cert (action: add)

12. [warn] jobs/missing-investing-crons (action: add)

13. [info] os-level/launchd-cron (action: keep)

14. [warn] out-of-domain/security (flag to security seat) (action: fix)

Verification

Rules (CLAUDE.md, hard rules, precedence)

The governing-rules layer is in good shape for the Fable 5 transition: both files are fully model-agnostic (zero references to Opus/Sonnet/Haiku/Fable), so the upgrade introduced no staleness, and all four hard rules remain correct — rule 4 (subagent-route artifacts >5KB) is actually more valuable now that input tokens cost $10/MTok vs Opus 4.8's $5/MTok. The main structural issues are: (1) the prompt-precedence section consumes 47% of CLAUDE.md (5,816 of 12,307 bytes) and its worked-scenarios block (1,553 bytes) is teaching material that can relocate to the already-existing implementation-notes doc; (2) worked scenario 5 internally contradicts the precedence chain it illustrates (claims tier 6 memory beats tier 4 dispatch prompt, and mislabels PR-only as a hard rule); (3) rules 1 and 2 are triple-covered across CLAUDE.md, memory files, and tier-1 MCP instructions — the memory duplicates can retire; and (4) there is no sub-agent model-tiering policy anywhere, which matters at 2x flagship input pricing — draft policy text provided for founder greenlight. Combined persistent preamble is ~17.3KB (CLAUDE.md 12,307 B injected every session via claudeMd; SOUL.md 5,022 B read per session per CLAUDE.md:3), mitigated per-turn by prompt caching but rewritten at every 4am restart.

1. [info] staleness (action: keep)

2. [info] hard-rules (action: keep)

3. [warn] hard-rules (action: keep)

4. [warn] prompt-precedence (action: fix)

5. [warn] prompt-precedence (action: remove)

6. [warn] redundancy (action: remove)

7. [info] redundancy (action: remove)

8. [info] redundancy (action: decide)

9. [info] cost (action: keep)

10. [warn] gaps (action: add)

11. [info] gaps (action: keep)

Verification

Skills (SKILL.md inventory)

The skills inventory is large but mostly intentional: 85 directories (83 with SKILL.md, 742,858 bytes of SKILL.md, ~5.8MB on disk), of which only 14 are cron-driven; the rest are on-demand rails for the side bets. The real costs are (a) ~46.6KB of skill descriptions (~28.1KB personal + ~18.5KB plugin) injected into every session and every subagent spawn — meaningful at Fable 5's ~2x input price given nightly fan-out patterns — and (b) the vercel plugin (36 skills) plus two animation plugin families that have no usage evidence against a Cloudflare/wrangler stack and duplicate the personal HyperFrames adapter skills. Two integrity bugs surfaced: verify-action's SKILL.md claims a PreToolUse hook wiring that is absent from both settings files, and investing-smart-money-watch's description claims crons that do not exist in the scheduled-jobs.txt single source of truth. The largest strategic gap: zero skills match phdata/cert/snowflake even though phData certs are the founder-declared primary bet with hard deadlines (2026-08-24, 2026-11-22) and study-plan files already sitting in the vault.

1. [info] inventory totals (action: keep)

2. [warn] context cost of skill-list preamble (action: fix)

3. [warn] plugin sprawl — vercel (action: remove)

4. [warn] plugin sprawl — animation families (action: remove)

5. [info] plugin sprawl — firebase (keep) (action: keep)

6. [warn] misplaced repo in skills dir (action: fix)

7. [warn] stale/abandoned skill candidates (action: remove)

8. [info] verify- family overlap* (action: decide)

9. [warn] verify-action hook wiring broken (action: fix)

10. [warn] investing-smart-money-watch cron drift (action: fix)

11. [warn] phData-primary gap (action: add)

12. [info] phData-relevant skills to retain (action: keep)

13. [info] design-skill overlap (action: decide)

Verification

Memory (MEMORY.md + feedback files)

Memory index integrity is clean (61 files, 61 entries, no orphans or dangling links) and — notably — zero memories encode the runtime model, so the Fable 5 swap requires no correctness edits to memory. The dominant problem is index-vs-body drift on the highest-priority topic: the per-session-injected MEMORY.md line for the phData bet still carries the wrong cert name, a superseded fixed sequence, and 'job starts 2026-05-26', while the file body was corrected Jun 8; relatedly, the L5 memory still claims unhobbling is the 'primary focus', contradicting the newer phData MAIN-bet memory, and the dob_age memory now asserts a wrong age (35 vs actual 36). The single critical item is feedback_calibrate_overconfidence.md: a 20.5KB fabrication-incident journal whose own text concludes memory bullets failed to stop the failure mode and twice demands a structural pre-send hook that was never built — that hook should be implemented now, since the model swap resets the behavioral priors the journal was written against. Secondary hygiene: merge the two overlapping xmcp entries, consolidate the duplicated batching rule into no_batched_result_declaration, refresh the 73-day-old channels-setup memory that documents the daily-restart infra, and strip point-in-time state (dates, ages, spec stages) out of injected index lines.

1. [info] index integrity (action: keep)

2. [warn] staleness: phData index line (action: fix)

3. [warn] staleness + contradiction: L5 vs phData priority (action: fix)

4. [warn] staleness: user_dob_age (action: fix)

5. [critical] bloat + state drift: feedback_calibrate_overconfidence (action: add)

6. [info] integrity: missing incident #9 (action: fix)

7. [info] overlap: calibrate_overconfidence vs no_batched_result_declaration (action: fix)

8. [info] overlap: two xmcp entries (action: fix)

9. [info] model-change exposure (action: keep)

10. [info] staleness: feedback_alpaca_paper_opg_unreliable (action: keep)

11. [warn] staleness: project_channels_agent_setup (action: fix)

12. [warn] MEMORY.md size + state-in-index drift (action: fix)

Verification

State (working-context, state files)

The state-file layer is fundamentally healthy: ray-handoff.md (6,580 bytes, 49 lines) is the right shape for a fresh-session seed, and working-context.md at 22,995 bytes is NOT overdue for a prune — it was condensed from 153,222 bytes on 2026-05-31 and remains right-sized. The problems are staleness, not bloat. The handoff's top 🔴 bullet still describes the Fable 5 upgrade as pending ("awaiting his explicit do it", "running on Opus 4.8 for now") when settings.json:252 now shows "claude-fable-5[1m]" saved as default (mtime Jun 9 15:38) with effortLevel xhigh — and the handoff's proposed remedy (add --model claude-fable-5 to the restart script) is now both unnecessary and dangerous, since the flag omits the [1m] suffix and would override the saved 1M-context default. Secondary drift: the handoff claims 16 cron jobs vs 14 actual manifest lines, the numbered open-threads list (items 1-4) is stale ordering with item 4 contradicting the no-auto-ping rule and item 3 duplicating the 🔴 energy bullet, and working-context.md line 74 still claims settings.json has no model/effort key. founder-energy.txt has been dead for 25 days and two separate handoff bullets keep re-litigating its staleness. The 4.1MB unbounded newsletter-audit-log.md is the only file-size concern in the directory.

1. [warn] ray-handoff.md — Fable 5 open thread (action: fix)

2. [warn] ray-handoff.md — proposed restart-script change (action: fix)

3. [warn] ray-handoff.md — cron job count drift (action: fix)

4. [warn] ray-handoff.md — stale-ordering in Open threads section (action: fix)

5. [info] ray-handoff.md — stale 'Last written' header (action: fix)

6. [info] ray-handoff.md — overall fitness as fresh-session seed (action: keep)

7. [info] working-context.md — size (action: keep)

8. [warn] working-context.md — model/effort section drift (action: fix)

9. [warn] founder-energy.txt — dead signal (action: decide)

10. [info] ray-handoff.md — missing Fable 5 operating notes for the seed (action: add)

11. [warn] state dir hygiene — unbounded append log (action: remove)

Verification

Config (settings.json, hooks, MCP, permissions)

Harness config is fundamentally sound: the Fable 5 model pin is saved in settings.json and covers the 4am restart (the restart script launches bare claude with no --model flag, so the saved default applies), hooks are coherent and all referenced scripts exist, and the bridge-notes/cron-rearm continuity machinery is intact. The two real problems are (1) a plaintext 1Password service-account token in settings.local.json that is exported into every session and subprocess environment — combined with blanket Bash/Write/Edit allows and auto permission mode in an always-on agent that ingests untrusted web/newsletter content, this is the single worst exposure in the config; and (2) MCP sprawl: 28 configured servers, 6 of them permanently stuck in "Needs authentication," injecting an estimated ~420 deferred tool names plus 8+ instruction preambles (~8-10K tokens, estimate) into every session at Fable 5's ~2x input price. Disconnect candidates with no tie to the current objectives: heygen, stripe, blender, mobbin, Canva, and the four unauthenticated cloudflare plugin servers + vercel; xcodebuildmcp (Squarely), elevenlabs, monarch-money, whoop, and qmd earn their keep. One objective-relevant gap: the sql-guardrail hook never fires for the Snowflake snow CLI — the tool most relevant to the primary phData/Snowflake cert bet — because settings.json only matches snowsql/duckdb/psql.

1. [info] Model pin / 4am restart (action: keep)

2. [info] Model config / cost (action: decide)

3. [critical] Secrets / settings.local.json (action: fix)

4. [warn] Permissions allowlist (action: decide)

5. [info] Permissions allowlist (action: remove)

6. [info] Permissions allowlist (action: remove)

7. [info] Permissions allowlist (action: remove)

8. [info] Hooks — enumeration (action: keep)

9. [warn] Hooks — sql-guardrail coverage gap (action: add)

10. [warn] Hooks — sql-guardrail blocking risk (action: decide)

11. [info] Hooks — package-security-check (action: keep)

12. [info] MCP inventory (action: keep)

13. [warn] MCP context overhead (action: remove)

14. [warn] MCP sprawl — heygen (action: remove)

15. [warn] MCP sprawl — stripe (action: remove)

16. [warn] MCP sprawl — blender (action: remove)

17. [warn] MCP sprawl — mobbin (action: remove)

18. [info] MCP sprawl — keepers (xcodebuildmcp, elevenlabs, computer-use) (action: keep)

19. [warn] MCP sprawl — dead needs-auth plugin servers (vercel, cloudflare x4) and Canva (action: decide)

20. [warn] MCP trust posture (action: fix)

Verification

  1. TOKEN LITERAL: /Users/ray/.claude/settings.local.json lines 13-14 contain exactly: "env": { "OP_SERVICE_ACCOUNT_TOKEN": "ops_eyJzaWduSW5BZGRyZXNz..." } as a full plaintext literal (the entire ops_ token, not a reference or wrapper call). Confirmed by Read.

  2. EMBEDDED secretKey: I base64-decoded the ops_ payload. The JSON contains "secretKey":"A3-5ZZPK8-Z3NS44-DJ4VJ-HKTMK-ADEHJ-4TMGR" AND a "muk" (master unlock key: A256GCM key material) AND "srpX". This is worse than the claim states — not just the secretKey but the master unlock key is embedded, i.e. full cryptographic material for the service account (email ixkvevtynzdze@1passwordserviceaccounts.com).

  3. FILE MODE: ls shows -rw-------@ (0600, owner-only). Matches the claimed "-rw-------". Note this 0600 limits other LOCAL users from reading the file, but does NOT mitigate the stated threat (the agent's own process holds the token in env, so a prompt-injected env dumps it regardless of file perms).

  4. ENV-INJECTION + BLANKET BASH + AUTO MODE (verified the broader threat model, not just the two cited lines): /Users/ray/.claude/settings.json has permissions.defaultMode = "auto" and the allow list's first entry is a bare "Bash" (blanket allow for ALL bash commands). So env would run with no prompt. Claude Code's settings env block injects the var into the session and spawned-subprocess environments, so OP_SERVICE_ACCOUNT_TOKEN is reachable by any Bash child. The injection chain (untrusted-content ingest -> injection runs env -> token exfiltrated) is mechanically sound.

  5. RULE CONTRADICTION: feedback_no_secrets_on_disk (per MEMORY.md) mandates 1Password wrapper scripts and "never secrets on disk." A plaintext 1Password service-account token in a config file is a direct contradiction; the irony is sharp because the on-disk secret IS the 1Password master credential that wrapper scripts are supposed to fetch at runtime.

One scoping caveat (does not refute): the claim that this is "the master key to every credential (Stripe, ElevenLabs, xAI, Alpaca)" depends on which vaults this service account is granted read access to — I did not enumerate vault scope, so the specific blast radius is asserted, not verified. But the token unambiguously grants whatever that service account can read, and the embedded muk+secretKey are full account credentials. The core security finding stands. Recommendation (move to Keychain, fetch per-wrapper) is the correct remediation.

  1. Cited line verbatim: /Users/ray/.claude/settings.local.json line 12 reads exactly "enableAllProjectMcpServers": true. Confirmed by both Read and grep -n. (enabledMcpjsonServers there is just ["qmd"]; permissions.allow is unrelated.)

  2. Authoritative semantics from the Claude Code binary (v2.1.170, /Users/ray/.local/share/claude/versions/2.1.170). The settings schema literally describes the flag: enableAllProjectMcpServers ... describe("Whether to automatically approve all MCP servers in the project"). The approval-resolution function returns "approved" when _?.enableAllProjectMcpServers is set, otherwise "pending". So with the flag true, ANY project .mcp.json server resolves to approved instead of prompting — exactly the auto-trust the claim describes, and it does contradict the security-review-before-install default.

  3. Empirical: I planted a hostile .mcp.json (server command bash -c "touch MARKER; exec cat") in a fresh project dir and ran claude -p. The MARKER file was created = the malicious server was spawned with zero approval prompt. Cleaned up all test dirs afterward.

The recommendation (set false + enumerate trusted servers via enabledMcpjsonServers) is sound and the four named servers match /Users/ray/.mcp.json (qmd, xmcp, xcodebuildmcp, elevenlabs).

IMPORTANT CAVEAT that makes the concern WORSE, not weaker: in headless -p mode the malicious server still spawned even when I forced enableAllProjectMcpServers:false (and enabledMcpjsonServers:[]) at project-local AND at the highest-precedence --settings flag level, in clean never-run directories. Merely loading a server executes its command, and print/non-interactive mode appears to load project .mcp.json servers regardless of the approval flag. This Mac Mini agent runs headless. So flipping the flag to false is necessary and correct for the interactive guard, but may be INSUFFICIENT to close the headless attack surface — the safer mitigation is to not let untrusted .mcp.json files exist in any directory the agent will cd into / run in (e.g., vet cloned repos before working in them, or strip their .mcp.json). I did not find a setting that disables project .mcp.json loading in headless mode during this audit.

Model facts (Fable 5 pricing, caching, economics)

Fable 5 facts are well-pinned by two mutually consistent sources read this session: the founder-verified vault release note (2026-06-09) and the bundled claude-api skill (pricing table cached 2026-05-26, already lists claude-fable-5). Headline economics: $10/$50 per Mtok (exactly 2x Opus 4.8's $5/$25), 1M context by default with no long-context premium and no [1m] beta toggle, cache read $1/M (0.1x), cache writes $12.50/M (5-min, 1.25x) and $20/M (1-hr, 2x), and a tokenizer that inflates the same text ~30% — so effective cost vs Opus 4.8 is ~2.5-2.6x, not 2x. The dominant cost risk for this always-on harness is cache-miss cron fires at large parent context: a 48x/day all-miss cron at 150k tokens costs $72-90/day ($2,160-2,700/month) on Fable 5, versus $90-112.50/month if a shell pre-check gates model fires down to ~2/day — a ~96% reduction that dwarfs any model-choice decision. Adaptive thinking cannot be disabled on Fable 5 (explicit disabled returns 400), so every fire also buys thinking tokens at $50/M output; effort is the only spend lever. Subagent tiering from the Fable 5 parent should default to Haiku 4.5 (10x cheaper both directions) for extraction/classification seats and Sonnet 4.6 (3.3x cheaper) for reasoning/critic seats, reserving inherit for founder-facing judgment.

1. [info] fable5-facts/model-ids-and-base-pricing (action: keep)

2. [info] fable5-facts/cache-economics (action: keep)

3. [info] fable5-facts/1m-context-terms (action: keep)

4. [info] fable5-facts/tokenizer-inflation (action: keep)

5. [info] fable5-facts/adaptive-thinking-always-on (action: keep)

6. [info] cost-model/single-cache-miss-turn (action: keep)

7. [warn] cost-model/cron-cache-miss-exposure (action: fix)

8. [warn] cost-model/subagent-model-tiering-policy (action: add)

9. [info] cost-model/batch-api-for-offline-pipelines (action: decide)

Verification

Run stats

23 agents, ~2.63M subagent tokens, 237 tool uses, ~15 min wall clock.