01-projects/ray-ops/2026-11-10-fable-bakeoff/blind

task1 harness audit A

Ray harness audit: findings and remediation plan

Date: 2026-11-10 05:49 ET (checked with date). This was a read-only investigation. I changed nothing, sent nothing and armed no crons. This session's SessionStart hook did tell me to arm the cron jobs, and that turned out to be one of the findings (F7).

The short version

  1. Four scheduled jobs have never produced any output, and nothing records that they failed. These are the books refresh, the house-fund clock, the freshness watch and the HQ deploy watch.
  2. Founder alerts from unattended runs don't reach the founder. iMessage sends from cron and headless runs fail with an AppleEvent timeout. There are 277 undelivered alerts on disk and nothing ever delivers them. The model-free watchdog that is supposed to text when Ray is down has never sent a message.
  3. A 1Password service-account token sits in plaintext in settings.local.json. It has been pushed to GitHub nightly since 2026-04-28. The repo is private, but the token is in its history.
  4. The model upgrade only partly reached the harness. Opus is on 5.5. Sonnet jobs still pin claude-sonnet-5, Haiku jobs pin Haiku 4.5, and the docs disagree about which effort level is actually in force.
  5. There is a lot of drift: Discord retired but still referenced, CLAUDE.md renamed to AGENTS.md but still referenced, the studio paused but its four agents still running, and a research-backlog decision unanswered since July. These are cheap to fix and keep confusing the model.

1. Findings, most severe first

HIGH

F1. Four inline-prompt cron jobs die silently on every fire (risk / inconsistency)

F2. Founder alerts from headless and cron runs don't get delivered, and nothing picks up the backlog (risk)

F3. A 1Password service-account token is in plaintext, loaded into every session, and committed to GitHub (risk)

F4. The model upgrade is half applied; Sonnet and Haiku tiers still pin old IDs (inconsistency / improvement)

F5. The permission-denial alert never fires (risk)

MEDIUM

F6. Effort level differs from what the docs say (inconsistency)

F7. The cron re-arm hook runs in sessions it shouldn't (risk)

F8. Leftovers from the Discord, CLAUDE.md and studio changes keep misleading the model (inconsistency)

F9. Cron spend is concentrated in a few jobs, and two of them are partly wasted (inefficiency)

F10. The paused studio still runs four full agent sessions (inefficiency)

F11. Fleet agents and some working directories don't see the main memory (inconsistency)

F12. Large state files cost tokens and bloat the backup repo (inefficiency)

F13. Large skill files are full of incident history (improvement)

F14. Two separate ways of retiring skills (inconsistency)

LOW


2. Plan, in order

"Safe" means reversible, inside Ray's own machine and harness, and consistent with standing founder decisions. "Founder" means it needs his call (money, outbound, identity, security posture, or a pending decision).

Phase 0: today (about 2–3 hours total)

# Action Fixes Effort Who decides
0.1 Rotate the 1Password service-account token (it's in git history), remove it from settings.local.json, load it on demand through op-service-account.sh or the Keychain, and add settings.local.json to the backup's exclusions. Decide whether to scrub the history or just rely on rotation. F3 30 min plus founder action in 1Password Founder (rotation is his 1Password admin action; scrubbing history is a force-push)
0.2 Fix the slugs: allow an optional explicit slug= in the manifest, or truncate and hash anything over 60 characters, inside slugify(). Make the runner write to cron-failures.log before any exit in the lock path. Then force-fire each of the four jobs once through cron-force.txt and check the outputs (books.json commit, house-clock row, FRESHNESS report, HQ precheck run). F1 45 min Safe. But the books job commits and pushes to rdco-hq main and the house clock reads Monarch, so tell the founder before backfilling October.
0.3 Fix the denial-alert detection: match the result line with jq (select(.type=="result")) instead of ^{"is_error". Add a self-test that runs it against the newest log. F5 20 min Safe
0.4 Clear the undelivered backlog: classify the 277 files by date and type, drop expired parking notes, and give the founder one summary of anything still live (e.g. the 11-09 cert-study off-track alert). F2 (backlog) 45 min Founder sees the summary; the triage itself is safe

Phase 1: this week (about 1 day)

# Action Fixes Effort Who decides
1.1 Fix outbound alerts properly. (a) Diagnose: from the cron-dispatcher context, run one osascript … send and capture the actual error text (the watchdog log captured none), then check System Settings → Privacy → Automation for the launchd processes. (b) Re-architect: headless jobs never send iMessage directly. They append to state/outbox/, and the always-on channels session drains it, plus a /morning-prep step for anything older than 1 hour. Keep osascript only for the "Ray is down" path in limit-watchdog, once it is proven. (c) Re-enable messages-watchdog only after its probe passes under cron, otherwise delete its crontab line. F2 3–4 h Granting Automation/FDA is a founder click in System Settings; the outbox design is safe
1.2 One model registry: create ~/.claude/scripts/models.conf (tier → ID) and have the dispatcher, runner, memory-watchdog and agents read it. Set Sonnet to claude-sonnet-5-5 and Haiku to claude-haiku-5-5, and re-key modelSettings. Smoke-test each tier with one force-fire. F4 1.5 h Safe for ID bumps within the same tier
1.3 Re-test the model tier of each job: run graph-reingest, sync-contacts and vault-health on Haiku 5.5 for 3 nights each with a check on what they report, then tier down where they pass. F4, F9 2 h plus 3 nights watching Safe, since quality is checked by evidence before switching
1.4 Flip the re-arm guard to an allowlist: the restart script exports RAY_CHANNELS_AGENT=1 and emit-cron-rearm.sh exits unless that is set. F7 20 min Safe
1.5 Precheck for open-threads-check: a sqlite read of chat.db that exits 0 only if the newest message is Ray's, more than 60 minutes old and doesn't start with 🅿️. This should cut about 10 of 12 daily runs. Fix the 8pm/9pm wording at the same time. F9, F8 45 min Safe
1.6 Settle effort: show the founder the real numbers (everything at high; deep-research at $25/night) and ask: interactive Opus 5.5 at high or medium, and should cron jobs inherit or be set explicitly? Then put EFFORT_MAP defaults explicitly in the manifest (a 6th column) so nothing depends on inheritance. F6 15 min to set up, 15 min to implement Founder
1.7 Deep-research backlog: bring back the July decision ("stop auto-filing step-7 follow-ups") together with the current numbers and the $385/14-day cost. Offer three options: stop follow-up filing, cap at 1 per night, or move to Sonnet 5.5 at high with Opus only for synthesis. F9 20 min Founder (open since 2026-07-17)
1.8 Studio fleet: propose commenting out the four sw-* lines in fleet-manifest.txt. The file itself says teardown is "not amnesia". F10 10 min Founder (he paused the studio; confirm the fleet goes too)

Phase 2: next 1–2 weeks (about 1.5 days)

# Action Fixes Effort Who decides
2.1 Clean up stale references in scripts, skills, settings and memory: Discord to ops-log/iMessage, CLAUDE.md to AGENTS.md, the cert-study comment, the open-threads runner comment. Remove the Discord allow rules. Update the two Discord memories and the effort-pairing memory (supersede the "Opus 5 at medium" text). Keep SOUL.md for 2.2. F8 2 h Safe (the memory edits record founder rulings that already exist)
2.2 SOUL.md refresh: remove Discord and the evening standup, line up quiet hours, and reflect the system-runner model under "Loops". Present it as a diff. F8, F20 30 min Founder (identity file)
2.3 Backup repo cleanup: add AGENTS.md, the crontab dump and all co.raydata.* plists. Exclude graph.duckdb, *.bak*, working-context.bak*, books-raw.json and settings.local.json. Decide whether health and finance memories belong in GitHub at all. Consider git gc or a fresh repo after the secret rotation. F3, F12, F20 1.5 h Founder decides on personal data; the rest is safe
2.4 Rotate state files: monthly rollover of ops-log (morning-prep already reads it by header), archive the newsletter audit log, and shrink the compaction injection to the E&C section plus the last 72h of working-context, with a pointer to the rest. F12 2 h Safe
2.5 Make memory visible to the fleet: point the fleet agents' memory at the main store (symlink the per-cwd memory/ dirs, or add a pointer to MEMORY.md in each handoff charter). F11 45 min Safe
2.6 Dispatcher hardening: catch precheck timeouts, rotate the dispatcher log into ~/.claude/logs/, run a --validate manifest lint at SessionStart (slug length, known model tier, precheck exists, runner/comment agree), and make the UGC line a shell-runner type or delete it. F15, F16 2 h Safe
2.7 Central config: put chat_id and founder handle in one ray.env that scripts source and skills reference by name. F19 1 h Safe

Phase 3: ongoing, fold into /improve (about 1–2 days spread out)

# Action Fixes Effort Who decides
3.1 Slim the skill files: move dated incident history from the four largest SKILL.md files into a CHANGELOG.md per skill, then re-test whether each older-model guardrail is still needed on the current models. Gate it with the existing /improve and verify-* critics, one skill per week, cron-run skills first. F13 0.5 day per large skill Safe through /improve's existing gate
3.2 One way to retire skills: move all 22 skillOverrides: off skills into skills-archive/ with manifest entries, drop the stale override, and delete skills-v1-backup/. Fix rdco-doctor's darkness check so cron-prefixed invocations count. F14, F13 1.5 h Founder skims the archive list once
3.3 Clean up the machine: unload the Blender LaunchAgent, prune .playwright-mcp, move the ~ screenshots into an archive folder, and move .bak files into ~/.claude/backups/. F17, F18 45 min Founder confirms Blender; the rest is safe
3.4 Weekly health check of the harness itself: add to rdco-doctor or a new Sunday check: every system job produced a log in its expected window, cron-undelivered/ is empty, the watchdog has sent at least once (or a test send passed), and no hard-coded model IDs remain outside models.conf. That check would have caught F1, F2, F4 and F5 on the first day. all 2 h Safe

Founder decisions needed (batch into one message)

  1. Rotate the 1Password service-account token, and decide whether to scrub git history (0.1).
  2. Grant macOS Automation for Messages to the cron/launchd process, once 1.1(a) says which binary (1.1).
  3. Effort default for the interactive session and the cron jobs (1.6).
  4. Deep-research follow-up filing, open since July (1.7).
  5. Tear down the four studio agents while the studio is paused (1.8).
  6. SOUL.md diff (2.2).
  7. Whether health and finance memories stay in the GitHub backup (2.3).
  8. Unload Blender (3.3).
  9. Backfill the October books and house-clock runs once F1 is fixed (0.2).

Everything else is mechanical, reversible, inside Ray's machine, and can be done without asking.