Ray harness audit: findings and plan (Fable 5.2 arm, 2026-11-10)
Bottom line
The harness's biggest problem is not the model tier: it reports green while several things are broken. Four scheduled jobs have never run, both Haiku jobs have been failing their writes since mid-August, alerts sent from outside the tmux session don't reach the founder, and the denial alarm is dead. cron-failures.log has no entry since 10-24. Separately, the 1Password service-account token has been pushed to GitHub nightly since April. The model upgrade itself is one-third done: only opus was bumped.
Nothing was changed. The SessionStart hook told this session to arm /morning-prep; I did not, because arming it here would double-fire the brief (that is finding 7).
I report spend as shares rather than dollars, per the standing memory rule.
Findings
High
| # | Finding | Evidence | Category |
|---|---|---|---|
| 1 | 1Password service-account token is in GitHub. It sits in plaintext in settings.local.json, and the nightly backup commits and pushes that file. The handoff knows about the on-disk copy but not the GitHub one. The same repo also carries books-raw.json, ~/.claude.json, and memory files holding health and home-address data. |
~/Projects/ray-config/bin/sync.sh:24-25; git ls-files shows dotclaude/settings.local.json tracked; git log -S puts the token in the first commit, 2026-04-28; .gitignore patterns match filenames only; state/ray-handoff.md:55. The env block is redundant: scripts/op-service-account.sh:7 already reads the token from Keychain. |
risk |
| 2 | Four scheduled jobs have never run. The slug is the whole prompt, so inline-prompt jobs get 617–1,270-character slugs. The lock-dir name exceeds the 255-character limit, mkdir fails, and the runner exits 0 silently. Affected: house-fund clock, books refresh, freshness watch, HQ deploy watch. |
cron-dispatcher.py:82-85,159; cron-job-runner.sh:28-36; a read-only stat on a 300-character name returns ENAMETOOLONG. The dispatcher logged FIRE on 11-02, 11-03 and 11-07, with no run logs and no ops-log entries for any of them, ever. rdco-hq/src/data/books.json was last committed 07-28; house-fund-tracker.md:32 (October row) is empty. |
inconsistency |
| 3 | Alerts from outside the tmux session don't deliver. Headless and cron sends time out with AppleEvent -1712. messages-watchdog.sh is disabled by an exit 0 but still fires from crontab every 5 minutes. Nothing drains cron-undelivered/. |
45 of 180 open-threads-check runs found an ask and failed to send. The 11-09 cert-study off-track alert is sitting in state/cron-undelivered/. state/watchdog/limit-watchdog.log shows SEND FAILED on 10-22 during a 2.5-hour login stall. cron-undelivered/ holds 279 files; morning-prep/SKILL.md:32 reads only cron-failures.log. |
risk |
| 4 | Both Haiku jobs fail their writes and claim success. In 14 days sync-contacts had 139 permission denials and graph-reingest 93, on ledger writes, ops-log.sh and the undelivered fallback. The contacts ledger has not been written since Aug 17, so the "seen 2+ times in 7 days" stub rule cannot trigger. |
state/sync-contacts-ledger.json and state/graph-reingest.log both have an Aug 17 mtime; neither job has an ops-log entry in 30 days; the 11-10 sync-contacts result says "State ledger updated". Both run on claude-haiku-4-5-20251001. |
inconsistency |
| 5 | The denial alarm is dead and success is never checked. The runner looks for a JSON line starting {"is_error"; the CLI now emits {"duration_api_ms" first. Exit 0 counts as success with no check that a report landed. |
cron-job-runner.sh:136; 248 denials in 14 days, zero *-denial-alert.log files, zero cron-denial-* ops-log entries. |
risk |
| 6 | Model upgrade is partial and effort is implicit. sonnet still maps to claude-sonnet-5 and haiku to 4.5, while sub-agents already run claude-haiku-5-5. "Inherit" jobs now run at high effort, though the dispatcher comment and the delegation memory both say medium. The open-threads-check line has model - (silently falls back to sonnet) under a comment calling it a session job. |
cron-dispatcher.py:48-52,66-67,194; hardcoded Haiku 4.5 at cron-job-runner.sh:161 and in memory-watchdog.sh; agents/fanout.md; settings.json:289-293; check-board/SKILL.md:8; scheduled-jobs.txt:93-98; memory/feedback_delegation_model_effort_pairing.md. |
inconsistency |
For finding 3, the likely cause is that launchd and cron process trees lack macOS Automation permission for Messages (messages-watchdog.sh:15-18 says so for cron). Messages.app wedging is also documented, so some failures may be that instead. For finding 4, my hypothesis is that Haiku 4.5 is not running under auto mode, so unlisted Bash is denied; I did not confirm it.
Medium
| # | Finding | Evidence | Category |
|---|---|---|---|
| 7 | The cron re-arm hook is a denylist. Any session on this machine that is not a cron run or fleet agent is told to arm /morning-prep; this session was. Arming also depends on the model obeying a hook. |
emit-cron-rearm.sh:24-30 |
risk |
| 8 | Spend is concentrated, with many no-op fires. deep-research is 56% of 14-day cron spend. Newsletter watch is 13%, with about 55% no-op runs. open-threads-check is 8%, with about two-thirds nothing-to-park and the rest mostly failed sends. check-board is about 30% no-op. The no-op rates are keyword estimates from result text. |
Aggregated from total_cost_usd in logs/cron-runs/*.log. |
inefficiency |
| 9 | Deep-research contradicts its own memory. It files 3–9 follow-ups a night while the memory says to hold them. The decision was asked on 07-17 over Discord, which the founder never read. | memory/project_deep_research_backlog_inflation.md; nightly results say "N new follow-ups added". |
inconsistency |
| 10 | Idle fleet. Six support agents respawn at 4am daily: four sw-* agents while Scribble Works is paused, and caf-architect on an Aug 16 handoff for a retired framing. |
scripts/fleet-manifest.txt:12-17; state/fleet-status/ last written Oct 1–9; 11 claude processes at about 2.9 GB. |
inefficiency |
| 11 | Skill bloat. There is 873 KB of SKILL.md text. improve is 56 KB, of which 52 KB is changelog, loaded on every Opus fire. 22 of 59 skills are switched off but still in the directory, and one override names a skill that doesn't exist (investing-edgar-watch). |
skills/improve/SKILL.md; self-review 43 KB; process-youtube 52 KB; settings.json:84-107. |
inefficiency |
| 12 | Dispatcher fragility. No catch-up for missed minutes, so monthly jobs are lost to a reboot. State is written before the precheck. The log lives in /tmp. A missing precheck fails open. Per-job config is split across four places. |
cron-dispatcher.py:184,237-239; TIMEOUTS, EFFORT_MAP, and the runner's case at cron-job-runner.sh:78-80. |
risk |
| 13 | Stale text after the Discord and CLAUDE.md retirements. | SOUL.md:20-22 (evening standup, Discord) and :74 ("small, overwriting is fine"; the file is 36 KB and re-injected whole by read-bridge-notes.sh). Six skills and cron-runner-context.md cite "CLAUDE.md hard rule". Discord permissions remain at settings.json:14-18 and in settings.local.json. channels/discord/.env still holds a bot token. 22 memory files mention Discord or CLAUDE.md. |
inconsistency |
| 14 | Backup gaps. The sync copies ~/CLAUDE.md, which no longer exists, and not AGENTS.md. It omits the cron-dispatcher and memory-watchdog plists and the crontab. |
sync.sh:47,53-57; home/CLAUDE.md in the repo dates from Sep 18. |
inconsistency |
Low
| # | Finding | Evidence | Category |
|---|---|---|---|
| 15 | Clutter. 38 .bak files in scripts/, 16 settings backups, one-off deploy scripts, and ask-model.sh for a retired critic. The backup repo's .git is 534 MB because graph.duckdb and evidence PNGs are committed nightly. |
ls scripts; git count-objects in ray-config. |
inefficiency |
| 16 | "Boil the ocean" is emphatic over-instruction. It pulls against the signal-to-noise and "meaningfully large tasks only" memories, and newer models follow such text more literally. | SOUL.md:45-51 |
improvement |
Plan
"Safe" means reversible work inside Ray's authority. Edits to settings.json, settings.local.json and agents/ are founder-gated today, so Ray drafts the diff and the founder applies it.
| Step | Action | Fixes | Effort | Who |
|---|---|---|---|---|
| 0. Today | ||||
| 0a | Check the ray-config repo's visibility. Rotate the token in 1Password, store the new one in Keychain only, and delete the env block. |
1 | 20 min | Founder |
| 0b | Add settings.local.json, .claude.json and books-raw.json to the sync excludes; add a pre-push secret scan using the patterns already in verify-action.py:56. |
1 | 30 min | Safe |
| 0c | Purge history. I recommend deleting and recreating the repo over rewriting history: simpler, and it also resets the 534 MB. | 1, 15 | 30 min | Founder decides |
| 0d | Revoke the Discord bot token and delete channels/discord/.env. |
13 | 10 min | Founder revokes, Ray deletes |
| 1. Restore what is silently dead | ||||
| 1a | Add an optional name= field to the manifest, cap slugs at 60 characters plus a hash, and make the runner record a failure when it cannot take a lock. Then force-fire house-clock (October) and books refresh to backfill. |
2 | 1.5 h | Safe |
| 1b | Fix the denial parser to select the "type":"result" line with jq. |
5 | 20 min | Safe |
| 1c | Make graph-reingest model-free: script plus qmd update && qmd embed, with a shell ops-log.sh call. Move sync-contacts to the current Haiku or to Sonnet, then confirm the ledger mtime advances. |
4 | 1.5 h | Safe |
| 1d | Add a postcondition to the runner: after exit 0, require an ops-log header for the slug since start time, else log a failure. | 4, 5 | 1 h | Safe |
| 2. Make alerts reach him | ||||
| 2a | One sitting at the Mac: grant Automation (Messages) and Full Disk Access to the dispatcher and cron process trees, then prove it with a dry-run send from each. This is the same ask as the pending limit-watchdog LaunchAgent offer. | 3 | 15 min + 1 h test | Founder hands |
| 2b | If the grants don't hold, have the dispatcher inject founder-facing jobs into the live session via tmux send-keys. |
3, 7 | 2 h | Safe |
| 2c | Have /morning-prep drain cron-undelivered/, delivering or expiring each file. Triage the 279 backlog once. Remove the dead messages-watchdog crontab line. |
3 | 1.5 h | Safe |
| 2d | Flip the re-arm hook to an allowlist: arm only when the restart script sets RAY_CHANNELS_AGENT=1. |
7 | 20 min | Safe |
| 3. Finish the model upgrade | ||||
| 3a | Put one tier map in one file, read by the dispatcher, runner and watchdogs. Smoke-test each ID with claude -p before switching. |
6 | 1 h | Safe |
| 3b | Set effort explicitly per job in the manifest instead of inheriting the interactive setting. I recommend medium for reporting and watch jobs, high for research. | 6 | 30 min | Founder confirms default |
| 3c | Update agents/fanout.md, the modelSettings key, and the delegation memory. |
6 | 20 min | Ray drafts, founder applies |
| 3d | Main-session model (Opus 5.5 or Fable 5.2). | 6 | none | Founder, after the bake-off |
| 4. Efficiency | ||||
| 4a | Deep-research: stop auto-filing follow-ups (the fix recommended 07-17), and decide whether it stays nightly. | 8, 9 | 30 min | Founder decides |
| 4b | Add a model-free precheck to open-threads-check (newest message is a Ray ask older than 60 minutes and not parked). It needs chat.db access, so it depends on 2a. |
8 | 1 h | Safe after 2a |
| 4c | Comment out the four sw-* fleet lines while Scribble Works is paused. Handoffs stay on disk. |
10 | 5 min | Safe; tell him |
| 4d | Retire or keep caf-architect. |
10 | none | Founder decides |
| 4e | Move changelogs out of SKILL.md into a sibling file, and have /improve append there under a size budget. Move the 22 off skills to skills-archive/. |
11 | 2 h | Safe |
| 5. Hardening and cleanup | ||||
| 5a | Dispatcher: catch-up window for monthly and weekly jobs, state written after the precheck, log moved to ~/.claude/logs/, timeout and effort moved into the manifest. |
12 | 3 h | Safe |
| 5b | Sweep stale text: "CLAUDE.md" to "AGENTS.md" in six skills and the runner context, plus the Discord remnants. Fix the backup gaps. | 13, 14 | 1.5 h | Safe, except the settings lines |
| 5c | Edit SOUL.md (rhythms, scratchpad line, "Boil the ocean"). |
13, 16 | 20 min | Founder; it is his voice |
| 5d | Sweep .bak files and one-off scripts into a dated archive directory. |
15 | 30 min | Safe |
Steps 0 and 1 come first because they are the only ones where harm is accruing daily. Step 2 before step 3: a model bump is hard to trust while the failure surfaces are blind.
Decisions for the founder
- Token and backup: rotate now, and recreate the repo or rewrite history. Also whether health, address and finance data should go to GitHub at all.
- macOS grants: ten minutes at the Mac, or accept session injection as the alert path.
- Tiers and effort: current Sonnet and Haiku for crons, and the default effort for reporting jobs.
- Deep-research: stop filing follow-ups, and nightly or not.
- Fleet: tear down
sw-*during the pause; keep or retirecaf-architect. - SOUL.md edits.
Not verified
- The
ray-configrepo's GitHub visibility (I made no outside calls). - The root cause of the Haiku denials.
- Whether every AppleEvent timeout is a permission problem rather than a Messages wedge.
- That
claude-sonnet-5-5andclaude-haiku-5-5resolve on this plan; hence the smoke test in 3a.