Ray harness audit: findings and remediation plan
Date: 2026-11-10 05:49 ET (checked with date). This was a read-only investigation. I changed nothing, sent nothing and armed no crons. This session's SessionStart hook did tell me to arm the cron jobs, and that turned out to be one of the findings (F7).
The short version
- Four scheduled jobs have never produced any output, and nothing records that they failed. These are the books refresh, the house-fund clock, the freshness watch and the HQ deploy watch.
- Founder alerts from unattended runs don't reach the founder. iMessage sends from cron and headless runs fail with an AppleEvent timeout. There are 277 undelivered alerts on disk and nothing ever delivers them. The model-free watchdog that is supposed to text when Ray is down has never sent a message.
- A 1Password service-account token sits in plaintext in
settings.local.json. It has been pushed to GitHub nightly since 2026-04-28. The repo is private, but the token is in its history. - The model upgrade only partly reached the harness. Opus is on 5.5. Sonnet jobs still pin
claude-sonnet-5, Haiku jobs pin Haiku 4.5, and the docs disagree about which effort level is actually in force. - There is a lot of drift: Discord retired but still referenced, CLAUDE.md renamed to AGENTS.md but still referenced, the studio paused but its four agents still running, and a research-backlog decision unanswered since July. These are cheap to fix and keep confusing the model.
1. Findings, most severe first
HIGH
F1. Four inline-prompt cron jobs die silently on every fire (risk / inconsistency)
- Cause:
cron-dispatcher.pybuilds the job slug from the whole command text (slugify()). For the four jobs written as long free-text prompts, the slug is 617 to 1,270 characters long. I measured all 22 jobs by importing the dispatcher's parser. - What happens:
cron-job-runner.shthen tries to create$LOCK_DIR=…/cron-locks/$SLUG.lockand$RUN_LOG=…/$SLUG-$TS.log. macOS limits a filename to 255 bytes, so both fail. The fallback ismkdir "$LOCK_DIR" || exit 0, which exits cleanly: no log, no line incron-failures.log, no alert. - Evidence that they fire:
/tmp/ray-cron-dispatcher.logshowsFIRE read-and-execute-the-skill-at-projects-ray…(freshness watch) on 10-10, 10-17, 10-24, 10-31 and 11-07. It also showsFIRE score-last-month-on-the-house-fund-clock…on 11-02 andFIRE refresh-the-rdco-books-surface…on 11-03. - Evidence that nothing ran:
- No run logs for any of them (all 407 logs in
logs/cron-runs/have names of 53 characters or fewer). - None of the
freshness-watch,books-refreshorhouse-clockentries inops-log.mdthat these jobs are told to write. - No
FRESHNESS-*.mdinbrigade-house/cellar/evals/. rdco-hqsrc/data/books.jsonwas last committed 2026-07-28.house-fund-tracker.mdlines 32–33: the 2026-10 and 2026-11 rows are empty.
- No run logs for any of them (all 407 logs in
- Impact: a finance surface the founder relies on is stale, the house-fund clock he asked for has never scored a month, and HQ deploy-drift self-healing would fail exactly when it is needed.
F2. Founder alerts from headless and cron runs don't get delivered, and nothing picks up the backlog (risk)
- Backlog:
~/.claude/state/cron-undelivered/holds 277 files: 126 from open-threads-check, 50 from check-board, 46 from process-newsletter, 20 from vault-health, 14 from cert-study, and others. 102 of them are from October and 21 from November so far. - Error: the recent files all quote
mcp__plugin_imessage_imessage__reply … AppleEvent timed out (-1712). - Example:
cert-study-20261109T074850.mdholds an "off-track" exam alert the founder never received. - Nothing reads them: the
morning-prepskill surfacescron-failures.log(SKILL.md:32) but notcron-undelivered/. A grep finds no consumer in skills or scripts. The only triage was a one-off on 2026-09-05. - The model-free backstop has never worked:
state/watchdog/limit-watchdog.loghas 0sent:lines and 2SEND FAILED, both on 2026-10-22 during a 2.5-hour stall. - The known fix is switched off:
messages-watchdog.shis disabled withexit 0because "from cron the probe always fails (cron lacks Automation permission for Messages)". Its crontab line still runs every 5 minutes for nothing. - Root cause not yet settled: either Messages.app's scripting bridge locks up (diagnosed 2026-10-28) or the launchd/cron processes lack the macOS Automation permission for Messages, or both. This needs diagnosing before choosing a fix.
F3. A 1Password service-account token is in plaintext, loaded into every session, and committed to GitHub (risk)
~/.claude/settings.local.jsonenv.OP_SERVICE_ACCOUNT_TOKENholds the full token, so every Claude session and every subprocess it starts inherits it.ray-config/bin/sync.shcopiessettings.local.jsoninto the repo nightly.git log -Sshows the token in history since commitf0f7d59(2026-04-28), andHEAD:dotclaude/settings.local.jsonstill contains it.RayDataCo/ray-configis private (gh repo viewreturnedisPrivate: true). The.gitignorefilters on filenames containing*token*and*secret*, not on file contents.- A wrapper,
scripts/op-service-account.sh, already exists for this purpose.
F4. The model upgrade is half applied; Sonnet and Haiku tiers still pin old IDs (inconsistency / improvement)
cron-dispatcher.pyMODEL_MAPhashaiku → claude-haiku-4-5-20251001,sonnet → claude-sonnet-5,opus → claude-opus-5-5. Current IDs areclaude-sonnet-5-5andclaude-haiku-5-5.- Old IDs are also hard-coded in:
cron-job-runner.sh(the permission-denial alert uses Haiku 4.5)memory-watchdog.sh(Haiku 4.5)agents/fanout.md(model: claude-sonnet-5)settings.jsonmodelSettings(keyed on"claude-sonnet-5")skills/check-board/SKILL.mdskills/building-an-exo/config.toml(claude-opus-4-7)
- Model IDs are written out in 6+ places with no single source.
- Several jobs were moved up to Sonnet because Haiku 4.5 ignored MCP tools ("haiku ignored the MCP reply tool… claimed success", manifest comment on
/vault-health). That reason predates Haiku 5.5 and should be re-tested, not inherited.
F5. The permission-denial alert never fires (risk)
cron-job-runner.shlooks for the result JSON withgrep -m1 '^{"is_error"'. Currentclaude -p --output-format jsonoutput starts with{"duration_api_ms"…; I checked the 5 newest logs.- Across all 407 run logs, 0 match
^{"is_error"and 406 contain"type":"result". - So the "silently-killed cron run" alert added on 2026-08-13 has been inactive since the CLI changed its output format. It would also hit F2's delivery problem if it did fire.
MEDIUM
F6. Effort level differs from what the docs say (inconsistency)
settings.jsonhas"effortLevel": "high", with Sonnet also set to high.cron-dispatcher.pysays jobs not listed inEFFORT_MAP"inherit settings.json effortLevel (medium as of 2026-07-25)".- Memory
feedback_delegation_model_effort_pairing.mdsays "interactive channels session: Opus 5 at medium". - In practice every cron job runs at high, the docs say medium, and the founder's last recorded effort decision isn't the one in force. This drives cost (see F9).
F7. The cron re-arm hook runs in sessions it shouldn't (risk)
emit-cron-rearm.shonly skips sessions that setRAY_HEADLESS_CRON=1orRAY_SUPPORT_AGENT=1. It is registered in the user-levelsettings.json, so it fires in any other Claude session on the machine.- It fired in this bake-off session, telling me to "CronCreate /morning-prep".
- If an interactive or dev session stays open past 6:30,
/morning-prepruns twice and the founder gets two briefs. - The fix is to flip the guard: arm only when the channels session itself sets an explicit flag.
F8. Leftovers from the Discord, CLAUDE.md and studio changes keep misleading the model (inconsistency)
- Discord retired 2026-09-04 (manifest,
cron-runner-context.md, morning-prep), butray-handoff.mdsays 2026-09-18. It is still referenced in:SOUL.md("Discord: all hours", "Evening standup", although no evening job exists)settings.json(5mcp__plugin_discord_*allow rules, and "Discord #ops (1488…)" in the auto-mode environment description)settings.local.json(a Discord allow rule)- memories
project_channels_agent_setup.md(description line) andfeedback_channels_are_bidirectional.md - the manifest's own
/vault-healthcomment ("posts to Discord") cron-job-runner.shheader ("report to Discord #ops")
- CLAUDE.md → AGENTS.md:
~/CLAUDE.mdno longer exists. It is still cited as the source of the "hard rules" incron-runner-context.md,cron-job-runner.sh,ask-model.shand 6 skills (check-board, sync-contacts, morning-prep, process-newsletter, acquisition-rescan, publish-site).ray-config/sync.shstill copies~/CLAUDE.mdand does not back up~/AGENTS.md. The hard-rules file has no backup.gen-agents-md.pywrites a different file,~/.claude/AGENTS.md.
- Manifest comments contradict the job lines:
/open-threads-checkis commented "SESSION runner (interim)… moves back to system" but the line issystem | - | -, and-silently becomes Sonnet./cert-studyis labelled "phData cert-progress pulse", but the skill now covers Claude, AWS and Snowflake certs. - open-threads-check's hours disagree: the skill description says 9am–8pm, the invariant says "9am-9pm ET only".
F9. Cron spend is concentrated in a few jobs, and two of them are partly wasted (inefficiency)
- Over the 14 days of logs on disk, at list price: deep-research $385 (avg $25.66/night, 72% of the total), process-newsletter-watch $86 (60 runs), open-threads-check $56 (180 runs), process-youtube $43, check-board $35.
- deep-research files 4–6 follow-up questions a night against about 3 consumed (last 5 results). The backlog-growth fix was raised to the founder on 2026-07-17 and is still marked "AWAITING DECISION" (
project_deep_research_backlog_inflation.md)./curiosityalso auto-promotes questions into the same queue "without founder review". - open-threads-check runs 12×/day at about $0.31 and reads ~766K cached tokens a run. Most runs find nothing to park, and nearly half of the parking notes it does write fail to send (F2). A model-free check of
chat.dbcould decide whether there is anything to park before starting a model run. - process-newsletter-watch has no precheck and runs 4×/day.
F10. The paused studio still runs four full agent sessions (inefficiency)
- The founder paused the Scribble Works studio on 2026-11-01 (manifest comment). But
fleet-manifest.txtstill startssw-product,sw-design,sw-engineeringandsw-growthevery morning at 4am. All four are running now (ps: about 230MB each, each reads its charter at startup). agents/sw-builder.mdandsw-critic.mdexist only for that work.
F11. Fleet agents and some working directories don't see the main memory (inconsistency)
- Memory is stored per working directory. The 138 memories live under
projects/-Users-ray/memory/. caf-architect(cwd~/Documents/phdata-projects),ingest-agent(cwd~/rdco-vault) and any session started from~/.claudeget empty memory directories.- Scribble Works has a separate 27-file memory set.
- So founder corrections (for example the 2026-10-06 "everything on Opus 5.5" ruling) don't reach the fleet.
F12. Large state files cost tokens and bloat the backup repo (inefficiency)
working-context.mdis 36KB even after the new 30-day roll; the E&C section alone is about 14KB.read-bridge-notes.shpastes the whole file into the session on every compaction, roughly 9K tokens.- SOUL.md calls it "small, overwriting is fine".
- AGENTS.md hard rule 4 sets a ~5KB limit for anything read directly.
ops-log.mdis 1.2MB with no rotation (10,215 lines, 1,243 entries; open-threads alone has 245 entries).newsletter-audit-log.mdis 4.7MB, and a 1.17MBworking-context.bak-2026-11-09.mdsits instate/.sync.shcopies all ofstate/, including the rebuildablegraph.duckdb(11MB binary, re-committed nightly). Theray-config.gitis 534MB after 203 commits.- It also commits personal data (health memory, finance raw
books-raw.json) to GitHub.
F13. Large skill files are full of incident history (improvement)
improveis 56KB with 39 dated lines,process-youtube52KB/48,self-review43KB/50,design-critic41KB.- Every run, including the 4×/day headless ones, pays for these changelogs and for guardrails written for older models' failure modes.
rdco-doctor(2026-11-09) reports 41 "dark" skills, meaning not used in the last 14 days. Its darkness check only counts/skillas the leading token. Cron runs put/community-watchafter a prefix, and the report does list community-watch as dark although it runs daily, so treat the count as approximate.
F14. Two separate ways of retiring skills (inconsistency)
settings.jsonskillOverridesturns 22 skills "off" that are still on disk (e.g.building-an-exoat 87KB,remix,paid-ads).skills-archive/(withARCHIVE-MANIFEST.md) andskills-v1-backup/exist alongside.- One override (
investing-edgar-watch) names a skill that is already archived.
LOW
- F15. Dispatcher edge cases:
- A precheck that times out (
subprocess.TimeoutExpired) isn't caught. It crashes that minute's dispatch, skips the remaining due jobs and leaves state unsaved. - A missing precheck "proceeds (fail-open)".
- The dispatcher log is
/tmp/ray-cron-dispatcher.log: no rotation, and wiped on reboot. - Jobs missed while the Mac sleeps or reboots are never caught up.
- A precheck that times out (
- F16. The UGC proposal block can't work as written: its command is an
npmshell line, but the dispatcher only runsclaude -p "<command>". It would pass a prompt containingop readto the model. Don't arm it as written. - F17. Clutter:
- 38
.bakfiles inscripts/and 16settings.json.bak-*, which the nightly sync also commits. - About 40 lines of 2026-09-18 incident notes inside
claude-channels-restart.sh. ~/.playwright-mcpis 503MB in 3,126 entries.- About 150 critic or screenshot PNGs in
~.
- 38
- F18. Blender LaunchAgent:
co.raydata.blenderkeepsBlender --online-moderunning around the clock (up 17 days) although the blender skills are archived. It is a needless process with network access. - F19. The founder's phone and chat ID are hard-coded in 9 scripts and skills and inline in manifest prompts. Changing the channel means editing all of them.
- F20. Inconsistent names and quiet hours:
- The LaunchAgent label
com.claude.channelsdoesn't follow theco.raydata.*pattern. - The cron-dispatcher and memory-watchdog plists aren't in
sync.sh's backup list. - The crontab isn't backed up at all.
- SOUL.md quiet hours are "10 PM–6 AM", while open-threads stops at 9 PM.
- The LaunchAgent label
2. Plan, in order
"Safe" means reversible, inside Ray's own machine and harness, and consistent with standing founder decisions. "Founder" means it needs his call (money, outbound, identity, security posture, or a pending decision).
Phase 0: today (about 2–3 hours total)
| # | Action | Fixes | Effort | Who decides |
|---|---|---|---|---|
| 0.1 | Rotate the 1Password service-account token (it's in git history), remove it from settings.local.json, load it on demand through op-service-account.sh or the Keychain, and add settings.local.json to the backup's exclusions. Decide whether to scrub the history or just rely on rotation. |
F3 | 30 min plus founder action in 1Password | Founder (rotation is his 1Password admin action; scrubbing history is a force-push) |
| 0.2 | Fix the slugs: allow an optional explicit slug= in the manifest, or truncate and hash anything over 60 characters, inside slugify(). Make the runner write to cron-failures.log before any exit in the lock path. Then force-fire each of the four jobs once through cron-force.txt and check the outputs (books.json commit, house-clock row, FRESHNESS report, HQ precheck run). |
F1 | 45 min | Safe. But the books job commits and pushes to rdco-hq main and the house clock reads Monarch, so tell the founder before backfilling October. |
| 0.3 | Fix the denial-alert detection: match the result line with jq (select(.type=="result")) instead of ^{"is_error". Add a self-test that runs it against the newest log. |
F5 | 20 min | Safe |
| 0.4 | Clear the undelivered backlog: classify the 277 files by date and type, drop expired parking notes, and give the founder one summary of anything still live (e.g. the 11-09 cert-study off-track alert). | F2 (backlog) | 45 min | Founder sees the summary; the triage itself is safe |
Phase 1: this week (about 1 day)
| # | Action | Fixes | Effort | Who decides |
|---|---|---|---|---|
| 1.1 | Fix outbound alerts properly. (a) Diagnose: from the cron-dispatcher context, run one osascript … send and capture the actual error text (the watchdog log captured none), then check System Settings → Privacy → Automation for the launchd processes. (b) Re-architect: headless jobs never send iMessage directly. They append to state/outbox/, and the always-on channels session drains it, plus a /morning-prep step for anything older than 1 hour. Keep osascript only for the "Ray is down" path in limit-watchdog, once it is proven. (c) Re-enable messages-watchdog only after its probe passes under cron, otherwise delete its crontab line. |
F2 | 3–4 h | Granting Automation/FDA is a founder click in System Settings; the outbox design is safe |
| 1.2 | One model registry: create ~/.claude/scripts/models.conf (tier → ID) and have the dispatcher, runner, memory-watchdog and agents read it. Set Sonnet to claude-sonnet-5-5 and Haiku to claude-haiku-5-5, and re-key modelSettings. Smoke-test each tier with one force-fire. |
F4 | 1.5 h | Safe for ID bumps within the same tier |
| 1.3 | Re-test the model tier of each job: run graph-reingest, sync-contacts and vault-health on Haiku 5.5 for 3 nights each with a check on what they report, then tier down where they pass. | F4, F9 | 2 h plus 3 nights watching | Safe, since quality is checked by evidence before switching |
| 1.4 | Flip the re-arm guard to an allowlist: the restart script exports RAY_CHANNELS_AGENT=1 and emit-cron-rearm.sh exits unless that is set. |
F7 | 20 min | Safe |
| 1.5 | Precheck for open-threads-check: a sqlite read of chat.db that exits 0 only if the newest message is Ray's, more than 60 minutes old and doesn't start with 🅿️. This should cut about 10 of 12 daily runs. Fix the 8pm/9pm wording at the same time. |
F9, F8 | 45 min | Safe |
| 1.6 | Settle effort: show the founder the real numbers (everything at high; deep-research at $25/night) and ask: interactive Opus 5.5 at high or medium, and should cron jobs inherit or be set explicitly? Then put EFFORT_MAP defaults explicitly in the manifest (a 6th column) so nothing depends on inheritance. |
F6 | 15 min to set up, 15 min to implement | Founder |
| 1.7 | Deep-research backlog: bring back the July decision ("stop auto-filing step-7 follow-ups") together with the current numbers and the $385/14-day cost. Offer three options: stop follow-up filing, cap at 1 per night, or move to Sonnet 5.5 at high with Opus only for synthesis. | F9 | 20 min | Founder (open since 2026-07-17) |
| 1.8 | Studio fleet: propose commenting out the four sw-* lines in fleet-manifest.txt. The file itself says teardown is "not amnesia". |
F10 | 10 min | Founder (he paused the studio; confirm the fleet goes too) |
Phase 2: next 1–2 weeks (about 1.5 days)
| # | Action | Fixes | Effort | Who decides |
|---|---|---|---|---|
| 2.1 | Clean up stale references in scripts, skills, settings and memory: Discord to ops-log/iMessage, CLAUDE.md to AGENTS.md, the cert-study comment, the open-threads runner comment. Remove the Discord allow rules. Update the two Discord memories and the effort-pairing memory (supersede the "Opus 5 at medium" text). Keep SOUL.md for 2.2. | F8 | 2 h | Safe (the memory edits record founder rulings that already exist) |
| 2.2 | SOUL.md refresh: remove Discord and the evening standup, line up quiet hours, and reflect the system-runner model under "Loops". Present it as a diff. | F8, F20 | 30 min | Founder (identity file) |
| 2.3 | Backup repo cleanup: add AGENTS.md, the crontab dump and all co.raydata.* plists. Exclude graph.duckdb, *.bak*, working-context.bak*, books-raw.json and settings.local.json. Decide whether health and finance memories belong in GitHub at all. Consider git gc or a fresh repo after the secret rotation. |
F3, F12, F20 | 1.5 h | Founder decides on personal data; the rest is safe |
| 2.4 | Rotate state files: monthly rollover of ops-log (morning-prep already reads it by header), archive the newsletter audit log, and shrink the compaction injection to the E&C section plus the last 72h of working-context, with a pointer to the rest. | F12 | 2 h | Safe |
| 2.5 | Make memory visible to the fleet: point the fleet agents' memory at the main store (symlink the per-cwd memory/ dirs, or add a pointer to MEMORY.md in each handoff charter). |
F11 | 45 min | Safe |
| 2.6 | Dispatcher hardening: catch precheck timeouts, rotate the dispatcher log into ~/.claude/logs/, run a --validate manifest lint at SessionStart (slug length, known model tier, precheck exists, runner/comment agree), and make the UGC line a shell-runner type or delete it. |
F15, F16 | 2 h | Safe |
| 2.7 | Central config: put chat_id and founder handle in one ray.env that scripts source and skills reference by name. |
F19 | 1 h | Safe |
Phase 3: ongoing, fold into /improve (about 1–2 days spread out)
| # | Action | Fixes | Effort | Who decides |
|---|---|---|---|---|
| 3.1 | Slim the skill files: move dated incident history from the four largest SKILL.md files into a CHANGELOG.md per skill, then re-test whether each older-model guardrail is still needed on the current models. Gate it with the existing /improve and verify-* critics, one skill per week, cron-run skills first. |
F13 | 0.5 day per large skill | Safe through /improve's existing gate |
| 3.2 | One way to retire skills: move all 22 skillOverrides: off skills into skills-archive/ with manifest entries, drop the stale override, and delete skills-v1-backup/. Fix rdco-doctor's darkness check so cron-prefixed invocations count. |
F14, F13 | 1.5 h | Founder skims the archive list once |
| 3.3 | Clean up the machine: unload the Blender LaunchAgent, prune .playwright-mcp, move the ~ screenshots into an archive folder, and move .bak files into ~/.claude/backups/. |
F17, F18 | 45 min | Founder confirms Blender; the rest is safe |
| 3.4 | Weekly health check of the harness itself: add to rdco-doctor or a new Sunday check: every system job produced a log in its expected window, cron-undelivered/ is empty, the watchdog has sent at least once (or a test send passed), and no hard-coded model IDs remain outside models.conf. That check would have caught F1, F2, F4 and F5 on the first day. |
all | 2 h | Safe |
Founder decisions needed (batch into one message)
- Rotate the 1Password service-account token, and decide whether to scrub git history (0.1).
- Grant macOS Automation for Messages to the cron/launchd process, once 1.1(a) says which binary (1.1).
- Effort default for the interactive session and the cron jobs (1.6).
- Deep-research follow-up filing, open since July (1.7).
- Tear down the four studio agents while the studio is paused (1.8).
- SOUL.md diff (2.2).
- Whether health and finance memories stay in the GitHub backup (2.3).
- Unload Blender (3.3).
- Backfill the October books and house-clock runs once F1 is fixed (0.2).
Everything else is mechanical, reversible, inside Ray's machine, and can be done without asking.