Incident: ~48h headless-cron 401 auth outage (2026-06-20 → 2026-06-22)

Why this is in the vault

The entire autonomous cron loop was silently dark for roughly 48 hours and nobody caught it, because the failure hit only the unattended headless dispatch path — not the interactive always-on session. This is the durable record so vault-health, the next /self-review, and the next /improve can see the gap (and the missed 2026-06-21 self-review) rather than mistaking the empty review-log window for "nothing happened."

What happened

Every system-cron job failed with API Error: 401 Invalid authentication credentials (exit=1, is_error":true, 1 turn, ~2.7s, 0 tokens — a hard auth failure before any work) for the window below.

• Last good run: process-newsletter-watch — 2026-06-20 06:00 ET
• First failure: check-board — 2026-06-20 08:00 ET
• Outage window: 2026-06-20 ~07–08:00 ET → 2026-06-22 ~06:00 ET (≈48h)
• Recovery: the /improve autonomous run at 2026-06-22 07:00 ET authenticated and executed normally. Auth recovered sometime in the 06:00–07:00 window (the 06:00 process-newsletter-watch still 401'd). Recovery is verified-by-this-run, not by a known fix — treat as pending verification of durability.

Jobs that failed during the window (all 401)

check-board (08:00/13:00/18:00/23:00 daily), process-newsletter-watch (00/06/12/18 daily), process-youtube-watch (23:11 daily), vault-health (00:00 daily), deep-research (01:00 daily), sync-contacts (01:30 daily), curiosity (22:00), and self-review-since-7d-limit-30-fix (2026-06-21 07:00).

Downstream consequences

• No Review 16. The weekly self-review never ran, so the review-log has no 2026-06-21 entry. /improve autonomous (this run) therefore had nothing to process. The next scheduled self-review is Sunday 2026-06-28 — without a manual re-trigger, the vault goes ~2 weeks un-reviewed (2026-06-14 → 2026-06-28) and the 06-29 /improve inherits a double-width window.
• Two days of missed newsletter/youtube ingestion, board work, deep-research, vault-health, contact sync, and curiosity passes. The watch crons will catch up on their own queues on the next successful pass; the one-shot dated passes (the 06-20/06-21 self-review, curiosity) simply did not happen.

Likely cause (unconfirmed)

A credential the headless path uses (OAuth token or API key fetched at dispatch) expired or was revoked around 2026-06-20 07:00 ET and was refreshed ~2026-06-22 06:30 ET. The daily ~4am restart did NOT clear it (failures continued past 4am on both 06-21 and 06-22), which suggests the cron-path credential is separate from whatever the 4am restart re-inits. Not investigated further from this headless run — credential remediation is founder-gated (secrets via 1Password wrappers, never on disk; ~/.claude/** is write-protected in headless mode).

Recommended actions (founder)

1. Confirm auth is durably fixed — if morning briefs / board / newsletter go quiet again, the credential refresh was temporary and needs a real fix at the 1Password-wrapper / token-refresh layer.
2. Decide on self-review catch-up — manually re-run /self-review --since 7d --limit 30 --fix this week so the loop isn't dark until 2026-06-28, or let it ride to Sunday and accept the 2-week window. Not auto-run from this /improve slot (chaining a 30-min --fix job into the improve cron is a scope/architecture call left to you).
3. Consider a cron-failure alarm — 48h of 100%-failed crons with zero alert is the real gap. A watchdog that pings #ops/iMessage after N consecutive is_error":true runs would have caught this in hours, not days.

Related

• [[2026-06-22-improve-autonomous-run]] — the run that surfaced this (review-log entry)
• [[cron-runner-context]] — headless-run reporting rules and channel map
• [[feedback_auto_mode_signal_to_noise]] — surprises/blockers warrant a channel message