06-reference

every escaped ai autopilot

2026-11-03·reference·source: Every (Working Overtime)·by Katie Parrott
ai-autopilotautomation-complacencycognitive-forcing-functionsverificationevery-working-overtime

"How I Escaped AI Autopilot" — Katie Parrott / Every, Working Overtime (Nov 3 2026)

Why this is in the vault

A working essay on automation complacency with AI, grounded in real research (aviation automation studies, a fluency bias experiment, a 2021 cognitive-forcing-function trial, a 2026 AI-writing study), that independently validates several of RDCO's own verification patterns from outside the vault. Note: this issue is explicitly a "rerun" of Parrott's April 2026 essay ("We Need to Talk About AI Autopilot," not previously filed) repackaged under a new headline during Every's Think Week; content below is from the rerun as received.

The core argument

Parrott re-did a finished client assignment from scratch because she'd outsourced the work so completely to AI that her brain never registered having done it the first time. She traces this to automation complacency: 1990s cockpit-automation research found pilots stop cross-checking reliable autopilot even against contradicting instruments, and a 2010 review generalized the pattern — the more reliable a system gets, the less humans scrutinize it. AI compounds this with a fluency bias (a 1999 study found identical statements in easy-to-read fonts get rated as more true): polished, confident AI output triggers the same "looks finished, must be correct" reflex, and every model upgrade makes the illusion worse by removing the obvious mistakes that used to trigger scrutiny.

Her fix is three deliberate frictions, not more willpower: (1) write her own five-bullet position before asking the model, so she has something to argue with instead of floating along with the draft; (2) build in a time/surface gap between generation and review (draft one day, review another, or move the output to a different surface) because sustained review attention decays with repetition; (3) force herself to write one sentence justifying why she's accepting a specific AI output — she cites a 2026 study where requiring written justification cut mistaken acceptances roughly in half. She notes a 2021 trial found these "cognitive forcing functions" were the most effective intervention against AI overreliance and also the worst-rated by users, since the whole appeal of automation is avoiding exactly that friction. Her close: the risk isn't AI thinking for you, it's AI making it feel like you already thought.

Mapping against Ray Data Co

This is third-party research validation — not just founder intuition — for cognitive-forcing-functions RDCO has already institutionalized: the AGENTS.md hard rule "no batched result-declaration" (never declare a result in the same turn as the commands determining it; read literal output first) is Parrott's Change 1 and Change 3 combined — form your own view of the literal output before accepting the model's framing of it. The fresh-eyes critic pattern (verify-vault-write, verify-strategic-output, verify-dispatch, behavior-critic) is an institutional version of her Change 2 — distance and a different surface between generation and review, because the producer that drafted something has confirmation bias reading its own output. Where RDCO is weaker against this essay's argument: Change 1 ("write your own position before you ask the model") is baked into structured flows like station-spec-author, but ad hoc work — including this very newsletter-processing run — goes straight from fetch to classification with no forced pre-commitment of judgment first. Worth asking whether /process-newsletter should add a one-line forced "what do I expect this piece to argue, before I read past the subject line" step, mirroring Parrott's five-bullet pre-write.

Related

⚠️ Sponsorship

Mid-issue explicit sponsor block for "Traceable by Chargebee" (a virtual CFO-focused event), disclosed as a paid placement with its own CTA and advertisement_id in the tracking links — pure ad, no bearing on the essay's argument. Standard Every house footer also cross-promotes Every Agent (inline, mid-essay: "Stop explaining AI to your team and start showing them") and the Sparkle/Cora/Spiral/Monologue bundle. None of this affects the essay's content, which argues against automation overreliance generally rather than for any specific Every product (Proof is mentioned once, descriptively, as an example of the "design problem" — not pitched).