Amazon engineer on agentic coding — "not reading the diff" as a North Star

Provenance and its limits

The founder relayed a block of quoted text from an Amazon engineer describing how their team works with AI coding agents. The author is unnamed and the quote is unverified — no link, no public post, no way to confirm attribution or that the practices described are team-wide rather than one engineer's. Treat every number in it as self-report.

Recorded here because the shape of the argument is useful independent of whether the specific claims check out. Do not cite this note as evidence that "Amazon does X."

What the source claims

Upstream is where human judgment lives. Heavy emphasis on getting system architecture and boundaries right before agents touch anything — domain models, API contracts, architecture, DB schemas.
Tests are the guardrail, and they are human-authored. TDD/BDD; PRD and acceptance criteria require human judgment, and so does the actual test code. The source states this explicitly as the guardrail.
Downstream signal is automated and layered. Their PR tool deploys the change to an alpha environment immediately, runs integration tests, then rolls back. Plus LLM reviewers, static linters flagging high cyclomatic complexity and excessive method parameters, and deterministic security scanners.
The signal is consumed by agents, not humans. The stated payoff: agents act on these signals and self-correct "without my ever knowing that the PR was even submitted yet."
Diff review has not been abandoned — yet. The source is explicit that the tools are not confidence-inspiring enough to ignore the diff, but believes that is the direction. Self-reports approving after the first agent pass ~90% of the time.
The thesis: "we should be engineering these agentic systems with not reading the diff in mind as a North Star."

Synthesis for RDCO

The sharp bit is not "stop reading diffs." It is the consumer of the signal.

RDCO already has a substantial critic layer — the verify-* family ([[verify-vault-write]], [[verify-strategic-output]], [[verify-dispatch]], [[verify-pdf-output]]), plus [[design-critic]], [[video-critic]], [[station-critic]], and the newer [[behavior-critic]]. Structurally these match the source's "LLM reviewers" tier.

The difference is where their output goes. Our critics return prose verdicts to the parent session — to a human-facing report. The source's signal tier returns machine-checkable failures to the producing agent, inside a loop that closes before any human is involved. Same instinct, different consumer, and the consumer is what produces the compounding effect he describes.

That is a real and specific gap, and it is more actionable than "add another critic." Candidate direction: critics that emit a structured PASS/FAIL plus a machine-readable defect list the producing agent is required to drive to zero before the artifact surfaces. [[station-critic]] already returns PASS/FAIL + confidence for a convergence loop, so the pattern exists in-house — it just is not the default across the family.

Independent validation of an existing rule. Point 2 is the same claim as the standing plan → tests → implementation discipline (tests must precede the code; an assertion written after the implementation passes by construction and proves nothing). Useful as outside corroboration of a rule adopted on internal reasoning.

Direct application to CAF-engine. The 2026-07-27 architecture writeup ([[2026-07-27-caf-technical-architecture-and-backlog]]) already carries a "PR-based workflow for any changes" bullet under CAF engine. This note gives that bullet concrete content for ticket-writing: what the automated signal tier should contain, and the requirement that its output be agent-consumable rather than human-report-shaped.

What does not transfer

The infrastructure is Amazon-scale. Deploy-to-alpha-then-rollback on every PR, maintained integration suites, and deterministic security scanners are large standing investments. The principle is portable; the stack is not. Any RDCO version has to be built from cheap signals we already have.
The 90% figure is not a measurement. It is one engineer's recollection of their own approval rate, with no denominator, no time window, and no definition of "first pass." It should not be carried forward as a benchmark or a target.
Unfalsifiable as stated. The claim that agents fix problems "without my ever knowing" is precisely the claim that cannot be audited from the outside — including by the person making it.

Open thread

Offered to the founder 2026-07-28: fold a signal-loop ticket set into the CAF backlog doc. Awaiting his call.