SOP: Verification-as-independent-worker pattern

The pattern

When Ray (or a sub-agent under Ray) produces an artifact, the same agent CANNOT reliably verify its own work. Confirmation bias makes self-review read defects too generously. The fix is structural, not motivational: route every verification through a separate fresh-eyes sub-agent that has zero context on the build.

This SOP is the umbrella for that pattern across non-visual surfaces. Visual surfaces are already covered (/design-critic for static pages, /video-critic for video).

Source

Harness-engineering book Ch 7.5: "verification worker is independent and skeptical (run tests, investigate errors, do not rubber-stamp)." Cited in the 2026-05-18 book mapping at [[~/rdco-vault/06-reference/2026-05-18-agentway-harness-engineering-claude-code-design-guide.md]].
Existing precedent: feedback_fresh_eyes_subagent_for_own_artifacts (filed 2026-05-07) — operationalized for visual surfaces via /design-critic and /video-critic. Gap #3 in the book mapping flagged the missing equivalent for non-visual surfaces.
Founder approval: 2026-05-19 with stacked dispatch pattern (pre-decomposition + implementation-notes).

Three surfaces, three skills

Surface	Skill	Drift to catch
Vault writes	`/verify-vault-write`	Unverified claims; weak RDCO mapping; missed sponsor disclosure; copy-paste-wall instead of paraphrase; citation gaps; broken cross-links
Strategic outputs	`/verify-strategic-output`	Overconfidence vs evidence; specific tickers/dollar-amounts without verification; founder-pushback-walked-back pattern not applied; false convergence; cherry-picked backtests
Sub-agent dispatch decisions	`/verify-dispatch`	Spec ambiguity; over-broad scope; missing stacked-pattern requirements; classifier-blocking-pattern misses (paper-trade auth, settings.json mods)

Each skill has its own SKILL.md at ~/.claude/skills/<skill>/SKILL.md. They share the structural shape of /design-critic and /video-critic: read the source-of-truth fresh, score mechanically against a rubric, return a structured PASS / ITERATE / SCRAP verdict with actionable suggestions but NO auto-rewrite of the artifact.

When to use each (tiering)

The tiering follows the implementation-notes SOP shape (REQUIRED / OPTIONAL / SKIP). Each skill applies the gate ONLY at the REQUIRED tier by default; OPTIONAL is invoke-when-uncertain; SKIP means no gate.

`/verify-vault-write`

REQUIRED: research briefs, decision pages, concept articles meant to anchor a thesis, anything filed to 06-reference/ with a source: frontmatter, anything filed by deep-research / research-brief / process-newsletter / process-youtube
OPTIONAL: working-context updates, ephemeral state files, journal-class entries, vault notes with ambiguous classification
SKIP: file moves, formatting-only edits, link-fix-only edits, index-only updates

`/verify-strategic-output`

REQUIRED: /decisions/ HTML pages, paper-trade allocation recommendations, multi-bet portfolio framings, any sub-agent report ending in DECISION: surface, anything that would commit dollars or tickers
OPTIONAL: bet-strategy.md updates with no new dollar-amount or ticker commitment, thesis-doc refreshes
SKIP: pure-research findings without recommendation, vault notes filed under "research" without strategic verdict, devil's-advocate exploration notes

`/verify-dispatch`

REQUIRED: any dispatch the implementation-notes SOP marks REQUIRED (build scripts, financial-execution scripts, settings.json mods, multi-step refactors, ≥5 design decisions); any dispatch that will write to a paper-trade or production system
OPTIONAL: research dispatches > 30min wall-clock budget, multi-stage sub-agent fan-outs (process-newsletter batch, deep-research)
SKIP: single-file content edits, trivial transformations, dispatches with no founder-facing output

Trigger pattern (on-demand vs auto-pipeline)

Two ways to invoke:

On-demand slash command — /verify-vault-write <path>, /verify-strategic-output <path-or-recent-msg>, /verify-dispatch <draft-prompt>. Use when the producer skill doesn't auto-invoke OR when the parent agent wants a verification gate before proceeding.
Auto-pipeline — producer skills that emit REQUIRED-tier artifacts MUST invoke the matching verifier as the final step before returning to the parent. Examples:
- /research-brief and /deep-research auto-invoke /verify-vault-write on each output note before returning the path
- /investing:build-thesis, /investing:backtest-thesis, anything producing a /decisions/ page auto-invokes /verify-strategic-output before surfacing the recommendation
- /verify-dispatch is on-demand only (auto-gating every dispatch would 10x dispatch latency for trivial work; explicit invocation is the discipline)

How a verification skill must run (the shape)

Every skill in this family follows the same shape. The shape is enforced because skill-shape drift is itself a failure mode.

1. Fresh-eyes dispatch (MANDATORY)

The skill MUST run as a sub-agent dispatch. The parent that produced the artifact CANNOT run the skill directly — confirmation bias on its own work makes self-review unreliable.

Wrong shape:

Parent: "I wrote this vault note, let me verify it."

Right shape:

Parent: dispatch subagent with subagent_type=general-purpose
  Input: artifact path + spec/rubric path
  Subagent: reads spec fresh, judges from zero, NO build context
  Output: structured VERDICT block
Parent: ingests verdict, NEVER second-guesses the verifier

The sub-agent receives the artifact + the rubric. It does NOT receive any "the build was clean" framing or any prior notes from the producer. If the rubric has bugs, those propagate — but that's a separate problem (locked rubric discipline) and the critic surfaces them too.

2. Read source-of-truth fresh

The verifier reads the rubric / spec / source-of-truth file on EVERY invocation, never from cached memory. Stale rubrics score against the wrong target.

3. Mechanical scoring

The verifier applies the rubric as a checklist, not as a reasoning prompt. Each rubric item is PASS / FAIL with a one-line reason. Avoids the "reasoning toward a charitable verdict" failure mode.

4. Verifier pre-flight checklist (meta-drift defense)

Before scoring, every verifier confirms:

I am a fresh-eyes sub-agent with zero build context. (Otherwise abort — parent's responsibility to dispatch correctly.)
I read the spec / rubric / source-of-truth fresh on this invocation.
I will apply the rubric mechanically and not reason my way to a charitable verdict.

This is the structural defense against the verifier itself developing confirmation bias under repeated PASS pressure (per /design-critic's "agreement bias" failure mode).

5. Structured verdict output

Single shape across all verifiers:

VERIFY-<SURFACE> VERDICT: <PASS | ITERATE | SCRAP>
Subject: <artifact path / dispatch slug>
Source-of-truth: <rubric path>

RUBRIC RESULTS:
- <rubric item>: <PASS / FAIL> — <one-line reason>
- ...

FINDINGS (only if ITERATE / SCRAP):
- <finding tag>: <description> — <specific fix recommendation>
- ...

TOP 3 ACTIONS (only if ITERATE):
1. <specific action>
2. <specific action>
3. <specific action>

The verifier returns this block to the calling parent. The parent decides whether the founder sees it.

6. No auto-rewrite

Verifiers flag and suggest. They DO NOT modify the artifact. The producer (or parent agent) implements the fix. This preserves the human-or-parent-in-the-loop discipline for the actual change.

7. Recursion stop

Verification depth = 1. Verifiers DO NOT verify each other. If /verify-strategic-output flags a finding, /verify-vault-write does NOT verify the flag. The parent acts on the finding directly OR escalates to founder.

The escalation path on a critic ITERATE is: parent agent acts on the suggestions OR escalates to founder. Not "dispatch another verifier."

Audit + escalation rules

Max 2 verify-iterate cycles per artifact. After cycle 2, escalate to founder with the artifact + the verifier's last verdict + the diff history. Don't loop indefinitely — the verifier is B-tier and can converge on a local maximum.
Same finding twice = escalate immediately. If two consecutive verifier verdicts return the same failing rubric item with the same finding, the producer isn't successfully addressing the feedback. Burning more cycles won't help.
Founder override beats the loop. If the founder explicitly says "stop verify-looping this and just ship it" — ship it. Direct override beats the gate.

Failure modes to watch (cross-skill)

Stale rubric — if the rubric file isn't updated as the surface evolves, the verifier scores against the wrong target. On every run, the verifier should check the date: frontmatter on the rubric and flag if older than ~60 days.
Agreement bias — if the verifier always returns PASS, it isn't actually verifying. After 5 consecutive PASS verdicts on a skill, run a deliberate SCRAP-test by invoking the verifier against a known-bad artifact and confirming it returns SCRAP. If it doesn't, the rubric isn't being applied.
Parent second-guessing the verifier — once the verifier returns ITERATE, the parent MUST treat that as ground truth and act on it. If the parent dismisses the finding ("the verifier missed context"), the entire gate collapses. The right move on disagreement is: surface BOTH the verifier's finding AND the parent's counter-argument to the founder for a tie-break.
Verifier-skill drift toward the producer's frame — if the producer writes the rubric and the verifier reads the rubric, the verifier is implicitly downstream of producer framing. Rubric files should be owned by a separate concern (the founder or a meta-skill), not by the producer skill.
False-positive blocking trivial work — if /verify-dispatch flags every dispatch, the discipline collapses to noise. Calibrate the REQUIRED-tier criteria conservatively; expand only when a SKIP-tier dispatch causes a real failure.

What this pattern does NOT do

Replace founder review on PASS verdicts. PASS means "good enough to surface" — not "ship without founder ever seeing." For founder-facing artifacts (decisions, public content, paper-trade deploys), the founder still has the final call; the verifier ensures every artifact he sees is worth his 30 seconds.
Make code changes or rewrite artifacts. The verifier reads, scores, and recommends. The producer (or a downstream build pass) implements the iteration.
Apply to research-mode exploration. Pure devil's-advocate notes, working sketches, and journal-class entries are SKIP. The pattern is for artifacts that are intended to be load-bearing or that ship to founder / external surfaces.

[[~/rdco-vault/06-reference/2026-05-18-agentway-harness-engineering-claude-code-design-guide.md]] — Ch 7.5 source-of-record + Gap #3 in the RDCO mapping
[[~/rdco-vault/02-sops/2026-05-18-implementation-notes-pattern-for-sub-agent-dispatches.md]] — the sibling SOP that this one operationalizes verification for
[[~/.claude/projects/-Users-ray/memory/feedback_fresh_eyes_subagent_for_own_artifacts.md]] — the original founder directive (2026-05-07) that motivated the visual-surface critics; this SOP extends the same principle to non-visual surfaces
[[~/.claude/projects/-Users-ray/memory/feedback_calibrate_overconfidence.md]] — the standing rule this whole family of skills operationalizes
[[~/.claude/skills/design-critic/SKILL.md]] — visual-surface analog (static pages)
[[~/.claude/skills/video-critic/SKILL.md]] — visual-surface analog (rendered video)
[[~/.claude/skills/verify-vault-write/SKILL.md]] — vault-write verifier
[[~/.claude/skills/verify-strategic-output/SKILL.md]] — strategic-output verifier
[[~/.claude/skills/verify-dispatch/SKILL.md]] — sub-agent-dispatch verifier

Changelog

2026-05-19 — SOP created. Closes Gap #3 from the harness-engineering book mapping. Three new skills (verify-vault-write, verify-strategic-output, verify-dispatch) ship alongside, mirroring the design-critic / video-critic shape for non-visual surfaces.

verification as independent worker pattern