02-sops

multi agent pipeline config schema

2026-05-12·sop·status: draft-for-founder-review
sopschemamulti-agentpipelineyamlaxesconfigsrlhf

Multi-agent pipeline - config + axis schema SOP

This SOP documents the YAML schemas the multi-agent skill build-out pipeline depends on. Two artifact types are defined here:

  1. Per-domain config at ~/rdco-vault/01-projects/skill-pipelines/configs/<domain>.yaml - one per skill build-out domain (e.g. website-discovery.yaml, mac-matrix.yaml, haiku-generator.yaml).
  2. Axis fragment at ~/rdco-vault/01-projects/skill-pipelines/axes/<axis-name>.yaml - one per reusable critic axis (e.g. frontmatter-completeness.yaml, voice-match.yaml). Domains compose configs by referencing axis fragments by name.

Both schemas implement the founder-locked design choices from the architecture doc:

The shape encoded here is the topology described in [[../06-reference/concepts/2026-05-12-rdco-pipeline-rlhf-shaped]] - per-axis YAML fragments are structurally the per-principle entries in a Constitutional AI constitution; the structural checks are the deterministic property checks the critic does not have to make judgment on; the reference exemplars are the canonical-set the critic compares fuzzy axes against.

1. Per-domain config schema

File location: ~/rdco-vault/01-projects/skill-pipelines/configs/<domain>.yaml

The domain slug matches the workflow skill name (e.g. config for ~/.claude/skills/build-haiku-generator/SKILL.md is configs/haiku-generator.yaml).

Required fields

Field Type Required Description
domain string yes Short slug for the build domain. Must match the file name without .yaml and the trailing portion of the workflow skill name.
canonical_set_path string or null yes Path (vault-relative or absolute) to the folder of reference artifacts the critic compares fuzzy axes against. Set to null when in bootstrap mode (founder is the critic; no canonical set exists yet).
spec_template string yes Path to the markdown template the spec-author seat fills out. Lives next to the workflow skill or in ~/rdco-vault/01-projects/skill-pipelines/templates/.
critic_axes list of strings yes Names of axis fragments the critic fans out across. Each entry MUST resolve to a file at ~/rdco-vault/01-projects/skill-pipelines/axes/<name>.yaml. Soft target 3-7 entries; a warning logs if length > 7.
convergence_criteria string yes Natural-language description of when the loop stops. Convention: all axes return PASS with confidence >= high. Variants are allowed but must be parseable by the workflow command.
max_iterations integer yes Hard upper bound on convergence-loop iterations. Tiered failure rules apply: at floor(max_iterations / 2) log a warning to the run dir; at max_iterations archive the run and open a decision page.
reference_artifacts_path string or null yes Folder of 3-5 known-good exemplars the fuzzy axes consult. May equal canonical_set_path or be a subfolder of it. Null in bootstrap mode.
structural_checks list of strings yes Deterministic property-list checked before any LLM critic axis fires. Each entry is a short imperative the workflow command can mechanically verify (e.g. has 3 lines, total syllables 17, frontmatter has type: reference). Failures short-circuit the loop without burning critic-axis subagent dispatches.

Optional fields

Field Type Description
description string One-line human-readable summary of what the domain produces.
bootstrap boolean When true, the workflow routes critic verdicts to the founder for labeling instead of running the autonomous critic loop. Defaults to inferred-from-canonical_set_path: null.
triggering_surfaces list of strings Subset of [cli, notion-board, parent-subagent, decision-page-clickback]. Defaults to first three; the fourth is v1.1.
warnings_path string Override for where N/2 warnings get logged. Defaults to runs/<domain>-<timestamp>/warnings.md.

Validation rules

Inline example: haiku-generator config

This is the toy smoke-test domain. Lives at configs/haiku-generator.yaml.

domain: haiku-generator
description: Generate a 5-7-5 syllable haiku given a topic prompt. Smoke-test domain for the pipeline architecture.

# Bootstrap mode: no canonical set exists yet. Founder is the critic
# for the first N runs until per-axis label count crosses the
# critic-graduation threshold.
canonical_set_path: null
reference_artifacts_path: null
bootstrap: true

spec_template: ~/rdco-vault/01-projects/skill-pipelines/templates/haiku-spec.md

critic_axes:
  - syllable-count
  - image-coherence
  - novelty

convergence_criteria: all axes return PASS with confidence >= high

max_iterations: 5

structural_checks:
  - has 3 lines
  - total syllables 17

triggering_surfaces:
  - cli
  - notion-board
  - parent-subagent

2. Axis fragment schema

File location: ~/rdco-vault/01-projects/skill-pipelines/axes/<axis-name>.yaml

Each axis fragment captures the prompt, pass signal, and accumulated-label state for one critic dimension. Per the RLHF-shape concept doc, an axis fragment is structurally equivalent to one principle in a Constitutional AI constitution: a natural-language critic instruction with calibrated examples.

Required fields

Field Type Required Description
name string yes Slug matching the filename. Must match the entries referenced from per-domain configs' critic_axes list.
prompt string (multi-line) yes Natural-language critic instructions. Tells the critic-axis subagent what to look for, what counts as PASS, what counts as FAIL. This IS the codified founder feedback for this axis.
pass_signal string yes Short imperative description of what PASS looks like, for fast-skimming when reviewing axis fragments.
confidence_threshold string yes When a critic axis returns PASS, what counts as "high confidence" vs "PASS but weak"? Per section 3.4D hybrid - PASS+weak surfaces to founder visibility without blocking. Convention values: strict (anything less than perfect = weak), moderate (clear pass = high, edge cases = weak), lenient (any pass-shaped reasoning = high).
label_count integer yes Current count of accumulated founder labels for this axis. Starts at 0 on creation. Critic-graduation threshold is ~10 (per RLHF concept doc section 2). At label_count >= threshold, the axis can transition from founder-as-critic mode to autonomous-critic mode silently.
example_pass string (multi-line) yes 1-2 short illustrative examples of artifacts that PASS this axis.
example_fail string (multi-line) yes 1-2 short illustrative examples of artifacts that FAIL this axis, with one-line rationale for why.

Optional fields

Field Type Description
axis_type string One of mechanical (deterministic, could in principle be a structural_check), fuzzy (judgment-laden, requires LLM critic), or hybrid. Defaults to inferred from prompt shape.
graduation_threshold integer Override for label_count needed to graduate. Defaults to 10.
last_updated_by_improve date Set by the /improve consolidation hook when it rewrites the prompt based on accumulated labels. Audit trail for critic prompt evolution.
notes string Free-form notes - rationale for the axis, drift observed, etc.

Validation rules

Inline example: frontmatter-completeness (mechanical axis)

name: frontmatter-completeness
axis_type: mechanical
prompt: |
  Check the artifact's YAML frontmatter for required keys. The required
  set depends on the artifact type (`type: reference` vs `type: sop` vs
  `type: concept`), enumerated below. PASS when all required keys are
  present AND have non-empty values. FAIL when any required key is
  missing or has an empty/placeholder value.

  Required keys by type:
  - reference: date, type, source, author, newsletter_format, sponsored, tags
  - sop: date, type, project, status, tags
  - concept: date, type, status, source, tags
  - architecture-doc: date, type, project, status, tags

  Edge cases:
  - tags MUST have >= 2 entries (single-tag artifacts fail)
  - sponsored: true REQUIRES sponsor_entity field also present
  - placeholder values like "TBD", "TODO", "<fill in>" count as missing

pass_signal: All required frontmatter keys present, non-empty, with type-appropriate values.

confidence_threshold: strict

label_count: 0

example_pass: |
  ---
  date: 2026-05-12
  type: reference
  source: Stratechery
  author: Ben Thompson
  newsletter_format: thought-leadership
  sponsored: false
  tags: [strategy, platforms]
  ---

example_fail: |
  ---
  date: 2026-05-12
  type: newsletter         # FAIL: should be `reference`
  source: Stratechery
  tags: [strategy]         # FAIL: only 1 tag, need >= 2
  ---
  (missing: author, newsletter_format, sponsored)

notes: |
  This axis is mechanical and could in principle live as a structural_check
  in the per-domain config. It's kept as an axis fragment to keep the
  required-keys table editable in one place; structural_checks are short
  imperatives suited to simple list/count assertions.

Inline example: voice-match (fuzzy axis)

name: voice-match
axis_type: fuzzy
prompt: |
  Compare the artifact's prose voice against the founder's established
  voice as captured in the reference exemplars at the path provided in
  the per-domain config's `reference_artifacts_path`. PASS when the
  artifact reads like the same writer authored both. FAIL when there's
  detectable AI-tell, generic SaaS-marketing voice, or a tonal mismatch
  with the founder's existing pieces.

  Specific tells to flag as FAIL:
  - Em dashes (U+2014). Founder explicit no-em-dashes memory.
  - Three-word triplets ("agile, scalable, robust"). Founder doesn't write that way.
  - Hedging openers ("It's worth noting that...", "Interestingly..."). Founder skips these.
  - Generic abstractions where the founder would use a concrete example.
  - Mini-essay shape on social channels (founder X-voice is 1-2 sentences).

  Specific signals to flag as PASS:
  - Self-deprecating operational asides.
  - Concrete domain examples (MAC, SC, Squarely, etc.) wired into the argument.
  - Decision-led structure - "DECISION NEEDED" or equivalent up top.
  - Sharp verdicts when the source merits one (skip/skim/read/file).

pass_signal: Reads like the founder's existing work; no AI-tells; matches the surface's voice norms (X-voice on X, longer-form on the vault).

confidence_threshold: moderate

label_count: 0

example_pass: |
  "Squarely's PMF gate is: can the puzzle-a-day cadence sustain a 60-day
  retention curve without a paid funnel? Three weeks of data say maybe.
  We'll know by month-end."

  Why PASS: concrete domain reference (Squarely), specific question
  (PMF gate), concrete timeline (60d, month-end), no hedging.

example_fail: |
  "It's worth noting that our innovative puzzle product is achieving
  unprecedented engagement metrics across multiple dimensions, we're
  seeing significant uplift in retention, activation, and virality."

  Why FAIL: hedging opener ("it's worth noting that"), three-word triplet
  (retention/activation/virality), generic marketing abstractions, no
  concrete numbers, no domain anchor.

notes: |
  This axis is the prototypical fuzzy axis - it depends on reference
  exemplars in the canonical set, not a deterministic check. It will
  remain founder-as-critic for longer than mechanical axes because
  voice judgments are the highest-information-density labels the founder
  provides.

3. How the schemas compose at workflow time

When a workflow command (e.g. /build-haiku-generator topic:"first snowfall") fires:

  1. The workflow loads configs/<domain>.yaml.
  2. For each name in critic_axes, the workflow resolves and loads axes/<name>.yaml. Missing fragments are a hard error - the workflow refuses to start.
  3. After spec-author + test-author + code-author have produced their artifacts in the per-run scratch dir at runs/<domain>-<timestamp>/, the workflow runs the structural_checks deterministically against the code artifact. Failures short-circuit before any critic-axis subagent dispatch.
  4. If structural checks pass, the workflow dispatches N parallel critic-axis subagents (one per axis in critic_axes), passing each subagent its axis fragment + reference_artifacts_path + the artifact paths to evaluate.
  5. The critic aggregates the N axis verdicts. If all return PASS with confidence high, the loop converges. If any FAIL or any PASS+weak surfaces, the loop iterates the code-author seat with the diagnostic feedback.
  6. On hitting floor(max_iterations / 2) without convergence, a warning lands in the run dir. On hitting max_iterations, the run is archived and a decision page spawns at ~/rdco-hq/public/decisions/<date>-<domain>-build-stalled.html for founder review.

Bootstrap-mode wrinkle

When bootstrap: true (haiku-generator case during early runs), step 4 routes the critic verdicts to the founder via the channel reply tool instead of automating convergence. Each founder verdict that overrides the autonomous critic increments label_count in the relevant axis fragment. When per-axis label_count >= graduation_threshold, that axis silently graduates to autonomous mode on the next run. Domains can mix - some axes graduate first while others remain founder-gated.

4. Related

Changelog