Anthropic deprecated the list-of-principles structure our axis fragments are modeled on

The question

Verbatim: "What does the Constitutional AI v2 paper (or successor work post-2024) reveal about prompt-space reward shaping that differs from CAI v1, and which findings apply to RDCO's per-axis YAML fragments?" Context: the RLHF-shape concept article cites CAI v1 as its closest academic lineage, and the per-axis YAML fragments in the brigade critic stations are authored on that analogy.

The premise did not hold up. There is no paper titled "Constitutional AI v2." I searched for one and did not find it; nothing in this run's results points to such a paper existing. What exists is a lineage: Collective Constitutional AI (2024), Claude's constitution as a deployed and revised spec (2023, rewritten January 2026), constitutional classifiers, and the broader spec-driven alignment line.

Worth naming: the "CAI v2" phrasing appears to originate inside our own vault. [[2026-05-12-rdco-pipeline-rlhf-shaped]] closes by inviting us to "connect to a growing literature (CAI v2, RLAIF refinements, Self-Refine extensions)," and [[2026-05-19-cai-critic-graduation-per-axis-threshold]] refers to "the CAI v2 / 'Constitution or Collapse?' line." That was loose shorthand for a successor lineage, and it has now been read back as a citation to a specific paper. This brief is the correction: treat "CAI v2" as a vault-internal nickname, not a title, and stop using it.

What we already know (from the vault)

• [[2026-05-12-rdco-pipeline-rlhf-shaped]] makes the load-bearing claim under test: "RDCO's per-axis YAML fragments are structurally identical to a constitution's principles." It correctly notes the reward signal is encoded in prompt space, not weight space, and that the pipeline does no gradient descent.
• [[2026-05-12-multi-agent-pipeline-config-schema]] is the actual schema. An axis fragment requires name, prompt, pass_signal, confidence_threshold, label_count, example_pass, example_fail; optional axis_type (mechanical/fuzzy/hybrid), graduation_threshold, last_updated_by_improve, notes. There is no field for why the axis exists. Convergence is a flat conjunction: "all axes return PASS with confidence >= high," soft target 3-7 axes per domain.
• [[2026-05-19-cai-critic-graduation-per-axis-threshold]] already did the calibration work: ~30-200 labels per axis (not the concept doc's "~10"), an 85% agreement gate, and a mechanical-vs-semantic split with separate thresholds. It also catalogues six failure modes, of which Mode 2 (sycophancy/style-mimicry) and Mode 3 (constitutional loopholes, rule-letter vs rule-intent) matter most below.
• That same brief flags the RDCO-specific version of the risk: /improve "risks codifying the founder's word choices ('sharp verdict', 'skip/skim/read/file') in a way that produces artifacts that USE those words rather than DO those things."
• [[2026-05-22-reward-hacking-patterns-llm-critic-systems]] covers the policy-learns-the-critic's-blind-spots family that the axis design has to survive.

What the web says

Sources I actually fetched in this run are marked [fetched]. Everything else is marked secondhand or not verified.

• [fetched] Anthropic, "Claude's new constitution," published January 22, 2026 (https://www.anthropic.com/news/claude-new-constitution). It explicitly abandons the prior structure: "Our previous Constitution was composed of a list of standalone principles. We've come to believe that a different approach is necessary."
• [fetched] The stated reason is generalization, not tone: "in order to be good actors in the world, AI models like Claude need to understand why we want them to behave in certain ways, and we need to explain this to them rather than merely specify what we want them to do." And: "If we want models to exercise good judgment across a wide range of novel situations, they need to be able to generalize—to apply broad principles rather than mechanically following specific rules."
• [fetched] It introduces an explicit four-tier priority ordering for conflicts — broadly safe, broadly ethical, compliant with Anthropic's guidelines, genuinely helpful — with "In cases of apparent conflict, Claude should generally prioritize these properties in the order in which they're listed." CAI v1's principle list had no such ordering.
• [fetched] A footnote gives the sharpest transferable warning: "Training on rigid rules might negatively affect a model's character more generally... Claude might start modeling itself as an entity that cares more about bureaucratic box-ticking rather than actually helping people." The claim is that a narrow rule teaches something unintended and global about the grader's identity.
• [fetched] Anthropic's older constitution page (https://www.anthropic.com/news/claudes-constitution, dated May 9 2023, carrying a January 21 2026 update note) independently reports a length finding: broad principles "worked remarkably well: Whereas if we tried to write a much longer and more specific principle we tended to find this damaged or reduced generalization and effectiveness." Same page confirms the v1 sampling mechanic: "The model pulls one of these principles each time it critiques and revises its responses... It does not look at every principle every time."
• [fetched] Huang, Siddarth, Lovitt, Liao, Durmus, Tamkin, Ganguli, "Collective Constitutional AI: Aligning a Language Model with Public Input," arXiv 2406.07814, submitted June 12 2024, published at FAccT 2024 (https://arxiv.org/abs/2406.07814). Four-stage process (identify population, source principles, train, evaluate). The CCAI model showed lower bias across nine social dimensions with equivalent language/math/helpful-harmless performance. On contentious topics it tended to "reframe the matter positively instead of a refusal." The abstract does not characterize how public principles differed in form from developer-written ones; I did not find that detail.
• Secondhand, not verified: search results referenced next-generation constitutional classifiers and OpenAI's Model Spec / deliberative alignment as part of the same spec-driven family. I did not fetch these and make no claims about their contents.
• Not verified → RESOLVED 2026-07-19, and the suspicion was correct. [[2026-05-19-cai-critic-graduation-per-axis-threshold]] cited "Bai et al., 'Constitution or Collapse? Exploring Constitutional AI with Llama 3-8B' (arxiv 2504.04918, 2025)" for the mode-collapse finding. Verified against the arXiv record: the ID resolves and the title is right, but the paper has exactly one author — Zhang, Xue (submitted 2025-04-07). The "Bai et al." attribution was wrong, exactly as flagged here. Worse, the finding was also mischaracterized — Zhang reports token-repetition collapse in the policy model from emoji-noisy training data, not critic-side diversity rejection from low label volume. The mode-collapse claim is now struck, not merely unconfirmed. See [[2026-07-19-constitution-or-collapse-citation-verification]]. Process note: flagging this as suspicious-but-unverified rather than silently carrying it is what made the correction cheap — the instinct was right and the hedge was the right call.
• Recall caveat: my knowledge cutoff sits at January 2026, right on top of the new constitution's publication date. I am treating the fetched pages as ground truth over my own recall throughout, and my recall did not independently supply the rules-to-reasoning shift.

Convergences and contradictions

• Contradiction, and it is the finding. [[2026-05-12-rdco-pipeline-rlhf-shaped]] anchors the axis-fragment design on "structurally identical to a constitution's principles." As of January 22 2026, that structure is the one Anthropic publicly discarded. The analogy still holds — our fragments really are a list of standalone principles — but it now points at a superseded artifact rather than a validated one. The lineage claim survives; the endorsement it implied does not.
• Convergence, independently derived. The 2026 constitution's bureaucratic-box-ticking footnote and our own Mode 2 / Mode 3 catalogue in [[2026-05-19-cai-critic-graduation-per-axis-threshold]] are the same failure reached from opposite directions. We predicted that rubric-word codification produces artifacts that use the words rather than do the things; Anthropic reports that rigid rules distort character globally. Two independent arrivals at one failure mode is the strongest signal in this brief.
• A weighting correction. [[2026-05-19-cai-critic-graduation-per-axis-threshold]] concluded "weight production case studies over academic Anthropic papers," because the academic regime (huge models, thousands of iterations) is not ours. Fair. But the 2026 constitution is neither academic nor a third-party case study — it is a deployed spec revised after live experience, which is the evidence class that brief said to privilege. We under-weighted the one source that best matches our own criterion.

Synthesis for RDCO

The concrete gap: our axis fragments encode WHAT, and have no field for WHY. Walk the required schema in [[2026-05-12-multi-agent-pipeline-config-schema]]: prompt, pass_signal, example_pass, example_fail. Every one is a specification of what to match. The voice-match fragment is the clearest case — its prompt is a bulleted list of tells to flag ("Em dashes (U+2014). Founder explicit no-em-dashes memory." / "Three-word triplets" / "Hedging openers"). That is a box-ticking list in exactly the sense the 2026 constitution's footnote warns about. A critic subagent handed that fragment can only pattern-match; it cannot reason about a novel AI-tell that isn't enumerated, because nothing in the fragment tells it what an AI-tell is or why the founder objects. The fragment's own notes field sometimes carries the reasoning ("this axis is the prototypical fuzzy axis...") but notes is optional and framed as audit trail, not critic instruction. Change: add a required rationale field to the fuzzy-axis schema, dispatched to the critic subagent ahead of prompt, stating why the axis exists and what failure it is protecting against. The enumerated tells then become illustrations of the rationale rather than the definition of the axis.

But scope the change to fuzzy axes only, and the 2023 length finding is why. "A much longer and more specific principle... damaged or reduced generalization" is a claim about principles that need to generalize. frontmatter-completeness does not need to generalize: its long enumerated required-keys table is the correct authoring style, because a presence check has no novel situations to handle. This sharpens the mechanical/fuzzy split that [[2026-05-19-cai-critic-graduation-per-axis-threshold]] already proposed for label thresholds into a second, independent axis of the same split — authoring style. Mechanical axes: long, specific, enumerated, no rationale needed. Fuzzy axes: short, broad, reason-first, examples as illustration. Today both are authored in the same long-enumerated register, which is right for half of them and actively counterproductive for the other half. That is a real, cheap, testable schema edit.

The second gap is conflict resolution, and it may explain our stalls. The 2026 constitution's headline structural addition is an explicit priority ordering for when principles conflict. Our convergence criterion is a flat conjunction across 3-7 co-equal axes with no tiebreak. On the haiku domain, novelty and image-coherence can pull against each other; on a Sanity Check artifact, voice-match and any structural-completeness axis can. When two axes disagree, the code-author seat receives contradictory diagnostic feedback and has no basis for trading them off, so it oscillates. That oscillation is a plausible mechanism behind max_iterations exhaustion and the decision-page stall path — and it is a hypothesis worth checking against the run archive before we spend anything on it. If it holds, the fix follows the constitution's shape: a domain-level axis ordering in the per-domain config, so the critic's aggregate verdict can say "axis A yields to axis B" instead of returning an unsatisfiable flat AND.

CCAI transfers narrowly, and I want to be honest about how narrowly. Its finding is that publicly-sourced principles reduced bias without costing capability. RDCO has exactly one labeler, and [[2026-05-19-cai-critic-graduation-per-axis-threshold]] counts that as a benefit (no inter-annotator noise). CCAI is a mild counterpoint rather than a refutation, and the distinction is which axis you're asking about. For voice-match, founder bias is not a defect — it is the ground truth the axis exists to encode, and public input would corrupt it. For an axis like factual-accuracy or non-derivativeness, single-sourcing means the critic inherits the founder's blind spots with no correction path. So: CCAI argues for widening the input only on axes where the founder's taste is not definitionally the target. That is a small subset today, and I would not act on it now. Filed as a boundary to notice if the brigade ever grades work where correctness is external to the founder's preference.

Why this is in the vault

It corrects a specific load-bearing claim in [[2026-05-12-rdco-pipeline-rlhf-shaped]] — that per-axis YAML fragments are structurally identical to a CAI constitution's principles — by showing that structure was publicly retired by Anthropic on 2026-01-22, and it converts that into two concrete schema edits to [[2026-05-12-multi-agent-pipeline-config-schema]] (a rationale field for fuzzy axes; a domain-level axis priority ordering) that the active brigade critic-station work can pick up. It also retires the vault's "CAI v2" shorthand, which had started to read as a real citation.

Open follow-ups

• Does the max_iterations stall path correlate with domains whose axes pull against each other? Check the run archive before building the priority-ordering feature — the oscillation hypothesis above is untested.
• Verify the "Constitution or Collapse?" citation (arXiv 2504.04918, attributed to Bai et al.) carried by [[2026-05-19-cai-critic-graduation-per-axis-threshold]]. [CLOSED 2026-07-19 — citation failed verification.] Single author Zhang, Xue; finding mischaracterized; claim struck across three vault docs. See [[2026-07-19-constitution-or-collapse-citation-verification]].
• Does adding a rationale field measurably change fuzzy-axis critic behavior, or just add tokens? A/B one fuzzy axis (voice-match) with and without rationale against a held-out founder-labeled set. This is the ablation that would make the whole brief actionable rather than merely plausible.
• What did Anthropic's next-generation constitutional classifiers work actually claim? Not fetched this run; potentially relevant to the mechanical-axis half of the split.
• Does OpenAI's Model Spec resolve principle conflicts differently from the four-tier ordering, and is either shape a better fit for a 3-7 axis brigade config?
• Should /improve be forbidden from editing the rationale field, given the 05-19 brief's warning that /improve is exactly the mechanism that codifies founder word choices into box-ticking lists?

Related

• [[2026-05-12-rdco-pipeline-rlhf-shaped]] — the concept doc whose central structural claim this brief corrects
• [[2026-05-12-multi-agent-pipeline-config-schema]] — the axis-fragment schema the two proposed edits land in
• [[2026-05-19-cai-critic-graduation-per-axis-threshold]] — prior CAI-lineage brief; supplies the mechanical/fuzzy split and the failure-mode catalogue this brief extends
• [[2026-05-22-reward-hacking-patterns-llm-critic-systems]] — reward-hacking lineage behind the box-ticking risk
• [[2026-05-12-multi-agent-pipeline-architecture]] — architecture doc the schema implements

Sources

Vault

• [[2026-05-12-rdco-pipeline-rlhf-shaped]] (06-reference/concepts/)
• [[2026-05-12-multi-agent-pipeline-config-schema]] (02-sops/)
• [[2026-05-19-cai-critic-graduation-per-axis-threshold]] (06-reference/research/)
• [[2026-05-22-reward-hacking-patterns-llm-critic-systems]] (06-reference/research/)
• [[2026-05-12-multi-agent-pipeline-architecture]] (01-projects/skill-pipelines/)

Web — fetched and read this run

• Anthropic, "Claude's new constitution," 2026-01-22 — https://www.anthropic.com/news/claude-new-constitution
• Anthropic, "Claude's Constitution," 2023-05-09 (update note 2026-01-21) — https://www.anthropic.com/news/claudes-constitution
• Huang, Siddarth, Lovitt, Liao, Durmus, Tamkin, Ganguli, "Collective Constitutional AI: Aligning a Language Model with Public Input," arXiv:2406.07814, 2024-06-12 — https://arxiv.org/abs/2406.07814

Referenced but NOT fetched or verified this run

• Bai et al., "Constitutional AI: Harmlessness from AI Feedback," arXiv:2212.08073 (2022) — cited via vault, not re-fetched here
• Zhang, Xue, "Constitution or Collapse? Exploring Constitutional AI with Llama 3-8B," arXiv:2504.04918 (submitted 2025-04-07) — verified 2026-07-19; previously carried secondhand with a false "Bai et al." attribution and a mischaracterized finding. See [[2026-07-19-constitution-or-collapse-citation-verification]].
• Anthropic next-generation constitutional classifiers; OpenAI Model Spec / deliberative alignment — surfaced in search results only, contents not examined
• Not found: any paper titled "Constitutional AI v2"