/process-newsletter Output Invariants

Why this exists

Kingsbury's "Future of Everything Is Lies" critique landed a real punch: if the verification layer is itself an LLM, you've just stacked another correlated failure mode on top of the one you were trying to catch. Two LLMs that share training distributions, reinforcement signals, and prompt-injection susceptibility don't independently verify each other — they fail together, quietly, and in ways the user can't audit. RDCO took that critique seriously. This audit script is our concrete answer: a pure-Python deterministic test suite for /process-newsletter outputs that makes ZERO LLM calls and cannot hallucinate a pass. It can fail in known mechanical ways (regex misses an unusual heading variant, an edge-case YAML that parses but doesn't match intent), but those failures are inspectable, reproducible, and patchable. The audit doesn't replace LLM judgment — it provides a structural floor underneath it. If an LLM-generated newsletter note doesn't even have its required frontmatter fields or the load-bearing "Mapping against Ray Data Co" section, no amount of LLM-on-LLM review will save it; the structural check catches it cheaply, deterministically, and forever.

The 13 invariants

General 06-reference/ filing contract

This section covers ALL direct-Write files in ~/rdco-vault/06-reference/, regardless of source (X essays, Anthropic docs, vendor explainers, synthesis catalogs, research briefs, book notes, or any other ad-hoc write). The newsletter-specific invariants above (I1–I13) apply only to files produced by /process-newsletter. The general contract below applies universally.

CLAUDE.md guardrail: Extending this contract into a CLAUDE.md hard rule (pre-write check before any 06-reference/ Write) is an open architecture decision pending founder approval. Until then, this section is the canonical reference; sub-agent dispatches should explicitly cite it in their prompts.

Required frontmatter fields

Newsletter-specific fields (source, author, newsletter_format, sponsored) are required when content_type: newsletter and already covered by I3–I6. For non-newsletter files, sponsored: bool + sponsor_entity are required whenever the source has an undisclosed or self-promotional financial interest.

Required body headings

Every 06-reference/ file MUST contain:

1. ## Why this is in the vault — one or two sentences: what makes this worth the retrieval cost? A file without a stated reason is unauditable.
2. ## Mapping against Ray Data Co (or ## Mapping against RDCO) — the load-bearing section per vault canon. Without a mapping argument, the file didn't justify being filed.
3. ## Related — ≥ 2 wikilinks in [[basename]] format. Full ~/ paths ([[~/rdco-vault/...]]) are not valid wikilinks and will break vault navigation. Use [[basename]] only.

Sponsored-content handling: when sponsored: true or when a self-promotional financial interest is present, add a ## ⚠️ Sponsorship or ## ⚠️ Structural Bias section before the core-argument section — same rule as I11 for newsletters, now universal.

Applies to

• X long-form essays (processed via /process-newsletter watch or manually filed)
• Anthropic docs and vendor explainers filed as reference
• 06-reference/research/ deep-research outputs
• 06-reference/concepts/, 06-reference/cross-checks/, and any other subdirectory
• Any direct-Write to 06-reference/ regardless of agent or skill

Link to newsletter-specific section

The newsletter-specific invariants (I1–I13) above enforce a subset of this contract for /process-newsletter outputs and add source/sender-matching checks not applicable to other sources. The general contract is the floor; I1–I13 add a stricter ceiling for newsletter files.

How to extend

When adding a new invariant, follow this pattern:

1. Single concern. One invariant = one structural property. If you find yourself writing or between two unrelated checks, split them.
2. Deterministic. No LLM calls. No network. No probabilistic logic. Stdlib + PyYAML only. If a check requires judgment, it doesn't belong here — it belongs in an LLM-driven review skill.
3. Named I<N>. Sequential, never reused. If you retire an invariant, leave its ID dead and don't recycle it.
4. Document in this SOP. Add a row to the invariants table above with the same name as the script's INVARIANTS dict entry.
5. Add to the script. Implement in ~/.claude/scripts/audit-newsletter-outputs.py, return (invariant_id, message) tuples on failure, empty on pass.
6. Dogfood-test against the existing corpus before committing. Run with --since 2026-01-01 (or the full audit window) and inspect the violation distribution. If the new invariant generates >50% violation rate against existing files, either the invariant is wrong or the corpus has a real hygiene problem worth surfacing — be honest about which.

Failure handling

The audit is a post-condition check, not a gating check. When /process-newsletter finishes a batch or watch run, it runs the audit, includes the violation summary in the run report, and surfaces failing files for founder review. It does NOT auto-block, auto-retry, or auto-fix. The pattern is discover → audit → decide, not gate → block → error. The founder owns the decision of whether a violation matters enough to re-process the offending file. This keeps the audit cheap and honest: it never gets in the way, and it never silently "corrects" something it doesn't understand. If the audit becomes a gating check in the future (e.g. wired into a pre-commit hook on the vault repo), the --strict flag exists for that — but the default mode is observe-and-report.

Limitations (be honest)

These invariants do NOT catch:

• Semantic correctness of the Mapping section. I8 fires if the heading is missing. It cannot tell you whether the mapping argument is good, honest, or load-bearing. A note with ## Mapping against Ray Data Co\n\nN/A passes I8.
• Accuracy of sponsor disclosure. I11 fires if the ## Sponsorship heading is missing when sponsored: true. It cannot verify that the sponsor was correctly identified, that an undisclosed sponsor wasn't missed, or that the bias note is calibrated.
• Whether the citation is actually right. Nothing here checks source_url resolves, that the author byline matches the real author, or that quoted material is accurate.
• Whether the topic-slug is well-chosen. I10 only checks the sender prefix. The trailing topic-slug could be lazy, generic, or misleading.
• Whether the article should have been filed at all. Skip-triage (Step 2.5 of the skill) is judgment territory; we can't catch a file that shouldn't exist.
• Cross-document consistency. If two notes contradict each other, or if a note contradicts something already in the vault, the audit doesn't notice.

These remain LLM-judgment territory. Future versions could add semantic invariants that cross-reference against the QMD index or the typed knowledge graph at ~/.claude/state/graph.duckdb (e.g. "does this note's claimed author exist in the vault's author authority list?"), but that re-introduces LLM contamination if the index/graph were themselves built by LLM-driven processes. Keep that semantic layer separate, name it differently (e.g. semantic-audit-newsletter-outputs.py), and never confuse the deterministic structural audit with the probabilistic semantic audit. The whole value of this script is that it doesn't pretend to do something it can't.

Changelog