06-reference

improve process newsletter run

2026-06-24·reference·source: internal-review·by Ray (COO agent)
skill-improvementprocess-newsletterself-reviewnewsletter-pipeline

/improve process-newsletter — 2026-06-24

Run triggered as part of Notion task 340f7d49-36d1-81cf-b64e-f41e2bee8639 ("Integrate skill-creator evals into /improve workflow"). This is the first on-demand /improve run on the process-newsletter skill since the April 2026 full backfill run.

Data sources examined

Diarized profile

SKILL: process-newsletter
RUNS SINCE LAST IMPROVE: ~400+ watch-mode emails processed since June 2026 audit window
AUDIT WINDOW: 2026-06-14 to 2026-06-24
FILES AUDITED: 77
PASS: 59 (77%)
FAIL: 18 (23%)

WHAT WORKS WELL:
  - Watch path output is consistently A-grade per self-review Reviews 13–15
  - Schema compliance on all major fields: date, type, source, author, sponsored, tags
  - Sponsor detection is working — sponsor_entity present and paired correctly
  - RDCO mapping section quality is high on strong-relevance senders (Stratechery, AlphaSignal)
    e.g. 2026-06-24 Stratechery note names specific RDCO artifacts: Mac Mini setup, phData DSA role,
    plugin marketplace direction
  - newsletter_format enum drift issue (from Reviews 11-13) is resolved — no new enum violations
  - Sub-agent fan-out is keeping parent context lean; no context pressure issues
  - Content triage / skip logic is correct — no sales funnel emails being filed

WHAT'S MEDIOCRE:
  - M1 (HIGH FREQUENCY): I12 violations — curation/hybrid format notes are omitting the required
    `## Curation section` or `## Issue contents` heading. Notes use `## The core argument` with
    `###` subsections for the curation content instead. The content is structurally correct but
    the canonical section header is missing, which fails the deterministic audit check. 15 of 18
    failing files (83% of failures) are I12 violations.
  - M2 (ISOLATED): YAML parse error from colon-in-value in unquoted author string.
    `2026-06-18-analytics-engineering-roundup-duckdb-agent-moment.md` has
    `author: Dan Poppy (host: Tristan Handy; guest: Jordan Tigani)` — the `: ` inside the parenthetical
    triggers a YAML parse failure (I2 invariant). The content is correct; the encoding is wrong.
  - M3 (CROSS-FILE PATTERN): Spurious cross-links in `## Related` sections. The vault document
    `2026-06-09-claude-md-prompt-precedence-full` (a CLAUDE.md engineering SOP) appears in Related
    sections of both the Mostly Metrics S1 analysis and the Secret CFO mailbag — no topical
    connection. Looks like template bleed from a subagent carrying state across articles.
  - M4 (LENGTH DRIFT): `## Why this is in the vault` section drifts from the intended one-sentence
    rationale to multi-sentence content summaries in 4 of 6 sampled files. Not a schema violation
    but blurs the boundary between rationale and content summary sections.
  - M5 (HEADING ORDER): Not Boring WDoO #198 has headings out of canonical order — `## Issue contents`
    appears before `## Why this is in the vault`. Canonical order: vault rationale → [content summary]
    → mapping → related. Minor deviation, not audited.

WHAT FAILS:
  - No outright content failures — all 59 passing files are substantively correct
  - The 18 failures are structural/schema violations, not content errors
  - Non-watch-path failures (3 files: Satya Nadella note, owner-mindset reflection, Udemy cert notes)
    are typed-wrong or missing fields entirely — these are not newsletter files and aren't the skill's
    responsibility; they confirm the Watch path vs. non-watch path quality divergence pattern identified
    in Review 15

GAP:
  - I12 is the persistent dominant failure across every audit window (Reviews 11-15 + this run).
    The fix that was applied (checklist enforcement in Mode 4 watch subagent prompt) didn't fully
    close it — subagents are still reaching for `## The core argument` as the curation container.
    The instruction needs to be made more explicit: `## The core argument` does NOT satisfy I12.
  - YAML quoting requirement wasn't codified for the `author` field with host/guest attribution.
    The guest-post format makes this scenario common (podcasts, interview newsletters).
  - Related section quality is not enforced — subagents can and do paste ops/SOP cross-links
    that have no topical relevance to the newsletter content.

Review log synthesis (process-newsletter specific)

All review entries 2026-05-24 through 2026-06-14 are marked improve_processed. Key newsletter-specific pattern trajectory:

Proposed changes

Applied (low-risk)

Fix A — I12 compliance made explicit in Step 5 and Mode 4 checklist

Fix B — YAML colon-in-value quoting rule

Fix C — Related section quality rule

Queued (structural — needs founder review)

Structural 1 — 06-reference/ folder structure at scale

Structural 2 — Deep-fetch threshold revision

Quality benchmark (6-note sample)

Note Schema I12 Mapping quality Related quality
Stratechery vibe-coding Pass N/A (thought-leadership) Excellent — names RDCO artifacts Good
AlphaSignal agents-optimize-skills Pass Pass Excellent — action-tied Good
Secret CFO mailbag Pass N/A (mailbag) Strong Weak (spurious SOP link)
Ship30for30 AI workflows Pass N/A (thought-leadership) Medium-strong Weak (aspirational links)
Mostly Metrics Lime S1 Minor (no source_url) N/A (curation, no I12 check on audit window) Thin but honest Weak (spurious SOP link)
Not Boring WDoO #198 Pass Pass (has Issue contents) Thin Rote (backward-links only)

The "no one-off work" test

Would this improve run look different for /process-youtube? Yes — the I12 failure pattern is newsletter-specific (curation/hybrid format trigger). The YAML colon-in-value issue is generic and applies to any skill that writes YAML frontmatter with human names in multi-role fields. The Related-section quality rule is generic. Consider adding both to a shared filing template or 06-reference writing SOP that all content-ingestion skills reference.

Related