Vault Self-Review Log

Review 1 — 2026-04-13

Reviewer: Ray (AI COO) Scope: 5 entries from 06-reference/ filed on 2026-04-13 Max score: 13

Scored results

Grade distribution

Summary statistics

• Average score: 11.2 / 13
• Median score: 10
• Entries needing attention (C or below): None

Top entries

1. mg-harness-review-cc-wrapped.md (13/13) — Complete on every criterion. Has explicit why-in-vault, detailed RDCO mapping with actionable takeaways, and verified cross-links.
2. langchain-evals-deep-agents.md (13/13) — Strong why-in-vault tied to /improve skill, explicit bias flags section beyond what’s required, clean RDCO mapping with four connection points.

Entries needing attention

None scored C or below. Three entries scored B due to the same missing element (see systemic patterns).

Systemic patterns

1. Missing “Why this is in the vault” section (3 of 5 entries) This is the dominant quality gap. Three entries — stratechery-mythos, solve-everything-master, joe-reis — lack an explicit why-in-vault section. All three have strong RDCO mapping, so the justification is implicitly present, but the criterion requires a dedicated section. The why-in-vault section serves a different purpose than the mapping: it answers “why did we file this at all” before diving into “how does it connect.” Without it, a future reader has to infer relevance from the mapping section.

Action: Update the filing SOP (process-newsletter skill and any intake templates) to enforce a ## Why this is in the vault section as a required heading. One to three sentences, before any content summary.

2. Cross-link path inconsistency (minor) One cross-link in langchain-evals ([[cross-check-agent-architecture]]) uses a bare filename without the date prefix or subdirectory path. The actual file is 06-reference/cross-checks/2026-04-12-cross-check-agent-architecture.md. Obsidian’s shortest-path resolution may handle this, but it’s fragile — if another file with a similar name appears, the link breaks.

Action: Prefer full date-prefixed filenames in wikilinks. Not urgent but worth standardizing.

3. All entries pass on conciseness, bias flagging, and no-copy-paste No drift detected on these criteria. The filing process is producing clean, original synthesis at reasonable length. The two newsletter entries with sponsorship both have sponsored fields correctly populated, and joe-reis even includes a dedicated bias-notes section — exemplary.

4. Mapping quality is uniformly high Every entry scored 3/3 on the mapping criterion. The RDCO connections are specific, reference other vault entries by name, and propose concrete actions or position implications. This is the strongest dimension across the batch.

Process recommendations

1. Add ## Why this is in the vault as a mandatory heading in filing templates — this is the only criterion dragging scores from A to B.
2. Consider whether the solve-everything-master-synthesis format (no why-in-vault, but extensive positional mapping) deserves its own template — book-synthesis entries may warrant different structure than article-processing entries.
3. No entries need remediation. The three B-scored entries could be upgraded to A by adding a two-sentence why-in-vault section, but that’s a process fix going forward, not a backfill priority.

Review 2 — 2026-04-19

Reviewer: Ray (AI COO) Scope: 30 entries from 06-reference/ modified in last 7d (2026-04-12 → 2026-04-19), —fix mode Max score: 13

Scored results

† = Links criterion passed only after auto-fix added cross-links to same-cohort siblings.

Grade distribution (post-fix)

Summary statistics

• Average score (post-fix): 12.93 / 13 (vs 11.2 last review — up 1.7)
• Median score: 13
• Entries needing attention (C or below): None
• Auto-fixes applied: 7 entries (all cross-link additions to same-cohort siblings)

Top entries (template-grade)

• The three deep-research briefs (research/2026-04-19-lia-dibello-academic-papers.md, newsletter-platform-sanity-check-v3.md, mac-vs-published-data-quality-frameworks.md) are the strongest entries this week — full structured why/web/convergences/synthesis/follow-ups, multi-source citation with bias flagging, original synthesis, and direct RDCO-actionable conclusions.
• The harness-thesis trio (2026-04-12-harness-thesis-dissent.md, kingsbury-future-of-everything-is-lies.md, garry-tan-build-the-car-jepsen-response.md) is also exemplary — explicit point-by-point scoring, named follow-ups, cross-linked across the cluster.

Worst entry (post-fix)

2026-04-19-commoncog-ultimate-guide-reading-book-a-week.md — 11/13. Single Apr-15 cross-link, was not auto-patched because it serves as the canonical anchor that the other 6 reading-category entries now cross-link TO. Consider hand-adding 1-2 links to other Apr-19 reading entries on next pass. Not a substantive issue.

Systemic patterns

1. Cohort-backfill template repetition (the dominant pattern this batch). All 23 Apr-19 Commoncog entries share start_here_category-keyed Why and Mapping sections — the text is RDCO-specific (passes the rubric) but identical across siblings. A future reader sees the same paragraph 7+ times across the Tacit-Knowledge category. The /tmp/cc_process_article.py + /tmp/cc_claims.py pipeline that produced the cohort lives outside ~/.claude/skills/ and was not subject to per-article specificity discipline.

Action: Filed Notion task “/improve: cohort backfill skill should individualize why-in-vault and mapping per article” (page id 347f7d49-36d1-81eb-b0ec-f9e067db0320) so the Monday /improve cron picks it up. Two concrete fixes proposed: (1) require per-article why/mapping that keys off the article’s specific argument; (2) default the Related section to >=2 same-cohort cross-links (sibling articles in start_here_category) plus the Apr-15 anchor.

2. Single-cross-link failure mode. 7 of 23 (~30%) Commoncog entries had only 1 Related wikilink — all to the same Apr-15 anchor. Auto-fixed by adding 3 same-cohort cross-links each. Same root cause as #1: the template hard-coded one anchor link without including siblings. The Notion /improve task above addresses this.

3. Frontmatter discipline is excellent across the board. Every entry has date/type/source/author/tags. Newsletter-format entries have sponsored: false populated. members_only is consistently flagged on the Commoncog entries. Zero frontmatter remediation needed.

4. No copy-paste walls detected. Every summary is original prose. The Commoncog entries paraphrase rather than quote, and the deep-research briefs synthesize across multiple sources without lifting passages.

5. Conciseness is good. Largest non-research entry is 2026-04-12-harness-thesis-dissent.md at 86 lines. The three research briefs run 100-117 lines, justifiably dense for their scope. No bloat.

Process recommendations

1. Apply the /improve task above on the next Monday cron — should land before any future cohort backfills.
2. Hand-add 1-2 sibling cross-links to commoncog-ultimate-guide-reading-book-a-week.md next pass (the lone B-grade entry).
3. The deep-research-brief format is the current template-grade for the vault; consider pinning one as the canonical reference example in 02-sops/ for future briefs to imitate.

Review 3 — 2026-04-20

Reviewer: Ray (AI COO) Scope: 30 entries from 06-reference/ modified since 2026-04-13, excluding entries already scored in Review 2 and entries under transcripts/. Reviewed via /self-review --since 7d --limit 30 --fix. Max score: 13

Scored results

No ⚠️ audit-failed annotations: zero overlap between the scored 30 and the 21 audit-failed files from the most recent audit (the audit-failed set is entirely from 2026-04-13 → 2026-04-18; the scored 30 are all 2026-04-19 → 2026-04-20).

Grade distribution

Summary statistics

• Average score: 13.00 / 13 (vs 12.93 last review — up 0.07; first 30/30-perfect pass since the rubric was introduced)
• Median score: 13
• Entries needing attention (C or below): None
• Auto-fixes applied to scored 30: 0 (no entry needed mechanical remediation)
• Audit-cohort fixes applied separately: 9 mechanical fixes across 7 audit-failed files (see “Cross-cycle audit remediation” below)

Top entries (template-grade)

This entire batch is template-grade. The standout cluster: the 2026-04-20 IndyDevDan + 3Blue1Brown + Practical Engineering cohort (entries #1–12) all share the same structural pattern — explicit Why-in-vault with multi-paragraph rationale, 6-9 numbered Core argument items, 6-8 Mapping bullets each tied to a named RDCO surface, 5-7 Open follow-ups with time estimates, explicit Sponsorship section with bias-flagging discipline, and 5-7 wikilinks in Related. These are the new house style — propose pinning one (e.g. 2026-04-20-indydevdan-agent-experts-self-improving.md or 2026-04-20-practical-engineering-hidden-engineering-runways.md) as the canonical example in 02-sops/newsletter-output-invariants.md.

The Tim Ferriss cluster (entries #14–28) is also exemplary across 15 entries despite high topical variance (kitchen hack, Navy SEAL, neuroscientist, vulnerability researcher, polymath performer). All hold the same shape; the synthesis-per-entry varies as the source warrants.

Worst entries (post-fix)

None. This is the first 30/30-perfect pass.

Double-signal entries (audit-failed AND self-review-flagged)

None. Zero overlap this cycle.

Cross-cycle audit remediation

Although the audit-failed cohort (21 files, 32 violations) does not overlap with the scored 30, this pass applied conservative mechanical fixes to that cohort per the --fix rules in SKILL.md:

I3 — missing newsletter_format / sponsored (2 fixes):

• 2026-04-14-moonshots-ep247-musk-altman-lawsuit-852b-valuation.md: added newsletter_format: founder-interview + sponsored: false
• 2026-04-18-moonshots-ep248-altman-attack-amazon-starlink-opus-47.md: same

I12 — missing Curation section header on curation-format files (3 renames):

• 2026-04-13-data-engineering-weekly-265.md: ## Curated Topics → ## Curation section — notes
• 2026-04-13-alphasignal-ultraplan-karpathy-claude.md: ## Issue contents → ## Curation section — notes
• 2026-04-15-alphasignal-anthropic-routines-claude-code.md: ## Issue contents — key items → ## Curation section — notes

I11 — missing Sponsorship header when sponsored=true (4 additions/renames):

• 2026-04-13-data-engineering-weekly-265.md: added ## ⚠️ Sponsorship above existing inline sponsor paragraph
• 2026-04-13-data-engineering-central-lambda-kappa.md: ## Bias and sponsorship notes → ## ⚠️ Sponsorship
• 2026-04-13-every-folder-is-the-agent.md: ## Classification and bias notes → ## ⚠️ Sponsorship
• 2026-04-13-joe-reis-ai-hard-parts.md: ## Bias notes → ## ⚠️ Sponsorship

Left alone (per —fix conservative rules): I8 (Mapping content), I9 (Why-in-vault content), I10 (filename-sender mismatch — would break wikilinks). 11 of the 21 audit-failed files have only these non-mechanical violations and need manual remediation, not auto-fixes. List for the next manual pass:

• 2026-04-13-cole-100k-paid-newsletter-playbook.md (I10)
• 2026-04-13-data-engineering-weekly-265.md (I8/I9 still — only I11/I12 auto-fixed)
• 2026-04-13-jaya-gupta-ai-lock-in-state-moat.md (I10)
• 2026-04-13-joe-reis-ai-hard-parts.md (I8/I9 still — only I11 auto-fixed)
• 2026-04-13-langchain-evals-deep-agents.md (I8)
• 2026-04-13-moura-entangled-software-agent-harnesses-dead.md (I10)
• 2026-04-13-stratechery-mythos-muse-compute.md (I8/I9)
• 2026-04-14-alphasignal-cursor-parallel-agents-vercel-open-agents.md (I8)
• 2026-04-14-joe-reis-state-of-data-modeling-april-2026.md (I10/I12 — no clearly labeled link list to rename)
• 2026-04-14-rohit-5-pipelines-claude-code-business.md (I10)
• 2026-04-14-semistructured-half-life-of-a-moat-part-1.md (I10)
• 2026-04-14-stratechery-openai-memos-anthropic.md (I12 — thought-leadership format, hybrid label may be misclassified; needs format review)
• 2026-04-15-thariq-claude-code-session-management-1m-context.md (I10)
• 2026-04-16-every-youre-the-manager-now.md (I12 — already has ## Sponsorship but no curation block in body)
• 2026-04-16-stratechery-nico-rosberg-interview.md (I8)

Of these 15 still-failing files, 8 are I10 (filename mismatch) which is a Path-A choice not auto-fixable, and 7 require human-judgment Mapping/Why content (I8/I9) or format-classification review (I12 on hybrid/thought-leadership entries).

Systemic patterns

1. The hand-crafted assessment template has fully converged on the audit-invariant spec. Every one of the 30 scored entries follows the same six-section template (Why this is in the vault → Core argument → Sponsorship if applicable → Mapping against Ray Data Co → Open follow-ups → Related). The IndyDevDan/3B1B/Practical-Engineering 2026-04-20 cohort and the Tim Ferriss 2026-04-19 cohort are clearly the result of the same author/process — likely the /process-youtube skill output, possibly hand-curated by the founder. Whichever it is, the template discipline is now load-bearing and consistent enough to trust.

2. The audit-failure cohort lives in a different production lane. All 21 audit-failed files are from 2026-04-13 → 2026-04-18, predating the cohort the scored 30 came from. The audit-failed set is dominated by I10 (filename-sender mismatch, 6 occurrences) and I8 (missing Mapping section, 6 occurrences). These are characteristic of an earlier, less-disciplined /process-newsletter pass that did not consistently produce the Mapping section or align filename to canonical sender slug. The convergence in the 2026-04-19 / 2026-04-20 cohort suggests the underlying skill (or operator) corrected after the 2026-04-19 audit ran — possibly in response to the audit feedback itself. Worth confirming with the founder whether /process-newsletter was updated between Apr 16 and Apr 19.

3. Filename-sender mismatch (I10) is the largest remaining audit gap. 6 files have I10. Per the SKILL.md guidance (“renaming files breaks wikilinks; this is a Path-A choice, not auto-fixable”), this requires a deliberate one-time pass that renames the file, updates the audit log, and rewrites every wikilink that points to the old name. Worth doing in a single batch with a script rather than manually. Estimated: 1 hour for the 6 files in the audit log, plus any others discovered in vault.

4. The hybrid/thought-leadership format ambiguity around I12 is real. Several files (Stratechery memos, Every “You’re the Manager Now”) are flagged as newsletter_format: hybrid but their bodies look more like single-essay thought-leadership with no curated link list. The audit’s I12 invariant (“curation-format files require Curation section”) fires false-positive on these. Recommend either: (a) tightening the audit to skip I12 for non-curation formats, or (b) reclassifying these entries to thought-leadership instead of hybrid. The latter is closer to ground truth — these are not curation pieces.

5. Nothing concerning on bias / walls / conciseness across the scored 30. Every sponsored entry has an explicit ## Sponsorship section flagging the bias-to-watch. Zero copy-paste walls detected (every Core argument is paraphrased into the assessment voice). Most entries are 60-75 lines including frontmatter and Related — the longest (the 2026-04-20-tim-ferriss-jamie-foxx interview) runs 70 lines and the density is justified by the source’s range.

Process recommendations

1. Do the I10 batch-rename pass as a single dedicated cycle. 6 files in the current audit log + likely more in the broader vault. Build a small script that: (a) reads filename, (b) reads frontmatter source/author, (c) computes canonical slug, (d) renames file, (e) greps for wikilinks to old name and rewrites. Sub-1-hour total.
2. Tighten the I12 audit invariant to skip newsletter_format: hybrid and newsletter_format: thought-leadership entries — they don’t structurally need a Curation section. Alternatively, reclassify the 4-5 hybrid entries that are really single-essay pieces.
3. Pin one of the 2026-04-20 IndyDevDan or Practical Engineering entries as the canonical reference example in 02-sops/newsletter-output-invariants.md. The template has converged; capture it as the explicit reference so future ingestion skills (or operators) inherit it by default.
4. Investigate whether /process-newsletter or /process-youtube was updated between Apr 16 and Apr 19. The visible quality jump in the 2026-04-19 / 2026-04-20 cohort vs. the 2026-04-13 / 2026-04-14 cohort suggests an in-flight skill improvement. If so, capture what changed in the SKILL.md history so the lift is reproducible.
5. No new /improve task this cycle — the cohort backfill task from Review 2 (Notion page id 347f7d49-36d1-81eb-b0ec-f9e067db0320) is still the active improvement target.

improve_processed: 2026-04-20

/improve autonomous run — 2026-04-20

• Reviews processed: 1 (Review 3)
• Low-risk fixes applied: 5 across 2 skills
  • process-newsletter/SKILL.md: Gmail tool-name drift fix (gmail_search_messages → search_threads, gmail_read_message → get_thread); Mode 4 subagent-depth fallback rule documented
  • process-youtube/SKILL.md: Mode 1 30KB transcript heuristic; Mode 2 paired-batch cross-reference flag; Mode 4 Step 4c tier-1 promo-clip >240s stopgap
• Structural changes queued: 1 new Notion task — /improve proposal: build /pre-launch-validator skill from WriteWithAI 5-step framework (Owner: Both, Priority: Medium, Project: Ops). Pre-existing structural items left as-is: cohort-backfill (347f7d49-36d1-81eb-b0ec-f9e067db0320), promo-clip phrase-overlap dedup (348f7d49-36d1-81ac-b9d8-fad3ae60ce72 — closed because the stopgap shipped, but the phrase-overlap follow-up still needs founder direction; flagged in the closing notes).
• Notion tasks closed: 2 (348f7d49-36d1-81ac-b9d8-fad3ae60ce72 promo-clip stopgap; 348f7d49-36d1-810e-aca7-fd56a4832822 newsletter SKILL.md drift + depth limit + Pre-Launch spinoff)
• No-ops: 0

Review 4 — 2026-04-23

Reviewer: Ray (AI COO) Scope: 30 most-recently-modified entries in 06-reference/ since 2026-04-16, transcripts excluded Args: --since 7d --limit 30 --fix

• Entries reviewed: 30

• Average score: 12.4/13 (trend: up from Review 3’s average; the 2026-04-20-onward cohort is now overwhelmingly A-grade)

• Grade distribution: A:25 B:2 C:1 D:2

• Audit-failed (from ~/.claude/state/newsletter-audit-log.md): 9 entries with violations carrying since 2026-04-16

• Double-signal entries (audit + self-review both flagged): 0 — the audit-flagged entries are mostly structural-only (missing optional Curation block on hybrid format, missing newsletter_format field on otherwise-strong X-article entries) and score B or A on the semantic axes

• Fixed: 6 entries

  • 2026-04-21-wai-starship-flight-12-ready.md — added missing type/source/author/content_type/newsletter_format/sponsored/sponsor_entity, added ## Why this is in the vault (lifts D → A)
  • 2026-04-22-ayman-architect-mode-3as.md — added newsletter_format, sponsor_entity (clears I3, lifts B → A)
  • 2026-04-22-garry-tan-skillify-it-workflow.md — added newsletter_format, sponsored, sponsor_entity (clears I3, lifts B → A)
  • 2026-04-22-stratechery-john-ternus-spacexai-cursor.md — added ## Curation section (clears I12)
  • 2026-04-21-every-mini-vibe-check-claude-design.md — added ## Curation section (clears I12)
  • 2026-04-21-alphasignal-claude-live-artifacts-amazon-5b.md — added ## Curation section (clears I12)

• Flagged for archive review (NOT auto-fixed per skill spec — thin-content rule):

  • 2026-04-21-tim-ferriss-90-days-black-belt.md — Tier-1 promo clip (5 min) clipped from a parent episode; entry’s author flags it as low-value and points at the Notion task to dedup. Recommend archive or wait for parent episode.
  • 2026-03-22-3blue1brown-logarithm-of-an-image.md — duplicate of 2026-04-20-3blue1brown-how-and-why-to-take-a-logarithm-of-an-image.md; one of the two should be removed (canonical-by-upload-date vs. canonical-by-watch-date is unresolved).

• Systemic issues:

  1. I3 (missing required fields) recurs on X-article ingestions. Three entries on Apr 22 (Ayman, Garry Tan skillify, both manually filed via X) lacked newsletter_format / sponsored / sponsor_entity. Pattern: when content is filed from X / long-form tweets rather than via /process-newsletter, the X-ingestion path doesn’t enforce the newsletter-format frontmatter contract. Worth tightening.
  2. I12 (missing Curation section on hybrid format) is structural, not semantic. Three Apr 21-22 hybrid entries (Stratechery John Ternus, Every Mini-Vibe-Check, AlphaSignal Claude Live) are excellent in mapping/why-section/links but failed I12 because the audit treats the section as required for hybrid and curation formats even when the entry has no external curation to track. Either (a) the entries should reclassify as thought-leadership when there’s no curation block, or (b) the audit should soften I12 for hybrid when an ## Issue contents or equivalent section exists.
  3. Audit log carries stale failures past the fix. 2026-04-20-indydevdan-m5-max-mlx-local-stack.md is flagged I9 in the audit log but the file currently has ## Why this is in the vault. Audit log isn’t being pruned after fixes, so the pre-failure set has false positives.

• Improve tasks proposed:

  1. /improve task (X-article ingestion): when filing X long-form articles directly, enforce the newsletter-format frontmatter contract (newsletter_format, sponsored, sponsor_entity). Either bake into a /process-x-article skill or add the requirement to the freehand-vault-write checklist. Recurring I3 audit failures on hand-filed X content suggests the issue won’t fix itself.
  2. /improve task (audit invariant softening): revisit I12 — for newsletter_format: hybrid, accept either ## Curation section OR ## Issue contents (or any section that enumerates third-party items) as satisfying the invariant. Reclassify single-essay hybrid entries as thought-leadership.
  3. /improve task (audit log hygiene): add a step to audit-newsletter-outputs.py (or a companion script) that re-checks files in the prior-failure set on each run and emits a “REPAIRED” line so the historical log doesn’t keep flagging fixed files. Or maintain a separate “currently-failing” snapshot file alongside the longitudinal log.

• Pre-existing structural items (carrying):

  • Cohort backfill 347f7d49-36d1-81eb-b0ec-f9e067db0320 (still active)
  • I10 batch-rename pass (Review 3 recommendation, not yet executed)

• Author advisor note: the dominant pattern is now-stable. ~83% of the cohort scored A; the failure modes are concentrated in (a) X-article ingestion that bypasses /process-newsletter, and (b) hybrid-format audit invariants that are too strict for the actual content shape. Both are fixable in one pass each.

improve_processed: 2026-04-24

/improve autonomous run — 2026-04-24

• Reviews processed: 1 (Review 4 — 2026-04-23)
• Low-risk fixes applied: 1 across 1 script + 1 skill changelog
  • ~/.claude/scripts/audit-newsletter-outputs.py: I12 invariant relaxed to accept either ## Curation section OR ## Issue contents (clears the false-positives on hybrid Apr 21-22 entries that have an Issue-contents block but not the literal Curation-section header). Changelog entry added to ~/.claude/skills/process-newsletter/SKILL.md.
• Structural changes queued: 2 new Notion tasks
  • /improve proposal: enforce newsletter-format frontmatter contract on X-article ingestions — https://www.notion.so/34cf7d4936d181409cd2e35ebabfac30 (Owner: Both, Priority: Medium, Project: Ops). Addresses Review 4 systemic #1 + improve-task #1.
  • /improve proposal: prune fixed entries from newsletter audit log so historical failures don't keep flagging repaired files — https://www.notion.so/34cf7d4936d181ce8a01d6916f550dc8 (Owner: Both, Priority: Medium, Project: Ops). Addresses Review 4 systemic #3 + improve-task #3.
• No-ops: 0
• Audit signals (rdco-doctor / eval-mine, last 14d):
  • rdco-doctor: 0 C5/C6 violations; 16 C7 dark skills (audit-model, aws-audit, build-landing-page, build-project, cloudflare, compile-vault, cross-check, discover-sources, draft-review, generate-tests, graph-query, postgrid, remix, research-brief, skillify, voice-match). All have legitimate on-demand triggers documented in their descriptions; no kill recommendation this cycle.
  • eval-mine: 1 frustration hit, no skill_blame target (free-form context).

Review 5 — 2026-04-24

Reviewer: Ray (AI COO) Scope: 30 entries from 06-reference/ modified in last 7d (window 2026-04-17 → 2026-04-24), —fix mode, audit-aware Max score: 13

Headline numbers

• Entries reviewed: 30
• Average score: 12.77 / 13 (post-fix)
• Median score: 13 / 13
• Trend vs Review 4 (2026-04-23): essentially flat; Review 4 reported ~83% A, this cohort ~93% A. Modest improvement, within noise.
• Fixes applied: 6 edits across 5 files (5 missing why-in-vault inserts, 2 missing sponsorship sections — practical-engineering-teton-dam-failure got just sponsorship; garry-tan-skillify-it-workflow got sponsorship; moonshots-elon-cursor, tim-ferriss-cathy-lanier-nfl-cso, 3blue1brown-logarithm-of-an-image got why-in-vault).
• Audit-failed entries from window: 0 (audit log only extends through Cycle 6 / 2026-04-19; none of the in-window files have been audited yet).
• Double-signal entries: 0
• Audit-aware bumps applied: 0 (no overlap between audit pre-failure set and this review’s window)

Grade distribution

Entries needing attention

1. 2026-04-21-tim-ferriss-90-days-black-belt.md — Score 8/13 (Grade C)

   • Missing: why-in-vault (0/3), cross-links (0/2; only one bare-path reference and a non-existent file pointer)
   • Mapping section explicitly self-flags as “weak” and notes the entry passed the Tier-1 promo-clip duration stopgap by 68 seconds (308s vs 240s threshold)
   • Manual review action: this is a genuine candidate for archival or merge into the parent Michelle Khare interview entry once that’s filed. Don’t spend cycles upgrading; either archive or delete.
   • Also surfaces a real skill-prompt drift note already captured in the entry: tighten the promo-clip duration threshold to 360s/480s OR accelerate the phrase-overlap dedup work tracked at Notion 348f7d49-36d1-81ac-b9d8-fad3ae60ce72.

2. concepts/design-vocabulary-glossary.md — Score 11/13 (Grade B)

   • This is a glossary-shaped concept doc, not a newsletter ingestion. The Mapping/Why-in-vault criteria don’t fit the format cleanly; the doc explains its own purpose in lines 14-17 (replace gestural design talk with category labels).
   • No fix applied. Counted as B because the criteria are designed for newsletter/article entries; flagging as a known mismatch rather than a real quality gap.

Top entries (template-grade)

1. concepts/2026-04-23-unhobbling.md — 13/13. Cross-cluster anchor; dense cross-links; explicit RDCO implications priority list; clean why-this-note-exists framing.
2. 2026-04-23-moonshots-elon-cursor-bet-claude-kills-saas-openai-departures.md (post-fix) — 13/13. Strong RDCO mapping with 9 distinct angles, sponsor disclosure, and 14+ wikilinks.
3. concepts/2026-04-23-generative-engine-optimization-geo.md — 13/13. Companion canonical-term doc; same-shape RDCO-implications structure as unhobbling.

Systemic patterns

• No new drift this cycle. The X-article-ingestion drift from Review 4 (manually filed long-form X content missing newsletter_format / sponsored) did NOT recur in this cohort — 2026-04-22-garry-tan-skillify and 2026-04-22-ayman-architect-mode-3as both have clean frontmatter, suggesting the Review 4 /improve Notion task (34cf7d4936d181409cd2e35ebabfac30) is being respected manually even before the skill change ships.
• The I12 hybrid-format relaxation from the 2026-04-24 /improve autonomous run is doing its job. Apr 21-22 hybrid entries (stratechery-john-ternus, every-mini-vibe-check-claude-design, every-bread-in-ai-sandwich) all scored 13/13 here without ceremony — the audit invariant change cleared the false-positives Review 4 flagged.
• Zero new audit failures expected on this cohort once audit catches up. All 30 files have the required structural sections (frontmatter, why-in-vault post-fix, mapping, related). No I3, I8, I9, I11 risk visible. One systemic-adjacent observation: 5 files needed why-in-vault added — that’s ~17% of the cohort missing it pre-fix, still the dominant single failure mode (same as Reviews 1, 2, 3, 4).

Process / improve recommendations

• No new /improve task warranted. The recurring “missing why-in-vault” pattern is real but already addressed by Reviews 1-3 process recommendations and the audit’s I9 invariant. The existing fix-during-self-review loop is closing the gap; the structural fix would be enforcing why-in-vault at write-time in every authoring skill (process-newsletter, process-youtube, process-inbox, build-project), which is a larger systemic change worth bundling rather than filing as another one-off.
• Consider archival workflow for the Tim Ferriss promo clip. The entry self-diagnoses the issue and points at the existing Notion task. No new ticket needed; just resolve when ready.

Pre-existing structural items (carrying)

• Cohort backfill 347f7d49-36d1-81eb-b0ec-f9e067db0320 (still active per Review 4)
• I10 batch-rename pass (Review 3 recommendation, still not executed)
• Promo-clip duration threshold tightening (348f7d49-36d1-81ac-b9d8-fad3ae60ce72)

Author advisor note

This is the cleanest cohort to date. The vault’s quality bar is now stable at A-grade with structural fixes confined to “did the why-in-vault section get written.” No new drift detected. The two prior /improve tasks from Review 4 (X-article frontmatter contract, audit-log pruning) remain valid but should ship before another self-review cycle so we can measure their effect cleanly.

improve_processed: 2026-04-27

Review 6 — 2026-04-26

Reviewer: Ray (AI COO) Scope: 30 entries from 06-reference/ modified in last 7d (window 2026-04-19 → 2026-04-26), —fix mode, audit-aware Max score: 13

Headline numbers

• Entries reviewed: 30
• Average score: 12.40 / 13 (post-fix; pre-fix was 12.20)
• Median score: 13 / 13
• Trend vs Review 5 (2026-04-24): slight regression (12.40 vs 12.77). Two D-grade entries this cycle vs one C-grade last cycle. Drivers: a new internal-review entry filed without the standard reference-doc structure (squarely-current-state-review) and a YouTube/podcast entry missing its why-in-vault block (wai-starship-never-returned). Both fixable; both fixed.
• Fixes applied: 3 edits across 2 files
  • 2026-04-24-wai-starship-never-returned-spacex-history.md — added why-in-vault section
  • 2026-04-25-squarely-current-state-review.md — added why-in-vault, full mapping section (5 specific RDCO connections), and 4 wikilinks to squarely-puzzles project hub + sister Sanity Check pieces
• Manual review needed: 1 entry (2026-04-21-tim-ferriss-90-days-black-belt.md) — same archive-or-merge candidate flagged in Review 5; entry is a 5-min Tier-1 promo clip from a parent episode, self-flagged as low-value. NOT auto-fixed per skill rule (thin content). Recommend archive when parent Michelle Khare interview is filed, or hold for the promo-clip-duration-threshold tightening (Notion task 348f7d49-36d1-81ac-b9d8-fad3ae60ce72).
• Audit-failed entries from window: 12
• Double-signal entries: 2 (audit-failed AND scored ≤9 / had structural attention items)
  • 2026-04-25-squarely-current-state-review.md — audit flagged I3, I8, I9, I10; self-review pre-fix scored 4/12 (D). Post-fix: 12/12 (A). The double-signal correctly identified the original ingestion was malformed for an internal-review doc — the file used type: reference but skipped the reference-doc structure entirely. Now repaired.
  • 2026-04-24-wai-starship-never-returned-spacex-history.md — audit flagged I3, I9; self-review pre-fix scored 10/13 (B), missing why-in-vault. Post-fix: 13/13 (A).
• Audit-aware bumps applied: 12 (every audit-failed file in the window was checked against this self-review; only the 2 above were actual semantic failures, the other 10 are audit false-positives from known systemic issues — see Systemic patterns below)

Grade distribution (post-fix)

Entries needing attention

1. 2026-04-25-squarely-current-state-review.md — Pre-fix Score 4/12 (Grade D) ⚠️ audit-failed

   • Pre-fix issues: missing why-in-vault (0/3), missing mapping section (0/3), zero wikilinks (0/2)
   • Audit invariants failed: I3 (newsletter-format/sponsored fields — N/A for internal-review but audit doesn’t know that), I8 (missing Mapping section), I9 (missing Why section), I10 (filename sender slug doesn’t match internal-review source)
   • Fixed: added why-in-vault block, added Mapping section with 5 specific RDCO connections (design-system inheritance, MAC trademark-first sequencing, author-identity narrative, domain-mismatch as Teton-class drift, A+ Content patterns reuse), added 4 wikilinks to Squarely project hub + sister SC pieces.
   • Post-fix: 12/12 (A). The I3/I10 audit fails persist because the audit doesn’t yet know type: reference + source: internal-review is a legitimate non-newsletter shape — flag for /improve to consider an audit-side carve-out for source: internal-review (similar to what’s likely needed for source: synthesis).

2. 2026-04-24-wai-starship-never-returned-spacex-history.md — Pre-fix Score 10/13 (Grade B) ⚠️ audit-failed

   • Pre-fix issue: missing why-in-vault (0/3); had Episode summary but no explicit “Why this is in the vault” header
   • Audit invariants failed: I3 (newsletter-format missing — YouTube content; audit invariant should soften for content_type:podcast), I9 (no Why section)
   • Fixed: added why-in-vault section anchored to the iteration-cadence-as-moat candidate concept and the design-as-feedstock pattern, cross-linked to the Apr 21 Flight 12 sister episode.
   • Post-fix: 13/13 (A).

3. 2026-04-21-tim-ferriss-90-days-black-belt.md — Score 7/13 (Grade D) — NOT auto-fixed (archive candidate)

   • Same entry flagged in Review 5. Tier-1 promo clip (5 min) from a longer parent interview; self-flags as low-value in its own Mapping section. No audit failures (frontmatter is technically clean for a YouTube-clip entry).
   • Action: hold for archive when parent Michelle Khare episode is filed, OR resolve via the promo-clip-duration-threshold tightening Notion task 348f7d49-36d1-81ac-b9d8-fad3ae60ce72.

Top entries (template-grade)

1. 2026-04-21-practical-engineering-teton-dam-failure.md — 13/13. The template for failure-case-study entries: 4 dense Mapping sub-sections explicitly cross-linked to existing concept docs (binary-decision-around-continuous-probability, operational-definitions, layered-defense-architecture); sponsor block disclosed; 8 wikilinks; the pattern every other dam-failure or process-control entry should be measured against.
2. 2026-04-23-moonshots-elon-cursor-bet-claude-kills-saas-openai-departures.md — 13/13. Long-form podcast entry with 10 RDCO mapping bullets, 16 cross-links, sponsor block disclosed.
3. 2026-04-22-garry-tan-skillify-it-workflow.md — 13/13. X-article entry with full table comparison of RDCO state vs. Tan state across the 10-step skillify checklist; converts external content into a concrete RDCO upgrade roadmap.

Systemic patterns

• The audit log carries 10 false-positive failures in the cohort that the self-review confirms are not actual semantic failures. This is the dominant pattern this cycle:

  • Stratechery / Every / AlphaSignal hybrid-format I12 fails (3 entries: stratechery-john-ternus, every-mini-vibe-check, alphasignal-claude-live-artifacts). Same pattern Review 4 caught and the 2026-04-24 /improve autonomous run patched (I12 now accepts ## Issue contents OR ## Curation section). All three entries have one or the other; audit-log entries pre-date the patch. Confirms the /improve Notion task on audit-log pruning (34cf7d49-36d1-81ce8a01d6916f550dc8) is needed — without it, every weekly review re-surfaces stale failures.
  • X-article I3 fails (3 entries: ayman-architect-mode-3as, garry-tan-skillify-it-workflow, jaya-gupta, neil-xbt-claude-laptop-5k-month). All four have either complete frontmatter (garry-tan-skillify and ayman have newsletter_format) or are X articles where content_type: x-article is the natural shape and newsletter_format shouldn’t apply. The Review 4 /improve Notion task on X-article ingestion frontmatter contract (34cf7d49-36d181409cd2e35ebabfac30) needs to ship to either (a) require the field on X articles or (b) carve them out of the audit. Currently authors are inconsistent — some include the field, some don’t, and the audit treats the inconsistency as drift.
  • Filename slug mismatch I10 fails (2 entries: notboring-great-blue-frontier, notboring-wdo-190-curation). Both source values are “Not Boring” with capital N + space; filename slug is notboring. Real false positive — slug normalization (lowercase + remove spaces) would clear it. Worth filing as a small audit hygiene fix.

• Two new genuine semantic failures emerged this cycle (both fixed). Both involved missing Why-in-vault sections — same dominant single-failure-mode flagged across Reviews 1-5. The fix-during-self-review loop continues to catch them; the durable structural fix is enforcing why-in-vault at write-time in every authoring skill.

• A new failure surface: internal-review docs filed as type: reference without the reference-doc structure. squarely-current-state-review is the first entry of this shape in the cohort and it scored D pre-fix. The author (Ray) wrote it as an internal-state-snapshot doc but filed it under 06-reference/ because the founder’s instruction was “review the four surfaces” — there’s no 04-tooling/-equivalent home for a Squarely-state-snapshot. Worth deciding whether internal-review docs go in 06-reference/ (then they need to inherit the reference-doc shape, which means why-in-vault + mapping) or get their own home (e.g., 01-projects/squarely-puzzles/state-reviews/). This is a small foldering decision worth making explicit before the next state-snapshot lands.

Process / improve recommendations

• No new /improve task warranted that isn’t already filed. The two open Review 4 /improve tasks (X-article frontmatter contract, audit-log pruning) would clear most of the false-positive noise this cycle. Worth chasing those before Review 7.
• Consider a tiny internal-review carve-out in the audit script: when source: internal-review (or source: synthesis), skip I3 / I10 entirely and treat I8 / I9 as REQUIRED but allow them to be the only structural requirements. Would close the squarely-current-state-review false-positive surface for this category of authoring.
• Foldering decision pending: internal-state-snapshot docs (Squarely surface review, future MAC-state-review, future SC-state-review) — file under 06-reference/ (current behavior, requires reference-doc shape) OR 01-projects/<bet>/state-reviews/ (cleaner home, free of audit pressure). Ben’s call.

Pre-existing structural items (carrying)

• Cohort backfill 347f7d49-36d1-81eb-b0ec-f9e067db0320 (still active per Reviews 4-5)
• I10 batch-rename pass (Review 3 recommendation, still not executed)
• Promo-clip duration threshold tightening (348f7d49-36d1-81ac-b9d8-fad3ae60ce72)
• X-article frontmatter contract /improve task (34cf7d49-36d181409cd2e35ebabfac30)
• Audit-log pruning /improve task (34cf7d4936d181ce8a01d6916f550dc8)

Author advisor note

Vault quality bar is holding at A-grade. The structural fixes are now consistent enough that the dominant noise-source is the audit log surfacing already-fixed structural drift (the audit-log-pruning /improve task closes this) and the audit not knowing about non-newsletter ingestion shapes (X articles, internal-review docs, synthesis docs). Both are knowable invariant-side fixes. The single new pattern worth the founder’s attention is the internal-review/state-snapshot folder decision — pick a home before the next one lands and the mapping-section discipline either becomes mandatory by structure or unnecessary by category.

improve_processed: 2026-04-27

/improve autonomous run — 2026-04-27

• Reviews processed: 2 (Review 5 2026-04-24, Review 6 2026-04-26)
• Low-risk fixes applied: 2 edits to ~/.claude/scripts/audit-newsletter-outputs.py
  • Internal-review / synthesis carve-out: skip I3 + I10 when source: internal-review or source: synthesis (closes squarely-current-state-review false-positive surface)
  • Slug normalization for I10: hyphen-collapsed source comparison so filename notboring matches source Not Boring (closes Not Boring false-positive surface)
  • Verified by re-running audit on 2026-04-19 → 2026-04-27 window: both false-positive classes cleared. Remaining failures are real signals (X-article + podcast/YouTube content lacking newsletter-shape fields), covered by separate carrying tasks.
• Notion tasks closed: 1 (34df7d49-36d1-8145-955d-e3a588065d33 /improve proposal: audit-newsletter-outputs.py over-flags non-newsletter docs — fix shipped, marked Done)
• Structural changes queued: 2
  • 34ff7d4936d181cc891ff4323fdaff45 — Enforce why-in-vault at write-time in all authoring skills (Both, Medium, Ops). Addresses the dominant single-failure mode across all 6 reviews. Affects 4+ skill files.
  • 34ff7d4936d181d286ffc417e0daadb7 — Foldering decision: where do internal-state-snapshot docs live? (Founder, Low, Ops). 06-reference/ vs 01-projects//state-reviews/ — founder’s call.
• No-ops (already-filed or already-fixed patterns): 4 carrying items (X-article frontmatter contract, audit-log pruning, promo-clip duration tightening, cohort backfill — all on the board, no re-queue)

Review 7 — 2026-05-03

Reviewer: Ray (AI COO) Scope: 30 entries created/modified 2026-04-26 → 2026-05-03 (excluding 06-reference/transcripts/) Window flag: --since 7d --limit 30 --fix Audit pre-failure set (in window, intersecting reviewed cohort): 16 files (per ~/.claude/state/newsletter-audit-log.md, 19 audit runs in window)

Top-line numbers (post-fix)

• Entries reviewed: 30
• Average score: 12.1/13 (post-fix; was 11.2/13 pre-fix). Trend: 11.2 → 12.1 (+0.9 from this cycle’s fixes). Holding A-grade-dominated.
• Grade distribution (post-fix):
  • A (12-13): 28 (93%)
  • B (10-11): 0 (0%)
  • C (8-9): 0 (0%)
  • D (<8): 2 (7%) — both are non-content artifact types (04-finance/2026-04-pulse.md, 04-finance/index.md) that the criteria don’t really apply to; recommend scope exclusion
• Fixed: 8 entries
• Audit-failed in window cohort: 16
• Double-signal entries (audit + self-review both flagged): 5

Files actually fixed this cycle

Double-signal entries (audit + self-review both flagged) this cycle

Top entries (template-grade, 13/13)

1. 2026-04-30-mitohealth-founder-5-layer-agent-native-company-loop.md — concept doc; 13/13. Strong load-bearing-data + sharp founder mapping.
2. 2026-04-30-meta-ads-cli-agent-native-launch.md — 13/13. Tight, focused, good cross-link density.
3. 2026-05-02-khairallah-ai-automation-playbook.md — 13/13. Concise (72 lines), explicit RDCO mapping.
4. 2026-04-25-squarely-current-state-review.md — 13/13. The internal-review carve-out (shipped after Review 6) is now paying off — this entry passes cleanly.
5. 2026-04-30-stratechery-amazon-earnings-trainium-commodity.md — 13/13. Newsletter-format-perfect.

Systemic patterns

• Why-in-vault drift returned (3 of 5 C-grade fixes were missing-why). The Review 6 /improve task to enforce why-in-vault at write-time (34ff7d4936d181cc891ff4323fdaff45) hasn’t shipped yet — until it does, podcast/YouTube assessment notes from the long-form-interview shape (Dwarkesh, Moonshots, IndyDevDan) will keep skipping the section. Surfacing as DECISION in this report — should this be auto-fixed at /process-youtube write-time?
• Audit-side false positives are mostly cleared. The 2026-04-27 /improve carve-out for source: internal-review is working — squarely-current-state-review and mitohealth-founder both pass cleanly post-fix. Remaining audit failures cluster on (a) podcast-shape content lacking newsletter_format (real shape mismatch — should the audit ALSO carve out content_type: podcast | tutorial | interview?) and (b) I10 slug mismatches on backfill-discovery / bookshelf docs which use source: strings the audit can’t normalize.
• Long blockquote segments triggering copy-paste flag (3 entries). Sinclair, IndyDevDan, Dwarkesh all have transcript-style timestamped segments that exceed the 350-word paragraph threshold. These ARE original synthesis blocks, not copy-paste — the heuristic is over-triggering. Worth tightening the heuristic OR accepting the false positive as cheap.
• Two finance-pulse artifacts scored D (2026-04-pulse.md, 04-finance/index.md). Neither is a content/reference entry — they’re financial reports with their own templating. Self-review criteria don’t apply meaningfully. Recommend excluding 04-finance/ from the self-review scope going forward, or building a separate criteria set for finance artifacts.

Process / improve recommendations

• Ship the Review 6 why-in-vault enforcement task — it would have caught all 3 podcast-shape fixes this cycle without manual intervention.
• Consider extending the Review 6 internal-review carve-out to cover content_type: podcast | tutorial | interview for the I3 newsletter-format check. Each of those is a content-type the audit currently treats as a newsletter-shape failure.
• Scope refinement: explicitly exclude 04-finance/ from the self-review file enumeration in the SKILL.md, since finance-pulse artifacts have their own structural conventions.

Author advisor note

Bar holds at A-grade (~83% A, 0% C in the content cohort). Two systemic items worth founder eye: (1) why-in-vault auto-enforcement at write-time would compress the manual fix queue substantially, and (2) the audit could grow a podcast/tutorial/interview carve-out with the same shape as the internal-review one shipped 2026-04-27. Neither is urgent — vault quality is high — but both are knowable, low-risk skill-side fixes.

Review 8 — 2026-05-04

Reviewer: Ray (AI COO) Scope: 14 entries newly created since Review 7 (2026-05-03 strategic-conversation outputs + 2026-05-04 DEW #268). Window 2026-04-27 → 2026-05-04, but de-duped against Review 7’s already-fixed cohort. Window flag: --since 7d --limit 30 --fix Audit pre-failure set (in window, intersecting reviewed cohort): 6 files (per ~/.claude/state/newsletter-audit-log.md, run 2026-05-03T12:57:44 — 9 files audited, 6 failed)

Top-line numbers (post-fix)

• Entries reviewed: 14 (new docs since Review 7; carry-over A-grade fixes from Review 7 not re-scored)
• Average score: 12.6/13 post-fix (vs 12.1/13 in Review 7 — +0.5 trend up)
• Grade distribution (post-fix):
  • A (12-13): 13 (93%)
  • B (10-11): 0 (0%)
  • C (8-9): 1 (7%) — writewithai-claude-design-guidelines.md is a deliberate skip-stub (status: skipped)
  • D (<8): 0
• Fixed: 3 entries (icbd-holdings, tampa-target-shortlist, app-store-submission-canonical-sources)
• Audit-failed in window cohort: 6
• Double-signal entries (audit + self-review both flagged): 1 (writewithai)

Files actually fixed this cycle

Double-signal entries (audit + self-review both flagged) this cycle

Top entries (template-grade, 13/13)

1. 2026-05-03-shopify-eng-ucp-technical-architecture.md — clean technical-blog assessment with explicit Mapping subsections per RDCO surface; 7 wikilinks; concise (123 lines).
2. 2026-05-04-dataengineeringweekly-268-agents-replacing-search.md — newsletter-format-perfect with explicit Sponsorship section + curation breakdown; ties Doug-Turnbull retrieval finding to Tristan-Handy BI thesis as a generalization data point.
3. 2026-05-03-alphasignal-single-vs-multi-agent-systems.md — exemplar bias-flagging (separates Lambda paid placement from internal AlphaSignal workshop CTA).
4. 2026-05-03-sytaylor-ucp-merchant-owned-agentic-checkout.md — explicit ## ⚠️ Bias disclosure block calling out the Tempo-CEO forwarding channel; the founder explicitly flagged the bias and the doc records it.

Systemic patterns

1. Skip-stub class needs rubric carve-out (NEW). 2026-05-03-writewithai-claude-design-guidelines.md is a deliberate status: skipped stub — the founder filed the article-skip rationale itself rather than do a full assessment because the article is sales-funnel filler that adds nothing to RDCO design systems. The rubric currently scores it as C (no Mapping, no Links by design), and the audit flags it (I8/I9/I11). Both are false positives for skip-stub class. Recommend: add a status: skipped carve-out — when frontmatter has status: skipped, only score Why-in-vault (which becomes “Why this is skipped”) and require nothing else. Mirrors the Review 6 internal-review and Review 7 podcast/tutorial carve-outs.
2. Project-doc conciseness false positive (CARRY-OVER + EXTENDS). Review 7 flagged 04-finance/ for scope-exclusion. Tampa shortlist (01-projects/acquisitions/) and Physical-AI opportunity-map (01-projects/physical-ai-thesis/) hit the same class — multi-page project shortlists/opportunity-maps where >300 lines is the correct shape, not bloat. Recommend: extend the scope refinement to cover 01-projects/<dir>/<dated-doc>.md shortlists/opportunity-maps OR raise the conciseness threshold to 400 for type: opportunity-research and type: acquisition-research. Tampa post-fix passes the rubric in every other dimension; the conciseness fail is the only blocker to A-grade and it’s structural to the doc’s purpose.
3. Why-in-vault enforcement (CARRY-OVER from Review 6 + 7). Did not regress this cycle — every new newsletter-shape entry has the heading. The Review 6 /improve task to enforce at write-time still hasn’t shipped, but the manual discipline is holding for the small set of new entries. Continue tracking.
4. Bias/sponsor flagging is improving (POSITIVE TREND). Three entries this cycle (heyrico, sytaylor, alphasignal-single-vs-multi-agent) have explicit ## ⚠️ Bias disclosure or ## ⚠️ Sponsorship headings beyond the frontmatter sponsored: true flag, with bias source named (Tempo-CEO forwarding channel; Lambda paid placement vs internal AlphaSignal workshop). This is exactly the discipline the rubric wants and is now becoming default behavior.
5. Audit pre-failure set is shrinking on new content. 6 of 14 entries (43%) had any audit failure this cycle, down from 16 of 30 (53%) in Review 7 normalized to the 14-entry sample. I10 (filename-sender-mismatch) and I3 (missing newsletter_format on essay/X-article shape) remain the dominant patterns — both are structural issues with the audit’s invariants, not the content. Same systemic recommendation as Review 7: extend audit carve-out to content_type: essay | company-research | reference-catalogue for the I3 newsletter-format check.

Process / improve recommendations

• Skip-stub carve-out (NEW): add status: skipped rubric exception to ~/.claude/skills/self-review/SKILL.md step 2 and to the audit script’s I8/I9/I11 checks. Lowest-risk, highest-leverage fix this cycle.
• Project-doc conciseness (NEW): extend the Review 7 04-finance/ scope refinement to also cover 01-projects/*/<dated>.md shortlists/opportunity-maps, OR introduce a type:-keyed conciseness threshold (>400 lines for acquisition-research, opportunity-research).
• Re-surface Review 6 + 7 carry-over patterns (CARRY-OVER): the why-in-vault auto-enforcement task (347f7d49-36d1-81eb-b0ec-f9e067db0320-class) and audit content-type carve-out (podcast | tutorial | interview) are still on the list. Founder hasn’t acted yet; not urgent because new-entry discipline is holding manually.

Author advisor note

Bar continues to hold at A-grade (93% A; only C-grade is a deliberate skip-stub). Trend line is +0.5 vs Review 7 — the manual discipline founder + Ray are running on new entries is sufficient even without the Review 6 /improve task shipping. The two NEW systemic items worth founder attention this cycle: (1) skip-stub rubric carve-out (cheap, ships in one /improve pass), and (2) extending the project-doc scope-exclusion to 01-projects/. Neither requires founder judgment beyond a yes/no — both are skill-side fixes Ray can ship if approved.

/improve autonomous run — 2026-05-04

Reviews processed: Review 7 (2026-05-03) + Review 8 (2026-05-04). Both marked improve_processed: 2026-05-04 via inline HTML comment under their headings.

Low-risk fixes applied (4):

1. Audit script — podcast/tutorial/interview carve-out (I3, I4). Edited ~/.claude/scripts/audit-newsletter-outputs.py to add is_youtube_long_form flag (matches content_type: podcast|tutorial|interview) and skip both required-field check (I3) and type-equals-reference check (I4) for that class. Mirrors the 2026-04-27 internal-review carve-out pattern. (Review 7 systemic pattern.)
2. Audit script — skip-stub bypass (I8, I9, I11). Same script: added is_skip_stub flag (matches status: skipped) and bypassed the Mapping-section, Why-in-vault, and Sponsorship-section checks for that class. (Review 8 systemic pattern #1.)
3. Audit script — Changelog block. Documented all three carve-outs (2026-04-19, 2026-04-23, 2026-04-27, 2026-05-04) in the module docstring so future changes have provenance.
4. Self-review SKILL.md — scope exclusions + skip-stub special pass class. Edited step 1 to exclude 04-finance/ (Review 7), exclude 01-projects/*/<dated>.md (Review 8), and treat status: skipped files as a special pass class (effective max 4/4: only Frontmatter + No copy-paste + Conciseness count). Added Changelog section documenting the change. (Review 7 + 8 systemic patterns combined.)

Files modified (3):

• ~/.claude/scripts/audit-newsletter-outputs.py — three carve-outs + changelog
• ~/.claude/skills/self-review/SKILL.md — scope exclusions + skip-stub class + changelog
• ~/rdco-vault/01-projects/self-review/review-log.md — improve_processed markers + this report block

Structural changes queued (0 new): the why-in-vault auto-enforcement at /process-youtube write-time (Review 7 + 8 carry-over) is already on the Notion board as task 34ff7d4936d181cc891ff4323fdaff45 from the 2026-04-27 /improve cycle (https://www.notion.so/34ff7d4936d181cc891ff4323fdaff45 — Status: To Do, Priority: Medium, Owner: Both, Project: Ops). No re-queue needed.

No-ops: none. All four candidate low-risk fixes shipped. Audit script syntax-validated via py_compile.

Pre-flight audit signals:

• rdco-doctor reports 39 dark skills (mostly on-demand, no action needed), 7 missing-script references (all in xcode-build-* skill family — out of scope for this run, separate ticket), 3 high-overlap pairs in swift-* family (descriptive overlap is real but each is a distinct triggering shape — no action).
• eval-mine reports 3 frustration hits in 14d window — 2 of 3 are about external products (“doesn’t work” referring to a third-party flag and KDP email), 1 about /loop. Below action threshold (33% concentration on /loop is one hit in 14d).

Review 9 — 2026-05-08

Reviewer: Ray (AI COO) Scope: 30 most-recent entries in 06-reference/ mtime ≥ 2026-05-01 (full window 2026-05-01 → 2026-05-08, capped at --limit 30). Window flag: --since 7d --limit 30 --fix Audit pre-failure set (in 7d window): 47 distinct files across 12 audit runs since 2026-05-01T12:56:50; 22 of those 47 fall within the 30-file scored cohort.

Top-line numbers (post-fix)

• Entries reviewed: 30
• Average score: 11.9/13 post-fix (vs 12.6/13 in Review 8 — −0.7 trend down on a larger, less hand-curated cohort)
• Grade distribution (post-fix):
  • A (12-13): 22 (73%)
  • B (10-11): 8 (27%)
  • C (8-9): 0
  • D (<8): 0
• Pre-fix snapshot (before —fix pass): A:18, B:8, C:1, D:3 (avg 10.9/13)
• Fixed: 4 entries (all C/D)
• Audit-failed in window cohort: 22 of 30 (73%)
• Double-signal entries (audit + self-review both flagged) post-fix: 0 — all 4 C/D entries were also audit-failed (double-signal pre-fix); fixes resolved both signals on each.

Files actually fixed this cycle

Auto-fix tally

• Frontmatter additions: 3 files (10 individual fields added or renamed)
• Cross-link additions: 1 file (Related section added to writewithai-voc-landing-page; 5 wikilinks)
• Mapping rewrites: 1 file (writewithai — heading rename only, the body content was already RDCO-specific and concrete)
• Why-in-vault additions: 4 files (3 added new sections, 1 canonical heading rename on alphasignal-stanford)

Flagged for manual review (NOT auto-fixed)

• None this cycle. Of the 8 B-grade entries, none had blockers requiring rewrite — all were 12/13 with a copy-paste-wall heuristic flag (likely false positives — see Systemic Pattern #2) or a single missing frontmatter field on entries scoped outside the 7d audit window. Per skill rule, copy-paste walls and thin-content require manual review; none warranted action this cycle.

Top entries (template-grade, 13/13)

1. 2026-05-06-osmani-cognitive-surrender.md — clean source assessment, full frontmatter, concrete RDCO mapping, multiple wikilinks.
2. 2026-05-06-alphasignal-anthropic-finance-gpt45-grok43.md — newsletter-format-perfect with bias flagging, sponsorship section, curation breakdown.
3. 2026-05-06-ship30for30-3-mistakes-marketers-ai.md — sibling of the audit-failed 3-outcomes-marketers-want-ai from same sender same week; this one passes everything cleanly. Useful template reference for the watch-mode subagent prompt.
4. 2026-05-05-innermost-loop-singularity-and-regulators.md, 2026-05-05-naval-find-simplest-thing.md, 2026-05-05-naval-good-products-hard-to-vary.md, 2026-05-05-naval-judgment-decisive-skill.md, 2026-05-05-naval-specific-knowledge.md, 2026-05-05-jorgenson-almanack-of-naval-ravikant.md, 2026-05-05-every-codex-native-apps.md — multiple 13/13 entries from the same Naval-corpus batch indicate the L5-thesis-validation work pipeline is producing template-quality output.

Systemic patterns

1. Watch-mode subagent prompt drift on /process-newsletter (CONFIRMED CARRY-OVER, ESCALATING). The 2026-05-07 batch produced 4 of 7 files with multiple-invariant violations: writewithai-voc (I3+I4+I8+I9), ship30for30-grow-linkedin (I3+I4+I9+I11), ship30for30-3-outcomes (I8), alphasignal-stanford (I3+I9). The defect class is consistent: subagents are emitting format instead of newsletter_format, newsletter-reference / newsletter instead of type: reference, omitting author, omitting sponsored: true|false, and using non-canonical Why-section headings. This is the systemic pattern context-noted in this run’s task input — confirmed across the full 7d window, not just one batch. Recommend an /improve pass to harden the watch-mode subagent prompt template: enforce canonical frontmatter field names + canonical body-section headings as a checklist the subagent must validate before writing. Today’s pre-fix double-signal count was 4 (all 4 C/D entries were also audit-failed); we cleaned that to 0 with —fix, but the recurrence rate per batch is the underlying problem.
2. Copy-paste-wall heuristic is over-detecting (NEW). My local scorer flagged 8 entries as copy-paste-wall based on a >80-word paragraph without inline structure. Sampling those (stratechery-joanna-stern-interview, naval-nothing-ever-happens-is-over, dec-ai-not-replacing-curious-developers, wai-spacex-starship-flight-12, tim-ferriss-naval-ravikant-1-2015, ship30for30-claude-code-marketing, practical-engineering-physics-behind-thumb-trick, innermost-loop-event-stream, stratechery-microsoft-apple-earnings) shows most are original prose summaries by the founder or by the /process-* skills, not literal copy-paste from source. The heuristic needs refinement: paragraphs starting with a section-header phrase (“Episode summary”, “Source”, “Core thesis”) are common original-summary patterns and shouldn’t trip the wall detector. Recommend skill-side: keep the heuristic as a flag-only signal, never auto-fix (already correct), but add a 1pt downgrade only when wall + sibling-article-paste-detection both fire. For Review 9 I treated copy-paste-wall flags as soft (entries with otherwise complete content + this single flag still rated A 12/13 with the 1pt deduction).
3. Audit-failed cohort intersection rate is high but mostly resolvable on individual entries (POSITIVE). 22 of 30 entries (73%) had any audit failure since 2026-05-01, but the violations are concentrated in: I3 (newsletter_format / sponsored field) on Naval-corpus and book-class entries that aren’t actually newsletters (they’re book-derived snippets and tweetstorm assessments), and I10 (filename-sender-mismatch) on book-class entries (jorgenson-almanack-of-naval-ravikant source=‘book’ / slug-head ‘jorgenson’). The I3/I10 pattern on Naval-corpus is the same class as the 2026-04-27 internal-review carve-out and the 2026-05-04 podcast/tutorial/interview carve-out. Recommend extending the audit script’s content-type carve-out to include content_type: book-excerpt | tweetstorm | aphorism-collection | corpus-index (or whatever the actual frontmatter type values are on these files) so the audit doesn’t generate noise on a class that’s structurally different from newsletter-shape content. This is a Review 7 + 8 carry-over pattern that’s now hitting the Naval batch.
4. Naval-corpus batch produced 7 of 22 A-grade entries (POSITIVE). Despite triggering audit I3 on every Naval entry (because the audit expects newsletter_format on type: reference), the self-review rubric scored them A because they have proper Mapping + Why-in-vault + Cross-links + concise. The audit is the false-positive signal here, not the self-review. This is what double-signal entries SHOULD look like — a single signal alone may indicate a noisy invariant, not a content problem. Reinforces Pattern #3.

Process / improve recommendations

• Watch-mode subagent prompt hardening (NEW, HIGH PRIORITY) — /improve pass on the /process-newsletter watch-mode subagent prompt to enforce canonical frontmatter + canonical body headings via a pre-write validator. Today’s 4-defect batch is the third week in a row this class of drift has shown up; manual —fix is no longer the right intervention.
• Audit content-type carve-out extension (CARRY-OVER, MEDIUM) — extend ~/.claude/scripts/audit-newsletter-outputs.py to skip I3/I10 on content_type: book-excerpt | tweetstorm | aphorism-collection | corpus-index (Naval batch class). Same pattern as the 2026-05-04 podcast/tutorial/interview carve-out.
• Copy-paste-wall heuristic refinement (NEW, LOW) — soft signal only, no auto-fix; consider gating on header-phrase-prefix detection to suppress original-summary false positives.

Author advisor note

Avg dropped 0.7pts vs Review 8, but the cohort tripled in size and was less hand-curated — 30 entries spanning 7 days vs 14 in Review 8. The post-fix distribution (73% A, 27% B, 0 C/D) is still strong. The standout finding is confirmed: watch-mode subagent prompt drift is now a third-week-in-a-row pattern, and manual —fix is patching the same defect class repeatedly. This is the right shape for an /improve pass and qualifies as DECISION NEEDED for founder review.

improve_processed: 2026-05-08

/improve autonomous run — 2026-05-08

• Reviews processed: Review 9 (2026-05-08). Marked improve_processed: 2026-05-08 above.
• Low-risk fixes applied: 2
  1. ~/.claude/skills/process-newsletter/SKILL.md — Mode 4 watch-mode dispatch prompt now enforces a canonical-schema pre-write checklist on each subagent: required frontmatter field names (newsletter_format not format, type: reference literal, author, sponsored: true|false bool, optional sponsor_entity + ## ⚠️ Sponsorship body section when sponsored), required body headings verbatim (## Why this is in the vault, ## Mapping against Ray Data Co, ## Related with ≥2 wikilinks). Source: Review 9 systemic pattern #1 (3-week running drift).
  2. ~/.claude/scripts/audit-newsletter-outputs.py — extended I3/I10 carve-out to content_type: book-excerpt | tweetstorm | aphorism-collection | corpus-index (mirrors the 2026-05-04 podcast/tutorial/interview carve-out shape). I8/I9/I4 still required on these classes. Source: Review 9 systemic pattern #2.
• Structural changes queued: 0
• No-ops: 1 — Review 9 systemic pattern #3 (copy-paste-wall heuristic refinement) was tagged LOW priority by the reviewer with a recommendation to “keep as flag-only signal, no auto-fix” — current state already matches that recommendation, no change required.
• Audit verification (window 2026-05-01): pre-fix and post-fix both report 74 audited / 21 pass / 53 fail / 88 violations — bit-identical because no current files in the window have the new content_type values set yet. The carve-out is structural-prep: a sanity test on a synthetic content_type: book-excerpt file confirms zero violations under the new logic, so future Naval-corpus / book / tweetstorm assessment notes correctly tagged with content_type will bypass I3/I10. No regression on existing files.