01-projects/ray-ops/2026-11-10-fable-bakeoff/blind

task2 sw release B

Scribble Works release system: diagnosis, design and plan to drain the backlog

As of Tue 2026-11-10. Checked against origin/main @ 8fc7f2e0, the 45 open PRs, the studio state files and live HTTP checks. Nothing was changed.

TL;DR

The pile-up has one root cause: the studio built games faster than anything could publish them, and the publish path was broken for 30 days. The founder had withdrawn back pressure because he believed games were already "released via database and feature flags." That wasn't true.

The fix is three separate lanes, each with one owner:

  1. Integrate. A bot merges hidden games and low-risk changes without him.
  2. Deploy. Unchanged: auto-deploy on green main, but skip deploys when nothing visible changed.
  3. Publish. A database flag, flipped by him with one tap. No PR.

On top of that: guardrails coded into the planner (stop-the-line, WIP caps), not left as questions for him. His Day-0 cost is about 30 minutes; after that, about 5 minutes a day.


1. Why it jammed

# Root cause Evidence
R1 Publishing has been broken since 10-11. The website reads the catalog from D1 (wrangler.toml:149 CATALOG_SOURCE = "d1", cut over 10-08 under #413 phase 3). The release step that syncs D1 has no credential, so it prints a warning and exits 0. Any game flipped to live in meta.yaml is built but still 404s. deploy-website.yml step "Sync the game catalog to production D1": if [ -z "$CLOUDFLARE_API_TOKEN" ] … exit 0. gh secret list --env website-production shows only CLOUDFLARE_API_TOKEN. Today: letter-sound-hunt, caceria-de-sonidos, ten-frame-race and carrera-de-marcos-de-diez all return 404. The token was first asked for at escalations.md:591 on 10-11 (day 30 now). Founder ruling of 10-07 in a catalog-parity.yml comment: "no D1 secret yet; production syncs are hand-run for now." Nobody hand-ran them.
R2 #413 stopped halfway, leaving three sources of truth. Under D5 option 1, a page only exists if it was pre-built from meta.yaml status: live. Visibility is decided by the D1 row. The daily Worker still reads its bundled files (#481; #485 is still a draft). The publish endpoint and buttons (phase 4) were never built, so "ACCEPT" still means PR → merge → release → sync. Issue #413 comments from 10-08 (D5 → option 1, "publishing = D1 flip + site rebuild"). Issue #481 is open. launch-games/SKILL.md phase 8 is still "draft PR → ready → merge".
R3 His approvals had no path to execution. ACCEPT arrives by iMessage, and the agent has to run gh pr ready / gh pr merge / gh pr close. The classifier denies these as "Unrequested Commit in a Connected App", correctly, since iMessage isn't his terminal. His own 10-19 merge command never ran. escalations.md 10-18 10:29 (gh pr ready 368 denied), 10-21 item 5 (gh pr close 373 denied), 10-20 ("your 10-19 merge command … has NOT run"). Allow rules exist (~/.claude/settings.json:55-56) and denials happened anyway. #368, #377, #451, #452 and #461 were ACCEPTed and merged only as hidden on 10-28. All 10 slugs are pending-review on main and 404 today.
R4 Every game PR conflicts with every other one. Each game PR rewrites 6 committed generated files: src/data/catalog-status.js, creation-catalog.{js,json}, slot-manifests.{js,json} and workers/feedback-intake/src/games.js. Most also add a line to the scripts/scan-secrets.mjs ALLOW list and append to an 813 KB IMPLEMENTATION-NOTES.md. Merging one PR re-conflicts all its siblings, so the merge cost is O(N²). gh pr view 484/498 --json files. The batch-9 "recipe" is a 36 KB bash script plus union.py, allow_union.py and allow_verify.mjs (merge-prep/). Its dry run stopped #447 on a notes multiset failure. Currently 22 of 45 open PRs are CONFLICTING.
R5 Drafts never get real CI, so merging means replicating CI by hand. CI skips gate and smoke on drafts (ci.yml gate if: … !github.event.pull_request.draft). The studio only opens drafts because gh pr ready gets denied. Agents then spent whole slots rebuilding CI locally in scratch "merge trains". 41 of 45 PRs are drafts, with statusCheckRollup = SUCCESS,SKIPPED,SKIPPED. merge-prep/2026-10-26-merge-train.md §2–3.
R6 Every merge is a production release, so merging is treated as risky. A green main triggers the Website release. affectsDaily (scripts/deploy/daily.mjs:15) matches content/, src/ and scripts/, so a hidden game also redeploys the daily Worker. That triggered the charter's §8d merge cap and the Actions-minutes worry. October: 830 runs, ~4,770 wall-clock minutes against 3,000 included (escalations 10-30). Back-to-back merges make releases skip as stale (rounds.md:823). The 10-28 train had 2 overnight verify failures (runs 40810343550, 40833564955). Batch 9 has been "waiting on the §8d cap question" since 10-31.
R7 No back pressure, on a false premise. 10-11 founder: "are we not releasing them via database and feature flags now? We shouldn't have the PR back pressure." The backpressure rule was withdrawn. The studio kept running 4 slots a night while his verdicts stopped. The last founder studio verdict was "ACCEPT #461" on 10-21. #449 was shown in 12 packets with no tap. Main holds 115 pending-review slugs (about 57 games), against 93 live and 14 retired. 0 merges from 10-12 to 10-27, then a one-off 43-PR train on 10-28/29, then 0 again.
R8 The real blocker drowned in other asks. escalations.md is 160 KB. The nightly review-and-plan slot rotates through 7 angles and adds 3–5 founder decisions every night: dice, the age line, art budget, the 2-10 copy, and so on. The D1 token, the only true blocker, sat next to "is a die roll OK?" The throttle was asked 5 times with "no default." escalations.md 10-18 → 11-01. Brief line of 11-01: "Three asks … none default on silence."

In one sentence: the founder thought the database flag made PRs cheap, and the agents thought his silence meant "keep building." Neither side could see that publishing had been broken since 10-11, because the broken step reported success.


2. Target design: three lanes, each with one owner

                ┌───────── LANE 1: INTEGRATE (bot, no founder) ─────────┐
studio builder ─► PR (critic stamp @ head) ─► CI ─► merge-policy ─► main (game = pending-review, hidden)
                └────────────────────────────────────────────────────────┘
                ┌───────── LANE 2: DEPLOY (auto, as ratified 09-12) ────┐
main green ─► Website release (skip if diff = hidden-only) ─► verify ─► D1 sync (FAIL if no token)
                └────────────────────────────────────────────────────────┘
                ┌───────── LANE 3: PUBLISH (founder, one tap/game) ─────┐
packet card LIVE/KILL ─► publish.yml ─► preflight ─► D1 flip + event ─► rebuild ─► verify 200/410 ─► auto-revert on fail
                └────────────────────────────────────────────────────────┘

Game lifecycle

PR → (bot merges) → pending-review (hidden: 404, no PDF route, not customizable) → founder LIVE → live → founder RETIRE → retired (410, R2 object kept).

Lane 1: Integrate (studio-merge.yml)

Build this by generalizing the existing guides-auto-merge.yml. That workflow has never actually fired: the only guides-publish PR, #359, was merged by hand.

The bot runs as a GitHub App, not GITHUB_TOKEN. Pushes and merges made with GITHUB_TOKEN don't trigger CI or release workflows. It works one PR at a time (a single concurrency group, effectively a merge queue). For each eligible PR it:

  1. Marks the PR ready, so full CI runs. Agents no longer need gh pr ready.
  2. Merges main into the branch.
  3. Re-runs the generators and commits the output. This is needed until the generated files leave git (Week 3).
  4. Waits for gate and smoke.
  5. Runs gh pr merge --squash --match-head-commit.

Merge tiers are decided from the PR's file paths by scripts/merge-policy.mjs; labels are only a hint.

Tier What Merges when
T0 Only docs/**, studio/notes/**, root docs (ci-check-docs-only allowlist) CI green
T1 hidden game Only new content/games/<slug>/**, docs/evidence/<slug>/**, studio/notes/**, plus the per-slug scan-allow file. Every meta.yaml is pending-review. No existing game is touched. CI green and PR body critic: ACCEPT @ <sha> equals the head sha
T2 engineering src/, scripts/, functions/ excluding the T3 paths CI green and critic stamp at head. Listed in the morning brief as FYI.
T3 founder .github/**, scripts/deploy/**, wrangler*.toml, migrations*/**, auth/account/billing functions, privacy/terms/pricing/public positioning copy, any file of a live game, CODEOWNERS His GitHub "Approve" (CODEOWNERS + "require review from Code Owners") plus CI

Breakers:

Lane 2: Deploy

Keep the 09-12 rule (deploy on green main), with three changes:

Lane 3: Publish

Stage A: interim, available as soon as the D1 token exists.

  1. Ray builds a launch PR with launch-games: render, R2 upload, manifest, meta flips.
  2. The PR body is the review card: thumbnails, PDF links, critic flags. It touches live state, so it's T3.
  3. He taps Approve in GitHub mobile. That one tap covers the whole batch. To drop a game he comments "drop slug".
  4. The bot merges, the release deploys, the D1 sync writes, and a new verify-live workflow step asserts 200 for each new live slug and 410 for each retired one. On failure it opens a revert PR.

This replaces "Ray runs gh because of an iMessage" with "he approves in GitHub, the bot executes."

Stage B: target. This is #413 phase 4, rescoped.

What the founder approves, and what it costs

Action Who Cost to him
Merge a hidden game, docs, or low-risk engineering Bot (T0/T1/T2) 0. FYI line only.
Workflow, deploy, migration, secret, billing or legal-copy change; any change to a live game Founder, GitHub Approve (T3) ~1 min per PR, about 2–3 a week
Pending → LIVE, or → RETIRED Founder, one tap per EN game ~30 s per game; 10-card packet ≈ 5 min/day
Emergency unpublish of a live game (404, broken PDF, safety) Ray, then iMessage him right away 0
Credentials, money, the charter Founder Rare

3. Guardrails so this doesn't recur

All of these are code in the planner, CI or workflows. None is a question put to the founder.


4. Migration plan

Day 0: founder, about 30 minutes. The studio stays paused.

  1. Create two Cloudflare API tokens scoped to the accounts D1: D1 Edit and D1 Read. Ray's login can't create tokens. Then run:
    gh secret set CLOUDFLARE_CATALOG_D1_TOKEN --env website-production --repo RayDataCo/scribble-works
    gh secret set CLOUDFLARE_CATALOG_D1_READ_TOKEN --env website-production --repo RayDataCo/scribble-works
    
    Ray then runs gh workflow run catalog-parity.yml -f apply=true and confirms the 4 dead slugs return 200.
  2. Create and install a GitHub App "scribble-works-release-bot" with contents, pull requests and actions write. Store its key as an Actions secret.
  3. Rule on decisions D1–D8 (section 6).
  4. Run one close command for the PRs to close (section 5). This is his terminal; Ray's attempts were denied:
    for n in 306 332 351 352 373 387 407; do gh pr close $n --repo RayDataCo/scribble-works --comment "Closed in the 2026-11 release-system reset; branch kept"; done
    
  5. Check the October Actions bill at github.com/organizations/RayDataCo/settings/billing. Ray can't read it.

Week 1: build Lane 1 and fix Lane 2 (Ray, interactive, T3 approvals by him)

PR Content Tier
W1-a studio-merge.yml, scripts/merge-policy.mjs + tests, CODEOWNERS. He applies the branch-protection change (code-owner review). T3
W1-b Conflict removal. scan-secrets reads per-slug scripts/scan-allow.d/<slug>.txt. IMPLEMENTATION-NOTES.md is frozen; new entries go to notes/<date>-<topic>.md and notes-index generates the index. T3 (touches the secret scanner)
W1-c Release changes: D1 sync fails instead of skipping; skip hidden-only deploys; verify-live step. Merge #435. T3
W1-d Stage A launch PR for the 5 already-ACCEPTed games: #368 who-lives-here/quien-vive-aqui, #377 sailboat-snips/recorta-el-velero, #451 sky-sudoku/sudoku-del-cielo, #452 who-can-fly/quien-puede-volar, #461 trace-the-letter-c/repasa-la-letra-c. He already said yes to these. T3 Approve

Then the bot drains the backlog in the order in section 5. Expect ~30 PRs × ~15 min of CI ≈ 450 Actions minutes, spread over 3–4 days at 10 merges a day. Do not run the old 36 KB batch-9 recipe. It redeploys production 9 times and leaves the structural problem in place.

Week 2: Stage B publish

Week 3: restart the studio, then clean up

Restart only when all four hold:

Restart shape: 2 slots a night (1 game + 1 drain/engineering). Go back to 4 only when hidden-unverdicted is under 20. Re-arm scheduled-jobs.txt:145 on his word.

Cleanup: stop committing generated files (generate them in the website prebuild, the daily deploy and the feedback-intake deploy), and drop the "generated files are current" CI check. Move the PDF render and R2 upload into CI.


5. Triage of the 45 open PRs

Close: 7 PRs (his Day-0 command)

PR Why
#306 Hosted-generation "Revise build-the-word", draft since 09-16, waits on a "Good decision" that never came. Re-issue from main if still wanted.
#332 Monetization plan as a PR (conflicting). A plan belongs on issue #224; paste it there.
#351 PR previews: stale, conflicting, needs a website-preview token, not on the critical path. Turn it into an issue for later.
#352 Checklist wording, superseded by the W1 merge-policy docs.
#373 R28 diagnosis doc; R28 was resolved 10-04. He was already asked on 10-21.
#387 Batch-1 founder fixes that conflict with the parent_notes model on 5 live games. Re-queue as 5 separate live-game refresh PRs (T3) after Stage A.
#407 Egg to Butterfly: critic REVISE at cap and CI FAILURE.

Merge via bot, T0 docs

#371, #454. #371 is already non-draft and green.

Merge via bot, T2 engineering (after ready-CI)

PR Note
#379 render-games hardening. Merge first, because Stage A renders depend on it.
#487 test-seo, one file.
#428 Art library Phase S. Merge after W1-b. Its open #396 questions have defaults. It gates #427.
#356 (gate code only) Split: the cut-card gate code merges as T2. The six page fixes, which include the live rocket games, become a T3 refresh plus relaunch.

Needs his GitHub Approve (T3)

PR What it needs
#435 404 smoke. Merge after the token and heal, otherwise it turns every release red, by design.
#485 Daily D1 switch. Add the R68 test first.
#464 Site copy. Its privacy paragraph is flagged "founder should read this exact paragraph."
#467 Spacing fix. It touches ci.yml.
#382 Playset title page of the paid daily PDF, 6 weeks old. Re-qualify, then approve or close.
#356 (page fixes) Live game refresh, plus relaunch of build-a-rocket / arma-el-cohete.

Hidden game merges (T1, bot): 27 PRs, in this order

Batch PRs Notes
G1 #444 #442 #449 #486 #488 #489 #490 #491 #492 Clean in the 10-31 dry run. Main hasn't moved since (8fc7f2e0).
G2 #445 #479 Landed in the dry run. Concept questions (opposites as reading_words; grade-1 blending) go on the packet card, not into a merge hold.
G3 #493 #494 #495 #496 #497 #498 10-30 → 11-01, MERGEABLE, not yet dry-run. The bot qualifies them.
G4 (ordered) #427 after #428; #429 then #450; #383 then #381 (ES art dependency); #380, #447, #471, #484 Hidden merge is reversible, so open questions don't hold the merge. They become LIVE blockers on the card: dice (#380, #447), shoe sole (#471), and hand-coded table/chair against his 10-01 real-art rule (#484, which needs a Quiver redraw before LIVE). #447 needs W1-b first (notes multiset failure).
Hold #430 He tapped REVISE after the critic's ACCEPT. Merge only if the head shows that REVISE was addressed; otherwise close it and re-queue.

End state: 0 open studio PRs. About 84 hidden games are waiting for packet verdicts, and that queue is now the honest bottleneck.

Packet order: first the 5 already-ACCEPTed games (W1-d), then the oldest hidden games from the 10-28 train, at 10 a day. That clears the backlog in about 8–9 sittings.


6. Decisions the founder must make (ranked, each with a recommendation)

  1. D1: Provision the D1 Edit and D1 Read tokens today. BLOCKER, with no alternative. This reverses his 10-07 "no D1 secrets, hand-run syncs" ruling, because nobody hand-runs them. Without it, every LIVE produces a 404.
  2. D2: Authorize the release bot and the merge tiers T0–T3. This means creating the GitHub App and turning on code-owner review. Recommend: yes. Bear case: Ray's T2 code merges no longer wait for him. The mitigations are the critic stamp, CI, the breaker, and the FYI line in the brief.
  3. D3: Who can flip LIVE. Recommend: the founder only, one tap per EN game, with the ES sibling following. Revisit after 30 days of eval-ledger data comparing critic ACCEPT with his verdicts. The alternative, "critic ACCEPT with no flags goes live after 72 h of silence," is not recommended yet: #430 got his REVISE after the critic had accepted it.
  4. D4: D1 owns publish state, and meta.yaml status is removed. This settles #413 D4 and is what makes publishing work without a PR. Recommend: yes.
  5. D5: Ratify G1–G4 as automatic rules, with the numbers in section 3. This is the throttle he was asked about 5 times, now tied to a working publish path. Recommend: yes. Bear case: no new games until packets clear. That is intended.
  6. D6: Charter edits. Strike §8d's merge-count cap (charter:466) and replace it with a weekly Sunday hardening slot. Amend §8c ("nothing reaches production overnight") to allow hidden merges (ruling 38). Recommend: yes.
  7. D7: Cut the review-and-plan slot from 7 nightly angles to 1 weekly. Recommend: yes.
  8. D8: Let Ray unpublish a live game on a P1 defect without asking first (live → pending-review), then iMessage him right away. Recommend: yes.

Everything else in escalations.md either becomes a packet-card option (dice, opposites, #471 sole, #484 art, #449) or is closed as superseded (merge cap, overnight merges, throttle, close #373).


7. Risks and things to verify before building