"How my non-engineering team at Sentry learned to ship" — Justin Gage
Why this is in the vault
A first-person account of Sentry's marketing team moving ~2500 pages out of a CMS into Git + Markdown so coding agents could touch the whole site — the cleanest external proof yet that "the environment is the product" and that non-engineers ship once the environment is agent-shaped.
The core argument
The trigger was an asymmetry, not a cost line: a Claude Code session did a month of SEO link-fixing in an afternoon on the GitHub-resident parts of the site, then hit a wall at the CMS boundary (the blog and key pages the agent couldn't touch). That asymmetry — agent can update half the site instantly, half not at all — was the real pain, and it was compounding weekly as the team built more workflows around Claude Code. Velocity, not the Contentful bill, drove the migration (though the piece notes Cursor's CMS reportedly cost ~$57k/yr).
Execution: ~2500 pages migrated by 2.5 developers in a two-month window, most code written by Claude Code while engineers planned, reviewed, and directed. They migrated one directory at a time (docs/marketing first, then blog, then complex interactive pages), kept the CMS on a free tier for a soft cutover, and switched frameworks Gatsby→Astro mid-flight (build times 14min → under 6, then under 4 after a Marketo form cache). They consolidated ~200 bespoke pages into 3 templates — deliberately, because a simpler codebase is easier for agents to operate ("the agent doesn't have to learn the quirks of 200 one-off pages; it learns three templates").
The two reusable artifacts:
- Skills as templates — a Skill per page type (landing, product, solutions, blog, cookbook, workshop) that encodes structure, SEO requirements, brand patterns, and component library. It interviews the author for required fields, emits a schema-matching Markdown/JSON file, and opens a PR. Same consistency the CMS templates gave, except (his words, paraphrased) agents actually follow the pedantic rules humans skip.
- The Sentry Cookbook — short, code-first, executable recipes. Built because organic search to long-form blog posts has fallen for two years while LLMs and AI Overviews keep citing structured executable content. The format "collapses the distinction between content for SEO and content for users" — developers and LLMs both want the same shaped answer.
What was harder than expected (the honest section): design fidelity (pixel-matching old pages) took longer than the data migration — Claude is good at functional code, weaker at matching an existing design 1:1; a human-introduced form-validation rule tanked a conversion metric for days; build infrastructure was an under-appreciated failure source; and the team learning curve is real but smaller than expected — "I couldn't write code four months ago," and a handful of marketers now open PRs not because they became engineers but because the environment forced enough fluency to keep up. Next up: scheduled autonomous workflows (a Linear-ticket → Claude → PR loop already shipped for small well-scoped fixes) and AI code review to catch what human reviewers miss in 50-change batches.
Mapping against Ray Data Co
This is a direct, high-fidelity external validation of three RDCO positions at once.
"The environment is the product" / agent-deployer thesis. Sentry's whole migration is the thesis stated as a war story: the bottleneck was never the agent's capability, it was the environment (CMS boundary) the agent couldn't reach. The fix wasn't a better model, it was reshaping the substrate — pages into Git, 200 bespoke pages into 3 templates, page-types into Skills — so the agent could run end-to-end. This is exactly RDCO's wedge ([[2026-05-26-varick-how-to-transform-a-company-with-ai]]): you don't sell the agent, you sell the agent-shaped environment. The "consolidate to 3 templates so the agent learns 3 not 200" move is the load-bearing insight — legibility of the environment is the deliverable. RDCO's own skills/ directory IS this pattern applied to the COO harness ([[2026-05-03-every-context-window-codex-goes-to-work]]).
Non-engineers shipping with Claude Code + Git. The headline claim — a marketing team opening PRs four months after none of them could code — is the strongest public datapoint for the "developer role collapses to director/reviewer" reframe RDCO already filed ([[2026-01-26-every-claude-code-shipping]]). It also sharpens the GTM target: the buyer who feels this pain most isn't the eng team, it's the non-eng function (marketing, ops, growth) sitting behind a SaaS boundary an agent can't cross. That's a cleaner ICP than "data teams" in the abstract.
The founder's phData agentic work. phData's public posture is "you have the tools, we'll help you use them" ([[2026-05-20-phdata-cortex-agents-practice|phData Cortex Agents practice]]). The Sentry story is the same engagement shape one layer down: the value wasn't writing the migration scripts, it was the planning/templating/reviewing that made an agent safe to point at production. A forward-deployed RDCO/phdata engagement is precisely "come reshape your environment so your own non-engineers can ship" — and the Skills-as-templates artifact is a concrete, sellable deliverable, not vapor.
Do-things-that-don't-scale / schlep lens. The two highest-leverage artifacts (per-page-type Skills, the Marketo cache) were unglamorous schlep — codifying pedantic brand/SEO rules, eliminating a class of build failure they'd "been quietly working around for years." The Marketo cache was worth doing "before you need to." That's the GTM tell: the wedge isn't the flashy autonomous loop, it's the boring environment-prep schlep nobody else wants to do, which is exactly where a forward-deployed operator earns trust. The honest "design fidelity is still hard / human error tanked a metric / month one is slower than feels comfortable" caveats are the realistic expectation-setting RDCO should carry into any pitch — don't oversell single-shot agent autonomy.
Net: strong mapping. Reusable in positioning collateral and as phData-engagement framing. Worth lifting the "3 templates not 200" and "environment forced enough fluency to keep up" lines as positioning anchors.
Related
- [[2026-05-26-varick-how-to-transform-a-company-with-ai]] — enterprise-scale statement of the agent-deployer thesis; this is the SMB/single-team version
- [[2026-01-26-every-claude-code-shipping]] — "ship like a team of five," developer-as-engineering-manager reframe
- [[2026-05-20-phdata-cortex-agents-practice]] — phData's "you have the tools, we'll help you use them" posture; same engagement shape
- [[2026-05-03-every-context-window-codex-goes-to-work]] — skills/harness as the productized pattern RDCO built bespoke