01-projects/agentify-3000

IMPLEMENTATION NOTES

2026-10-04·implementation-notes

Implementation notes: [[2026-10-04-starter-story-pat-walls-assessment|Starter Story]] index (phase 1)

Pipeline

  1. robots.txt (Allow /, Disallow /admin). /sitemap redirects with a 302 to d1coqmn8qm80r4.cloudfront.net/sitemaps/sitemap.xml.gz (19,345 URLs). Filtering to ^/stories/<slug>$ gives 3,299 unique URLs.
  2. fetch.py: Python stdlib, sequential, request starts at least 1.1 s apart (measured 1.23 s/req), Chrome user agent, timeout 30 s, at most 2 retries with backoff of 5 s then 20 s, and a hard stop on 3 or more 403/429 responses in the last 10 requests. It caches gzipped HTML (356 MB for 3,299 pages) under /private/tmp/claude-501/starter-story-index/raw/ and logs every request to fetch_log.jsonl. Result: 3,299 × 200, zero retries, zero blocks, 68 min.
  3. parse.py → build.py → CSV/JSON + stats.json. Copies of all three scripts and stats.json are in data/scripts/. build.py paths point at the tmp working dir, and the raw cache is in /tmp only (ephemeral).

Where the public fields live (findings from inspecting 3 pages)

Design decisions

Validation

I hand-checked 10 random pages (seed 7) plus the 3 inspection pages, comparing the parsed output with the visible header text. Two bugs were fixed before the full run: names truncated in the Business tab ("Entrepreneurs on ...", now taken from the About tab), and idea fields marked present when #tab-7 was empty. One bug was fixed after the run: idea headers that begin "Become …" or similar instead of "Start …". There were zero parse exceptions on 3,299 pages.

Deviations

Open questions for phase 2 (scoring)

  1. Taxonomy: 21% are "other" or blank. Should we derive an agentification-relevant taxonomy (service vs product vs software, labor intensity) from description + idea + model via an LLM classifier pass?
  2. Unit of analysis: story or business? Dedupe on website (2,442 distinct) and keep the latest follow-up's revenue?
  3. Revenue trust: drop revenue_suspect rows, or cap them? Weight header vs headline?
  4. Opportunity signal: team_size per $ revenue (labor intensity) looks like the strongest public proxy for "agentifiable". Confirm with the founder.
  5. Foil signal: high-revenue solo/AI-native businesses (e.g., the $18K/mo app built with AI coding tools) are natural content foils. Should we tag video and 2025-26 stories separately?
  6. Idea-level stats cover only about 22% of rows. Fill the rest by joining on /ideas/<slug> pages (public, a separate polite crawl), or skip?

Phase 2: scoring (started 2026-10-04 14:03 EDT)

Decisions applied (from the brief)

  1. Unit = business. Dedupe key = normalized website host (www stripped; path kept for generic hosts such as etsy.com, amazon.com, apps.apple.com so distinct shops do not collapse). Fallback = normalized name; for the 129 breakdown/video pages with no header name or website, the name is derived from the slug (uuid and "-breakdown" suffix removed) and used as the key. Numbers come from the latest story (by published) that has a revenue figure; revenue_story_url records which. All story URLs are kept per business.
    • Result: 3,261 rows -> 2,566 businesses (2,437 by website + 129 by derived name). The brief expected ~2,440; the extra 129 are the nameless breakdown/video pages, kept because they are real businesses (Bannerbear, Chatbase, Browse AI, ...). 507 businesses have more than one story. 2,118 have a revenue figure; 59 carry revenue_suspect (any story flagged).
  2. revenue_suspect businesses are kept and flagged, and excluded from every aggregate (medians, opportunity ranking).
  3. Classification by model (fanout subagents, fixed prompt, strict JSON, validator + retry). Scheme derived first from a stratified sample.
  4. Opportunity ranking and phData exclusion as specified; [[DESIGN-scribble-works|Scribble Works]] adjacency flagged separately.
  5. Foils: named, linked, aggregates and short facts only.

Deviation from the builder discipline: the vault is a git repo with no remote (git remote -v is empty), so there is no PR. Outputs are written directly under 01-projects/agentify-3000/, uncommitted, for the parent to review.

Archetype scheme (written 2026-10-04 14:06 EDT, before any classification run)

Sample: 146 businesses, seed 42, strata = ss_category x revenue bucket (<10k, 10-100k, >=100k, n/a), proportional allocation with a floor of 1 for strata of 8 or more. Scheme and rubric verbatim (also the body of the classification prompt):

# Archetype scheme (derived 2026-10-04 14:06 EDT from a stratified sample of 146 businesses, seed 42, strata = ss_category x revenue bucket; written BEFORE any full classification run)

Assign exactly one code per business. Classify by HOW the business makes and delivers what it sells, not by the Starter Story category label.

| code | name | definition (what you sell, how it is delivered) |
|---|---|---|
| agency | Agency & productized service | B2B done-for-you work (marketing, SEO, ads, design, dev, video, content, lead gen, social selling) delivered remotely by a team on retainers, projects, or "unlimited" subscriptions. The deliverable is work product, not software. |
| saas | B2B SaaS & developer tools | Software sold to businesses on subscription or usage: SaaS, APIs, Shopify/WordPress/WooCommerce apps and plugins, open-source-plus-paid, hosting control panels. Customers are companies or merchants. |
| consumer_app | Consumer app & micro-tool | Software sold to individuals or prosumers: mobile apps, web apps, AI generators, Chrome extensions, digital planners, Notion templates, dating sites, games. One-off or subscription. |
| dtc_brand | DTC physical product brand | Designs or sources manufactured goods and sells them online under its own brand: apparel, beauty, food and beverage, supplements, gear, devices, med-tech hardware. Production is outsourced or factory-run, not by hand. |
| maker | Handmade & artisan goods | Founder or a small team makes the items by hand (leather, jewelry, candles, art, stickers, custom furniture). Capacity is the maker's hands. Etsy-style. |
| retail | Online retailer & reseller | Sells third-party or multi-brand goods: curated shops, online grocers, dropshipping, wholesale distribution, resale, print-on-demand with no own design IP. Margin comes from sourcing and curation. |
| marketplace | Marketplace & platform | Two-sided: matches supply and demand and takes a cut, listing fee, or lead fee. Booking platforms, tutoring platforms, influencer matching, lead-gen directories, outsourcing marketplaces, exchanges. |
| media | Content, media & audience | Blog, newsletter, podcast, YouTube, affiliate or review site, author or publisher. Monetized by ads, affiliate, sponsorship, paid subscriptions, or book sales. The product is attention. |
| edu | Courses, info products & communities | Sells packaged knowledge at scale: online courses, cohort programs, ebooks, audio lessons, exam prep, paid communities, edtech learning platforms. Many buyers, same material. |
| coaching | Coaching, consulting & professional services | Sells expert time 1:1 or in small groups: life/business coaches, consultants, fractional execs, accounting, legal, PR, advisory firms. Revenue is bounded by expert hours. |
| local | Local & in-person service | Delivered physically at a place or to a doorstep: cleaning, junk removal, gyms and training, spas, tours, venues (axe throwing), restaurants, events, trades, and franchises of these. |
| ops | Operations, logistics & B2B supply | Manufacturing, custom packaging, 3PL and shipping, print, white-label production, web hosting and infrastructure, wholesale supply to businesses. Heavy physical or infrastructure assets. |
| other | Other | Only if none of the above fits after honest effort. Must give a reason. Expected under 2 percent. |

Tie-breakers:
- Software that businesses pay for = saas; software individuals pay for = consumer_app. A plugin sold to merchants is saas.
- An agency that also sells a small SaaS = whichever produces most revenue per the description; default agency.
- A blog that mainly sells its own course = edu; a blog monetized by ads or affiliate = media.
- A coach who mainly sells a recorded course = edu; one who mainly sells calls or done-with-you = coaching.
- A brand that also makes items by hand at scale with staff and machines = dtc_brand; by hand by the founder = maker.
- A shop that sells its own branded goods it designed = dtc_brand; one that sells other brands = retail.
- Franchise systems are classified by what the franchisees do (cleaning franchise = local).

## Core-job scoring (fixed rubric)

Five core jobs, each scored 0-3 for how much an AI agent can carry TODAY (October 2026) in THIS business, given current agent capability (web and software actions, writing, voice and chat, data work, drafting and scheduling; no robots, no legal standing, no physical presence):

- 0 = agent cannot carry it: physical hands, licensed signature, in-person trust, taste the market pays for.
- 1 = agent assists: drafts, schedules, summarizes, but a human does the core act.
- 2 = agent does most of it; a human reviews, approves, or handles exceptions.
- 3 = agent runs it end-to-end with light oversight.

Jobs:
1. find: find customers (demand gen, SEO, content marketing, outbound, ads, partnerships, listings).
2. sell: sell and convert (qualify, demo, quote, close, checkout, onboarding, renewals).
3. deliver: make and deliver the product or service (the core production or fulfillment work).
4. support: customer support and success (tickets, chat, returns, retention).
5. admin: run the business (bookkeeping, invoicing, hiring and scheduling, compliance, reporting, vendors).

agent_share (0-100) = the share of the business's total weekly labor hours an agent could carry today. Weight the jobs by where the hours actually go in this archetype (a maker's hours are mostly deliver; an agency's are mostly deliver and find; a SaaS's are build, support, find). It is not a plain mean of the five scores.

moat = the one phrase naming what a human still has to bring for this specific business (e.g. "hands-on leather craft", "in-person trust with local homeowners", "founder's taste and audience", "regulatory approvals", "capital and inventory").

breaks = one line: what breaks first if an agent-native competitor tried to run this business with near-zero staff.

Classification prompt (verbatim; {INPUT}/{OUTPUT} are per-batch file paths, {SCHEME} is the scheme block above)

You are classifying small businesses for an "agentification" study. You get a JSON file of businesses (each with id, name, website, description, headline, category, monthly_revenue_usd, team_size, idea, idea_model). Classify EVERY business in the file using ONLY the scheme and rubric below. Do not browse the web. Do not skip any id. Be consistent: the same kind of business must get the same scores regardless of its revenue.

INPUT FILE: {INPUT}
OUTPUT FILE: {OUTPUT}

Steps:
1. Read the input file with the Read tool.
2. For each business produce one JSON object (shape below).
3. Write the full JSON array to the output file with the Write tool. The file must contain ONLY the JSON array (no prose, no markdown fences). Output every id exactly once.
4. Reply with one line: "done N items" where N is the count written.

OUTPUT OBJECT SHAPE (every key required, exact names, exact types):
{
  "id": "<id from input, unchanged>",
  "archetype": "<one code: agency|saas|consumer_app|dtc_brand|maker|retail|marketplace|media|edu|coaching|local|ops|other>",
  "archetype_reason": "<max 12 words; required if archetype is other>",
  "b2x": "<B2B|B2C|both>",
  "jobs": {
    "find":    {"s": <0-3 int>, "r": "<one line, max 15 words>"},
    "sell":    {"s": <0-3 int>, "r": "<one line>"},
    "deliver": {"s": <0-3 int>, "r": "<one line>"},
    "support": {"s": <0-3 int>, "r": "<one line>"},
    "admin":   {"s": <0-3 int>, "r": "<one line>"}
  },
  "agent_share": <0-100 int>,
  "moat": "<one phrase, max 8 words>",
  "breaks": "<one line, max 20 words: what breaks first if an agent-native competitor ran this with near-zero staff>",
  "phdata_overlap": <true|false>,
  "scribble_adjacent": <true|false>
}

Flags:
- phdata_overlap = true if the business sells data engineering, analytics, BI, data platform (Snowflake, Databricks, dbt), ML/AI consulting, AI implementation services, or custom AI/agent building for clients. A SaaS product that merely uses AI is false. A marketing or SEO agency is false.
- scribble_adjacent = true if the business serves kids, parents of young children, K-12 education, printables, worksheets, coloring, children's books, toys or games for children, or homeschool.

=== ARCHETYPE SCHEME ===
{SCHEME}

Pipeline and batch stats

Spot-check (30 random classified businesses, seed 2026, judged by the parent against the description)

Deviations and judgment calls

  1. 2,566 businesses, not ~2,440. The 129 breakdown and video pages with no header name or website are kept as businesses under a slug-derived name (Bannerbear, Chatbase, Browse AI and others are real). Attempted fold-in by normalized name matched 0 of them to a website-keyed business. Known dedupe miss: Boot.dev appears as the website-keyed business and again as the video "My Coding Game Makes 1M Per Month"; merged by hand in opportunities.md. Expect a handful of similar misses among the 129.
  2. Phase-2 suspect rule added. Phase 1's rule missed Remote Tools at $71.6M/mo (8 people) and TruTech Tools at $33.9M/mo. Added: revenue per person above $400k/mo or revenue above $20M/mo. 7 more flagged (also Stardew Valley, Catamaran Guru, Juniper Biologics, Mermar Electronics, Paperwallet). Suspect total 59 -> 66; suspect_reason column records which rule.
  3. "other" = 37 (1.4%), under the 2% target. 31 are YouTube and breakdown pages whose public description is a channel teaser with no business in it; 6 are real but unfittable (a SaaS holding company, a real-estate fund, a private lender, a nonprofit, a one-off island purchase, a vague production company). They are kept with reasons and excluded from nothing except by their own lack of revenue.
  4. Opportunity ranking merges one pair. Rankings.io and Hennessey Digital are the same kind (law-firm SEO agency) and sit at #10 and #12 by score; shown as one entry with both links so the top 20 is not three SEO agencies in a row. GR0 kept separately as the general case.
  5. No PR. ~/rdco-vault has no git remote. Files are written in place, uncommitted.
  6. Dedupe revenue precedence = latest story with a revenue figure, which can pick a 2024 follow-up over a 2026 video with a bigger number only if the video lacks a figure. For 507 multi-story businesses this is what the brief asked for.
  7. Capital factor uses idea-level startup cost where present (22% of rows) and an archetype prior elsewhere; it is the weakest leg of the opportunity score and is labeled as such in opportunities.md.

Open questions for the founder

  1. Is the agency gap (48% carryable, deliver 1.5, lowest B2B revenue per person) the thesis to pursue, or is the "already agent-native solo business" story (Pep AI, StandOut CV, GMass) the better Denominator spine?
  2. Should the 22 phData-overlap businesses be reviewed by hand? The flag is conservative (Salesforce consulting is flagged) and some may not be real conflicts.
  3. The scores are from 300-character descriptions. Re-scoring the 30 foils from the full public story page (not paywalled body) would make the posts safer. Worth a second pass before publishing any of them?
  4. Scribble Works: TeacherMade (worksheets to screen, $75k/mo) is the inverse of the SW principle. Is that a competitor signal or a validation that teachers pay for worksheet tooling?

Acceptance verification (2026-10-04 14:38 EDT)