The ask: a good, reusable skill-development pipeline. First big use case: Andrew's "Claude Cowork Intensive" departmental-stripes grid (~60 named skills across Marketing / Finance / Sales / Operations / Legal …, anchored to top certs per discipline).
TL;DR
- Use skill-creator for the scaffolding + eval layer, not the content. It standardizes SKILL.md shape, tunes trigger descriptions, and runs the eval/benchmark harness. It is a tool inside the pipeline, not the pipeline.
- The content comes from the cert anchor, run through one translation: certs encode what a practitioner knows; a skill encodes what they do, as a repeatable procedure. Transcribing a syllabus = knowledge dump. We want workflows.
- The pipeline already mostly exists as 4 domain-agnostic seats (spec → tests → author → critic). What's new is a per-domain config for "skill" as the target artifact + the cert-competency seeding step. Fast to stand up.
- Pilot one skill end-to-end first (Finance /
variance-analysis, a strong cert anchor) so you + Andrew can eyeball the quality bar before we batch the grid.
The principle (why this shape)
Thin harness, fat skills. A skill's value is the hard-won domain procedure baked into it — not the boilerplate. The slop-cannon failure mode is letting the model "flesh out" a skill from just its name: you get 60 plausible-sounding, generic skills with no real depth. The whole pipeline is built to force depth in (from the cert source) and gate slop out (via independent critics).
The pipeline — 4 seats, run per skill
We already have these as domain-agnostic skills (pipeline-spec-author, -test-author, -code-author, -critic). We point them at "skill" as the artifact type.
- Spec — input:
{skill name, one-line purpose, department + block context, cert-competency excerpt}. Output: a build spec — the procedure the skill must encode, its triggers, inputs/outputs, and a progressive-disclosure file plan. This is where cert knowledge → agent procedure translation happens. - Tests / acceptance contract — reads the spec only (independent of implementation). Output: concrete scenarios the skill must handle, trigger-accuracy cases (fires on the right asks, not the wrong ones), and a fat-content check (encodes real workflow steps, not generic advice).
- Author — reads spec + tests. Writes the SKILL.md + reference files. skill-creator runs here as the scaffolding engine (consistent shape, description tuning).
- Critic — fans out one sub-agent per axis, returns PASS/FAIL + confidence into a convergence loop (FAIL → back to author with notes, up to N rounds). Axes:
- Triggering precision (skill-creator's eval/benchmark — does it fire correctly?)
- Domain fidelity vs the cert competency (is the procedure actually right?)
- Procedure-not-knowledge-dump (workflow steps, not a syllabus restatement)
- Progressive-disclosure hygiene (size, file layout, lazy pointers)
- No-slop (specificity over plausible-generic)
Output per skill: the SKILL.md + a pre-registered acceptance contract + a critic verdict + an eval score.
Scaling to the grid (the fan-out)
~60 skills on a regular structure is a fan-out, not a 60× manual click-through. Run a pipeline() over the skill list — each skill flows spec → tests → author → critic independently (no barrier between stages), so wall-clock ≈ the slowest single skill, not the sum. Batch by department so each discipline's cert anchor is loaded once.
Quality gates (human-in-the-loop)
- Cert-anchor review per discipline — confirm the source material before building, especially weak anchors (below).
- Pilot sign-off — clear the quality bar on one skill before the batch.
- Batch spot-check — founder/Andrew review a sample per department, not all 60.
Cert-anchor strength map (starting hypothesis — validate with Andrew)
The anchor is uneven, and that determines how much we can lean on it:
- Strong (real accredited bodies of knowledge): Finance (CMA/CFA), Operations/Supply Chain (Six Sigma, APICS/ASCM CPIM-CSCP), Project Mgmt (PMP) if present.
- Moderate: Legal/Compliance (contract-mgmt CCCM/CFCM, privacy CIPP) — real frameworks, narrower coverage.
- Weak (vendor/methodology certs, not accredited BOKs): Sales (Challenger / MEDDIC / Sandler are methodologies) and Marketing (HubSpot / Google / Meta are tool training). Lean on these and those stripes come out shallow + tool-centric. For weak anchors we should supplement with practitioner-workflow sources, not just the cert.
What exists vs what I build
- Exists: the 4 pipeline seats;
/skillify(workflow → skill);/improve(feedback → skill rewrite); skill-creator (scaffolding + eval). - I build: the per-domain "skill" config (spec template + test axes + critic axes), the cert-competency seeding step, and the grid fan-out orchestration. Honest read: a few hours to wire + the pilot to prove it.
Distribution (where the skills land)
Per the CAF architecture call: two rails — org-admin-published skills (Cowork/Desktop/web, the dominant rail for the ~60 sales/ops users) + the Code plugin (the engineers). The pipeline produces the SKILL.md artifacts; distribution is the existing phData plugin/marketplace decision.
The pilot
- One skill, end-to-end: Finance /
variance-analysis(strong CMA/CFA anchor). - Deliverables you see: the spec, the acceptance tests, the authored SKILL.md, the critic verdict + eval score.
- Decision gate: does the quality bar clear? Yes → wire the per-domain config + run the grid in batches by department. No → tune the critic axes and re-pilot.
Open questions for founder / Andrew
- Cert source access — do we have the actual competency outlines/BOKs per discipline, or do we work from public cert syllabi + practitioner sources?
- Where do built skills live — phData Cowork org-published rail, the Code plugin, or both, for the Intensive?
- Weak-anchor disciplines (Sales, Marketing) — supplement with practitioner-workflow sources, or accept tool-centric v1 and iterate?
- Relationship to CAF's 106-skill collapse — is this the same skill-authoring engine, or a separate track? (My instinct: same engine, different artifact set.)