Scaffolding vs Harness — The Erosion Axis
Why this note exists
Founder asked (2026-08-01) what "scaffolding for agentic models" means and how it differs from a harness, after hearing the claim that new base-model releases erode all scaffolding over time.
The vault already cuts this material on one axis (portability — see the two-layer moat note). This note adds the orthogonal axis: durability across model releases. The two are not the same question, and conflating them produces bad build decisions.
The distinction
Harness — the runtime loop. Takes model output, executes the tool call, feeds the result back, manages the context window, handles retries, persists state. Claude Code, OpenCode, Codex, and Pi are harnesses in this sense.
Scaffolding — the domain-specific structure wrapped around the model so it performs one particular job well. Decomposed steps, prompts, forced output schemas, retrieval pipelines, hand-written verification passes, checklists.
Harness is how the agent runs. Scaffolding is how much of the thinking you pre-chewed for it.
⚠️ Terminology collision inside our own vault
2026-05-01-claw-vs-harness-engineering.md uses "harness engineering" in the OpenAI-Codex-team sense: a tracker-anchored orchestration architecture (Linear/Notion as source of truth, orchestrator polls, worker spawns, state writes back). That is a different referent from "harness = the runtime loop" used here.
Both usages are live in the field. Neither is wrong. When it matters, say which one you mean. This note uses the runtime-loop sense throughout.
Why scaffolding erodes and harnesses don't
Every piece of scaffolding is a bet that the model cannot do that step. When the next model can, the scaffolding becomes dead weight — and sometimes worse than dead, because it constrains a model whose judgment now exceeds the hand-written steps it is forced through.
Harnesses solve a different class of problem. Models still cannot execute their own tool calls, manage their own context window, or persist their own state. Those are I/O and plumbing problems, not intelligence problems. A more capable model makes a good harness more valuable, not less.
This is Sutton's bitter lesson applied to agent engineering: general methods that leverage computation beat hand-crafted domain knowledge, eventually. Scaffolding is hand-crafted domain knowledge.
The erosion claim is too broad as usually stated
Three kinds of scaffolding. Only one dies.
| Kind | What it is | Fate on the next model release |
|---|---|---|
| 1. Capability scaffolding | Compensates for what the model can't do yet. Decompose because it can't plan; verify because it hallucinates; few-shot because it misreads the format. | Erodes. Every release deletes some. |
| 2. Particular knowledge | What no model can know at any capability level. Founder's phone number; iMessage renders no markdown; the Calendar API timeZone label lies and the UTC offset is canonical; the refund rule must sit below the transfer rule in the books classifier or profit silently inflates. |
Compounds. GPT-9 doesn't know any of it. |
| 3. Taste and policy | What "good" means here. Sharp verdicts on shared links; no derivative Sanity Check pieces; em-dash discipline. | Compounds. Preference, not capability. |
The podcast claim is directionally right about category 1 and wrong if applied to 2 and 3.
The test to run on anything we build
Is this here because the model is dumb, or because the model can't know this?
- Dumb → build it cheap and expect to throw it away. Do not gold-plate a workaround for a limitation with a shelf life.
- Can't-know → this is the asset. Invest.
This is the same cut Thariq's Claude-5 guidance makes when it says to drop the long workflow-descriptive step lists but keep the founder-specific gotchas. It is the operating principle behind the in-flight skills-tuning project (rewrite the 7 daily-firing skills goal-oriented, preserve the particular knowledge).
How this relates to the portability axis
2026-05-10-harness-moat-two-layers-portability.md cuts the same material by portability — Layer 1 (universal, transferable to another operator) vs Layer 2 (personal-fit accumulation).
These axes are orthogonal, and it's worth holding both:
| Portable to another operator | Not portable | |
|---|---|---|
| Durable across releases | Harness plumbing; policy patterns | Founder-specific particular knowledge |
| Erodes across releases | Generic capability workarounds (prompt tricks, decomposition templates) | Bespoke workarounds for one founder's one workflow |
The bottom-left quadrant is the trap: portable but depreciating. It looks like a productizable asset — it is the easiest thing to package and sell — and it has the shortest shelf life of the four. Any Ray-as-a-Service / starter-kit packaging decision should price that quadrant as a wasting asset, not a moat.
Live application — KwikTrip Sprocket (onsite Wed 2026-08-05)
Applies directly to the engagement, and is worth being explicit with the client about:
- The document-preprocessing pipeline being built to fix diagram retrieval is capability scaffolding. It exists because multimodal retrieval over technical diagrams is weak today. Depreciating asset.
- The knowledge base itself — which manual covers which equipment serial, what "fixed" looks like on a hot dog roller, which failure modes recur at which store formats — is particular knowledge. Durable asset. No future model ships knowing KwikTrip's equipment fleet.
The strategic point: a client paying for the first and thinking they're buying the second is going to be unhappy in eighteen months. Naming which is which is a credibility move, and it argues for spending the engagement on knowledge-base structure over pipeline cleverness.
Prep brief: [[2026-07-31-kwiktrip-onsite-strategic-prep]]
Open questions
Does category 1 erode uniformly, or do some capability workarounds (e.g. verification passes against hallucination) persist longer because the underlying failure mode is structural rather than scale-solvable? Current guess: verification persists longest, because it's checking the model with something other than the model.
✅ RESOLVED 2026-08-19 by [[2026-08-19-agent-scaffolding-generation-survival]] — and it corrects two things above.
- The guess was half right. Verification does not persist as a category. External verification (checking against an artifact that exists outside the model) persists; same-model self-review (self-consistency, self-critique) erodes faster than few-shot, because its benefit is proportional to inter-sample disagreement and each generation supplies less of it.
- Task decomposition is mis-filed as category 1 in the table above. It did not lose value on stronger models — GAIA's controlled comparison found Opus 4.7 the most scaffold-sensitive model tested (27.9pp spread). Decomposition should not be treated as disposable.
- The durable cut is therefore not "is the model dumb, or can it not know this?" (line 61) but what is the scaffolding's input? Input = the model's own output distribution → erodes. Input = something external to the model → does not.
Is there a fourth category — scaffolding that exists for cost reasons rather than capability (routing cheap work to a smaller model)? That erodes on a price curve, not a capability curve, which is a different clock.
Sanity Check candidate
"The scaffolding you're proud of is the scaffolding that's about to die" — the three-category frame plus the portable-but-depreciating quadrant is an original re-frame of a claim currently circulating in podcast form, not a restatement of it. Meets the no-derivative-pieces bar.