In a world of abundant generic skills, the eval is not a quality-assurance gate — it is the product. An agent output that carries a verified lift score (base model vs. base model + skill, N-sample, pass-rate ± stddev) is structurally different from one that does not. The distinction is measurable, repeatable, and resistant to imitation by anyone who hasn't built the eval harness.
The market evidence (July 2026)
Three independent signals converge on the same claim:
Anthropic's public plugin marketplace has zero efficacy review. As of [[2026-07-21-anthropic-plugin-marketplace-brigade-strategy]], Anthropic reviews plugins for security but not performance. All 255+ listed plugins received identical approval regardless of whether they improve agent outcomes. The gate is structural (does it run?) not economic (does it work?). RDCO's eval-gated build discipline is the only mechanism that distinguishes performance from presence.
One vendor's 106-skill restructure isn't evidenced by any benchmark. [[2026-07-20-prompt-over-specification-capable-model-degradation]] documents a CAF provider's architectural pivot resting entirely on a vendor assertion — no A/B, no lift score, no hold-out — that contradicts what controlled benchmarking shows about over-specified prompts on capable models. The skill reorganization is presented as evidence of quality when it is evidence of effort.
Eval-gated endpoints convert 33× better. [[2026-07-23-executable-agentic-info-product-pricing-scan]] measured 368 probes across agentic info products: skills/tools endpoints with published eval evidence converted at 33% vs. 0–0.8% for live-data feeds at identical price points. The eval isn't just a signal — it is the primary purchase driver.
Why this is in the vault
Captures the July 2026 convergence of three independent signals into a single, evidence-backed claim that positions RDCO's eval discipline as a market primitive, not an internal QA preference.
Mapping against Ray Data Co
The brigade's execution-eval station (base model A/B, N-sample lift, noise-band threshold) is the mechanism. It was designed for build-time quality control. The market evidence says it is also the primary commercial differentiator in the agentic skills market. The implication: every skill RDCO publishes or deploys into a client environment should carry its eval score as a first-class artifact — not buried in internal notes but surfaced in the deliverable.
Tension to resolve: the eval station currently produces a pass/fail gate. Surfacing lift scores as deliverable artifacts requires a lightweight format contract (score + method + N + confidence interval). That format doesn't exist yet. See [[2026-07-18-assessment-brigade-v2-phase-gate-design]] for the phase-gate design that could carry it.
Related concepts
[[verifier-as-epistemology]] (the epistemic foundation — why verification is prior to belief) · [[2026-07-04-loop-engineering-vs-agent-brigade]] (the brigade as the structural home of the eval station) · [[2026-05-10-harness-moat-two-layers-portability]] (moat layer framing)