Cloudflare commoditized Jev's licence and interface, not its price or its quality: Clef leads Jev on public benchmarks and falls 5.3 points behind on held-out private tests
The question
"How do Cloudflare's Clef (classification) and Clairic (regression/range) compare to TypeSafe's Jev on pricing, latency, and positioning?"
Asked because Ray Data Co (RDCO) is actively adopting Jev for typed-set judgments, so a hyperscaler moving onto the same primitive is a live dependency question rather than a curiosity. Clef is the classification model and the actual Jev rival; Clairic is regression and extraction. The founder corrected that distinction on 2026-10-19.
Source-quality rules used in this brief
The three axes do not have equal evidence quality, so four labels appear throughout:
- (vendor) - the vendor's own claim. Primary evidence for positioning and for list pricing. Never accepted as evidence of performance.
- (vendor-measured) - a performance number the vendor measured, including a vendor measuring its own competitor.
- (independent) - measured by a party that sells none of these models.
- (U) - unverified or not published. Absence is reported, not filled.
Two measurement traps caught this brief during review and are worth stating up front, because the press has fallen into both. First, the Jev Decision Index publishes two different scores for every model, a public-benchmark score and a private held-out score, and they rank Clef and Jev in opposite orders. Quoting either alone misleads. Second, that index's # column is a tie-band, not a rank: its own tooltip reads "tied: within 0.9 points of the first model in its group." Jev's band label of =10 means 41st of 229, not tenth.
What we already know (from the vault)
- The vault has a deep Jev thread (eight-plus notes) but zero head-to-head coverage of Clef or Clairic. Those names appear in exactly one vault file, [[2026-10-20-technically-whats-a-decision-model]], and only as a pointer to the founder's memory note. That note explicitly flagged the gap it could not close: its source "is written before Cloudflare's Clef/Clairic announcement and doesn't mention it." This brief closes it.
- Prior vault figures for Jev were $0.042 per million input tokens with free output and 70-500ms latency ([[2026-09-23-every-jev-usage-guide]], [[2026-09-27-alphasignal-system-one-models-jev-laya-clm8b]]). Both re-verify at TypeSafe's own surfaces, and the independent data supports rather than contradicts the latency figure.
- The vault already logged that open rivals were arriving: Laya and Stanford/Nvidia's CLM-8B ([[2026-09-27-alphasignal-system-one-models-jev-laya-clm8b]]). That note called for a head-to-head benchmark before committing. Public independent data now exists, and it turns out to be the most important thing in this brief.
- The standing instruction to prefer Jev for classification and typed-set problems is [[feedback_jev_for_typed_set_problems]] (founder, 2026-10-11). The Clef/Clairic distinction is [[reference_cloudflare_clef_clairic]] (founder, 2026-10-19).
- RDCO's live integration is thin by construction.
/Users/ray/.claude/skills/ideas/jev_judge.pyis 301 lines, standard-library only, and hardcodes two constants:API_URL = "https://api.typesafe.ai/v1/systemone"andMODEL = "jev-latest". This is load-bearing below.
What the web says
- Cloudflare shipped Clef and Clef-flash on 2026-10-01 as Apache 2.0 open-weight models it calls "fully Jev-API compatible," and cut Clef-flash to $0.038/M on 2026-10-09 explicitly so it "makes it cheaper than Jev" (blog.cloudflare.com/clef-decision-models; developers.cloudflare.com changelog 2026-10-09). The Register: "its API is fully Jev compatible, allowing it to serve as a drop-in replacement."
- The compatibility claim has third-party corroboration. Arize AI called Clef through Workers AI and describes it as "Open weights, Jev-compatible API, can read images," so an outside party has exercised the interface. RDCO has not (U).
- Clairic, 2026-10-19, is a different product aimed at a different gap - regression and extraction rather than classification - and Cloudflare positions it as extending the category TypeSafe created, not contesting it (blog.cloudflare.com/clairic-extraction-model).
- The decisive independent result is a methodology split, not a verdict. On Jev Decision Index 0.4.1 (updated 2026-10-22), Clef scores 61.7 against Jev 1.14's 60.7 on public benchmarks, and 54.9 against Jev 1.14's 60.2 on the private held-out Full score. Cloudflare's leadership claim is real on the public view and reverses on the private one.
- Arize AI, testing on public benchmarks, finds parity: "Clef 27B matched Jev on accuracy in every round" (arize.com/blog/decision-model-benchmark/). That is consistent with the split rather than contradicting it.
- Every latency figure in wide circulation is Cloudflare's own. MarkTechPost correctly footnotes the much-copied 209.3 / 38.8 / 524.1ms triple as "Cloudflare's internal Decision Index run," and adds: "All numbers are vendor-reported, with no independent replication yet."
- The category commoditized in about five weeks, and not mainly by Cloudflare. OpenAI's Decisions API reached general availability on 2026-10-22 at $0.10/M input tokens only (community.openai.com). More tellingly, the Decision Index is a board of "228 open reproductions of TypeSafe Jev's 'Decision Model'," and 40 of its 229 entries outscore Jev 1.14 on private tests.
- TypeSafe neither cut price nor responded publicly. It reportedly raised instead, at a figure secondary coverage puts at $870M on a $7.5B valuation led by Andreessen Horowitz, eight days after Clef. The amount is (U): it traces to Bloomberg only through secondary blogs, one of which cites a conflicting $7.6B. The vendor's own site says only "a really big Series A led by a16z."
Head-to-head: pricing
All of these bill per million input tokens with output free, which is unusually lucky for comparison. The hazards are elsewhere.
| Jev (TypeSafe) | Clef | Clef-flash | Clef-omni | Clairic | OpenAI Decisions | |
|---|---|---|---|---|---|---|
| $ / M input tokens | $0.042 (vendor) | $0.240 (vendor) | $0.038 (vendor) | $0.150 (vendor) | $0.100 (vendor) | $0.100 (vendor) |
| Ratio to Jev (derived) | 1.00x | 5.71x | 0.90x | 3.57x | 2.38x | 2.38x |
| Output tokens | free, "too cheap to meter" | free | free | free | free, "generates no tokens" | free; no cache charges |
| Native billing unit | dollars per token | 21,818 neurons/M | 3,455 neurons/M | 13,636 neurons/M | 9,091 neurons/M | dollars per token |
| Free allowance | none published (U) | 10,000 neurons/day, including on the free Workers plan | same | same | same | (U) |
| Weights | closed, hosted API only | Apache 2.0, Hugging Face | Apache 2.0 | Apache 2.0 | not released (U) | closed |
| Self-host floor | n/a | 85 GB video memory (vendor) | 41 GB (vendor) | (U) | n/a | n/a |
| Maturity | "early access"; no general-availability language | no maturity label at all | no label | no label | explicitly "in preview" | generally available 2026-10-22 |
Sources: typesafe.ai homepage and docs.typesafe.ai/models.md; developers.cloudflare.com/workers-ai/platform/pricing/ (states last updated 2026-10-20); Cloudflare's rate of $0.011 per 1,000 Neurons; OpenAI's own community announcements of public beta (2026-10-06) and general availability (2026-10-22). The video-memory floors are Cloudflare product manager Michelle Chen paraphrased by The Register, qualified "assuming single concurrency and a 64k context window."
What had to be assumed to normalize, stated plainly:
- Cloudflare's dollar figures are its own neuron conversion. It bills in "neurons" and publishes both; I used its published dollars rather than recomputing.
- The free-allowance-to-token conversion is my arithmetic, not Cloudflare's (derived). At 10,000 neurons/day: roughly 458K Clef input tokens/day, 2.9M Clef-flash, 733K Clef-omni, 1.1M Clairic.
- Clairic bills schema tokens as input, and they dominate. Cloudflare's own worked example reads
state_tokens: 61, schema_tokens: 472, input_tokens: 655. Schema is 72% of billed input, and a further 122 tokens are neither state nor schema and go unexplained. TypeSafe publishes no equivalent breakdown (U). Per-token cost is comparable across vendors; per-decision cost is not. - Nobody sells "a million judgments" (U). No per-request or per-judgment stock-keeping unit exists at any of these vendors.
- Open weights are not free to run. Clef's 85 GB video-memory floor puts self-hosting above a single consumer or prosumer graphics card, so Apache 2.0 removes licence lock-in without removing cost.
The findings that matter more than the rates:
- Cloudflare took the price floor and named Jev when it did. The 2026-10-09 changelog: "Clef-flash now costs $0.038 per million input tokens, down from $0.090, which makes it cheaper than Jev."
- It bought that cut with context. Same changelog: "the hosted Clef-flash context window is now 24K tokens, down from 64K... only 0.24% of requests exceed 24K input tokens."
- At equal quality Cloudflare is not cheaper, it is 5.71x more expensive. The Clef model that competes with Jev on accuracy is the $0.240 one. Arize's measured spend corroborates the ratio in practice: $0.223 per 1,000 RAGTruth decisions for Clef 27B against $0.045 for Jev 1.13, about 5x (independent).
- At low volume Cloudflare is free and Jev is not. Jev publishes no free tier and no trial credits (U); access was a waitlist until 2026-09-27. Cloudflare's 10,000 neurons/day applies on the free plan, so below roughly 458K Clef input tokens/day Cloudflare's most expensive classification model costs nothing. For a workload RDCO's size the ranking inverts.
- OpenAI did not undercut, it premium-bundled at 2.4x Jev. Microsoft-Decision-1 reportedly matches Jev at $0.042/M, but that rests on a single aggregator blog's table and no primary source was reached (U).
TypeSafe's own hedge is worth keeping: its docs say output tokens are "currently free of charge," and the launch post concedes the company "can't prove it isn't subsidized."
Head-to-head: latency
Nobody has measured these models under one controlled setup, and both independent sources say so themselves. Percentile shorthand below: p50 is the median, p95 the 95th percentile.
| Source | Who | Clef | Clef-flash | Jev | Rig |
|---|---|---|---|---|---|
| Cloudflare's 43-eval suite | vendor-measured | 209.3ms median / 238.6 p95 | 38.8 / 122.4 | 524.1 / 536.0 | not stated (U) |
| Jev Decision Index 0.4.1 | independent | 102ms | 53ms | 196ms (1.14); 524ms (1.13) | local models on 1x RTX PRO 6000; Jev over HTTP |
| Arize AI | independent | 0.50s p50 | not tested | 0.08s p50 (1.13) | both hosted and called over the network |
TypeSafe's own figure, for completeness: "End-to-end response time is 70ms-500ms" (vendor) - a range, with no percentile and no average, self-measured and "generally run from our laptops on the West Coast."
Five cautions, each load-bearing:
- The two independent sources tested different Jev versions, so their numbers are not interchangeable. The Decision Index runs
jev-1.14.0; Arize ran Jev 1.13. The Index's own 1.13 figure is 524ms against 1.14's 196ms, so version choice moves the answer by a factor of nearly three. - Arize is the one source where both models were hosted endpoints called over the network, and there Jev is about 6x faster (0.08s against 0.50s p50 on RAGTruth). Arize's laptop caveat - "Local latencies are from one laptop and aren't comparable with the cloud numbers" - applies only to Kev 9B, Jeeves 9B, Strands Decider 2B and Laya. Clef 27B ran on Workers AI in what Arize calls the cloud conference. Arize does not itself claim this is uniquely apples-to-apples, so that reading is mine.
- The Decision Index runs Clef on a datacenter graphics card and Jev over the internet, and says so: "Jev's latency is a hosted HTTP round trip and is not a controlled comparison with local GPU runs." Its 102ms-versus-196ms ordering therefore measures hosting as much as modelling.
- Cloudflare's own two Clef tables do not reconcile. The 2026-10-09 post gives a pre-optimization Clef median of 262ms / 438ms p95; the 2026-10-01 launch post claimed 209.3 / 238.6 for the same unoptimized model. Cloudflare never addresses it. Neither table is the Clef latency.
- Clairic's headline speed claim is checkable and partly fails. Cloudflare's three-step insurance-claim demo reports 19.3s cascaded against 61.7s for an all-LLM pipeline, "about 3.2 times faster," "49% less" cost. The cascade arm sums correctly (0.46 + 0.22 + 18.6 = 19.28s). The baseline does not: 16.1 + 21.9 + 16.4 = 54.4s against the 61.7s stated, a 7.3s gap - 11.8% of the stated total - that Cloudflare never accounts for. On the summed baseline the speedup is 2.8x rather than 3.2x. Further: the baseline is GLM-5.3 rather than a frontier model; the 49% saving is driven by output tokens in the reply step (1,329 against 2,544), not by Clairic's input price; and 49% is cost while 3.2x is time, so the widely-copied "49% cheaper / 3.2x faster" pairing conflates two measurements.
Net read: there is no defensible claim that either model is faster than the other. Each independent source favours a different one, and in each case the rig explains the result. On the only hosted-to-hosted comparison available, Jev wins. TypeSafe's self-measured 70-500ms band is consistent with both independent figures for it (196ms and 80ms), so the vault's prior shorthand of "sub-half-second" was reasonable and needs no correction.
The accuracy result that decides the question
The question named three axes, but whether Cloudflare is commoditizing the primitive turns on a fourth, so it gets its own section. The headline finding is a split by methodology, and it is more interesting than either side's marketing.
Jev Decision Index 0.4.1 (independent), updated 2026-10-22. Run by one person (multimodalart on Hugging Face), footer reads "Not affiliated with TypeSafe AI," frozen suite of 110,201 requests across 37 benchmarks and 229 entries, local models on one RTX PRO 6000. It publishes two scores:
| Public-benchmark score | Private "Full score" | Calibration error | Latency | |
|---|---|---|---|---|
| Cloudflare clef (28B) | 61.7 (above both Jev versions) | 54.9 | 4.0 | 102ms |
| Jev 1.14 | 60.7 | 60.2 | 8.0 | 196ms |
| Jev 1.13 | 58.0 | 59.2 | 7.8 | 524ms |
| clef-omni (30B-A3B) | 56.5 | 45.0 | 4.0 | 191ms |
| clef-flash (9.7B) | 56.1 | 44.4 | 2.5 | 53ms |
The Full score is "0.60 x same-skill private + 0.40 x new domains," and the index excludes public benchmarks from it for a stated reason: "The 37 open tasks shown on this site. Anyone can see them, and train on them, so they no longer count toward the Full score."
So Cloudflare's leadership claim is not refuted, it is located. Clef genuinely leads Jev on the public benchmarks Cloudflare cited, and that lead reproduces independently. It then reverses by 5.3 points on tests nobody can train against. That pattern is the signature of benchmark contamination or overfitting to the public suite, and it is exactly what the index was restructured to expose. It is also the single most decision-relevant fact in this brief, because RDCO's workload is private data, not a public benchmark.
The split also explains the apparent disagreement among independent sources without needing a second theory:
- Arize AI (independent) tested on public benchmarks - RAGTruth, the Phoenix evaluator suites, and tool routing - and found parity, as the split predicts. Verbatim: "Clef 27B matched Jev on accuracy in every round, and along with Jev and OpenAI it was one of only three models to survive the wording twist." Per-challenge against Jev 1.13: RAGTruth 85.2% against 85.5%, Phoenix 91.5% against 91.1%, routing 50 of 52 against 49 of 52. Clef wins two of three by hair's breadths.
- Calibration, which is the whole point of this model class, is genuinely contested. The Decision Index has Clef materially better calibrated than Jev (4.0 against 8.0 expected calibration error, lower better). Arize has the reverse on its scale (Clef 0.062 against Jev 0.023). Different scales, different Jev versions, opposite conclusions. Neither should be quoted as settled (U).
- Robustness is a point for both and against the field. On Arize's reworded-prompt twist, Jev 1.13 held at 0.93 / 0.93 ROC AUC and Clef at 0.92 / 0.91, while Liquid d1 collapsed from 0.92 to 0.17 and several open models fell further.
Two things the brief should not overclaim:
- The cheap Clef is the weak Clef. Clef-flash, the $0.038 model that undercuts Jev, scores 44.4 on private tests against Jev's 60.2, and on Cloudflare's own CLINC150+OOS table it posts 66.77 macro-F1 against full Clef's 97.43 (vendor-measured). In fairness to Cloudflare, that same table has full Clef beating Jev at 97.43 against 89.27, which is one of the public-benchmark wins the private tests do not sustain.
- Cloudflare's own model card concedes Jev's general-reasoning lead: Jev ahead on GPQA Diamond 78.3 against 48.0, MMLU-Pro 82.7 against 65.9, BBH 92.9 against 73.7 (vendor-measured, reported by MarkTechPost from Cloudflare's card rather than by The Register).
Clairic has no independent evaluation of any kind (U). It is on neither leaderboard, reasonably, since both score decision models. Every Clairic performance number in circulation is Cloudflare's, and secondary coverage repeats them, with PPC Land's own headline conceding the saving was measured "in its own test." Cloudflare's one checkable mini-eval is small and candid (vendor-measured): 21 days of public Transportation Security Administration checkpoint counts, 3.3% mean error, 17 of 21 actuals inside the 80% interval, with Cloudflare noting three of the four misses fell on Columbus Day weekend because the model "doesn't account for events the series doesn't show." That is n=21 on one series.
Head-to-head: positioning
| Jev | Clef | Clairic | |
|---|---|---|---|
| What the primitive is for (vendor) | "unstructured state in, typed probabilistic decisions out"; a "frontier-intelligence function call" | "reads an input state and a set of typed questions, then returns a probability for every allowed answer... no free-form output to parse and no reasoning tokens to wait for" | "extract, compute, estimate, and forecast values from unstructured inputs" |
| Output shape | Choice / Noul / Score; Noul returns probability-of-yes with no separate confidence | probability per allowed answer; 1-64 questions per request | typed spans, numbers, dates, durations, lists; p_null plus confidence; estimates return median and an 80% interval |
| Buyer | "developers"; no stated ideal customer profile (U) | the agent builder; the pitch is removing human-in-the-loop | developers already making structured-output calls to frontier LLMs |
| Place in stack | hosted API callable from anywhere; no edge, on-premises or private-cloud offer (U) | inside the Worker, on the request path; Cloudflare's own header is "Built for the hot path" | same; env.AI.run(), REST, or AI Gateway |
| Modality | text; images gated behind a TypeSafe-Danger-Zone-Images header on 1.14 only |
vision encoder inherited from the Qwen backbone; clef-omni adds audio and video | text plus up to 4 images |
| Context | 64k per request / 32k per question | 65,536 (Clef); 24,576 (flash) | 65,536 |
Cloudflare's framing is frontal assault on one axis and category extension on the other, and it is explicit about both.
Cloudflare credits TypeSafe with creating the category, by name and repeatedly. The Clef post opens: "Jev introduces a new decision model concept into the world of AI." The Clairic post: "When System One models were announced by TypeSafe a month ago, it took the industry by storm... We were inspired by the concept of System One models." Cloudflare's own landscape table lists the Decide column as "Jev, Clef," placing itself inside TypeSafe's box rather than claiming a new one.
Then it does three things an ordinary fast-follower does not:
- It made the swap cheap. "These models are smarter, faster, and fully Jev-API compatible," and "Clef follows the System One API, so you can switch an existing Jev integration to Clef by changing the endpoint and model." Cloudflare adopted TypeSafe's interface and built the migration path on TypeSafe's behalf. In practice it is three changes rather than two, since a Cloudflare account and token are also needed, and Arize's successful hosted calls are third-party evidence the interface works.
- It benchmarks itself on TypeSafe's own scoreboard, claiming to beat Jev "in 3 of 4 areas" on "Typesafe's own workflow evals." As above, that holds on public tasks and not on private ones.
- It gave the weights away under Apache 2.0, against a closed hosted-only incumbent. With one asterisk: Cloudflare's product manager confirmed to The Register that the training data is not public, so "open source" here means open weights rather than an open model.
Clairic is not aimed at Jev. Cloudflare's carve is explicit: "Clairic handles the first [a number]... Clef handles the second [a choice from a fixed set]... LLMs handle the third." And: "Until now, every System One model has been a decision model: Jev and Clef... Clairic is the first model in a new class." It is a land-grab in territory Jev does not occupy, and the two chain: "Extraction models turn raw input into typed values, decision models turn values into decisions, and LLMs turn decisions into words." Clairic's weights are not released, so the commoditization logic that applies to Clef does not extend to it today.
TypeSafe has not responded publicly and has not cut price. Zero mentions of Cloudflare, Clef, or Clairic across its homepage, launch post, 1.14 post, enterprise page, and docs. Jev remains $0.042.
The category filled in about five weeks, which is the real context. Jev launched 2026-09-15; OpenAI announced Decisions at DevDay around 2026-09-29 (secondary-sourced, U) and reached general availability 2026-10-22; Amazon's Strands Decider 2B and Cloudflare's Clef both shipped 2026-10-01; Liquid AI's d1-ultra and Cloudflare's Clairic both landed 2026-10-19 (dates for Strands, d1-ultra and the DevDay announcement were not traced to primary sources this pass, U). Perplexity, Microsoft, Fastino and others sit on the boards.
Convergences and contradictions
- Convergence on the category, and it is not in dispute. Both vendors use TypeSafe's "System One" vocabulary, cite Kahneman, return calibrated typed outputs rather than text, and make the same caveat: typed output guarantees the interface, not the truth. TypeSafe's own skill file says "Typed output guarantees the interface, not truth"; Cloudflare's Clairic post concedes the model "can, of course, make mistakes." This matches the calibration skepticism already logged in [[2026-10-20-technically-whats-a-decision-model]].
- The apparent accuracy contradiction resolves into one mechanism. Cloudflare's public-benchmark lead over Jev reproduces independently; it reverses on private held-out tests; Arize's public-benchmark parity sits between them. Public-suite contamination explains all three observations at once, and the Decision Index says that is precisely why it stopped counting public tasks.
- The market contradicts the commoditization narrative, and so does the board. Eight days after Clef shipped, TypeSafe reportedly raised at a materially higher valuation (U on the amount). Meanwhile the deeper threat to TypeSafe is not Cloudflare but the long tail: the Decision Index is a board of "228 open reproductions" of Jev's own model, and 40 entries outscore Jev 1.14 on private tests, with Fastino GLiDE 1.2 leading at 71.5 and 93ms.
Synthesis for RDCO
Cloudflare is commoditizing the primitive, but it has commoditized the licence and the interface rather than the price or the quality, and the distinction decides what RDCO should do. The licence is unambiguous: Apache 2.0 weights on Hugging Face against a closed hosted-only incumbent. The interface is close behind, by Cloudflare's claim and with third-party corroboration that an outsider can call Clef through a Jev-shaped API. Price is where the headline breaks down. The Clef that competes with Jev on accuracy costs 5.71x Jev at list, and Arize measured roughly that ratio in practice at $0.223 against $0.045 per thousand decisions. The model that does undercut Jev, Clef-flash at $0.038, scores 44.4 against Jev's 60.2 on held-out tests. That is a cheaper, weaker product rather than a commoditized price, and the only regime where Cloudflare is genuinely cheaper at equal quality is inside its free daily allowance.
Quality is where the research actually paid off, and the finding is not the one the press is carrying. Cloudflare claimed to lead TypeSafe's own leaderboard. That claim reproduces on the index's public-benchmark view, where Clef posts 61.7 against Jev's 60.7, and it reverses on the private held-out Full score, where Clef trails 54.9 to 60.2. The index excludes public tasks from its headline for a stated reason: anyone can see them and train on them. So the honest reading is that Clef's advantage lives on the benchmarks it could have been tuned against and disappears on the ones it could not. Arize's independent parity result, run on public benchmarks, fits that pattern rather than contradicting it. For a buyer whose workload is private data, the private column is the one that matters.
Cloudflare's strategic intent is plainly to make typed judgment a cheap platform feature, though the margin story is my inference rather than its statement. Its own words are narrow: "The interest in Jev shows the need for a fast, small, specific, classifier model, and we chose this to be our niche to start experimenting with RL environments." The reading that the money is meant to land in Workers, AI Gateway, and the reinforcement-learning fine-tuning service is inference, and it stays unconfirmed until that service has a published price (U). What is observable is that TypeSafe is structurally exposed if the inference is right: its enterprise page offers no platform, no fine-tuning, no eval tooling, no dedicated capacity, no on-premises option and no service-level agreement. The product is one model behind one endpoint. The larger threat, though, is not Cloudflare at all. It is that the Decision Index is a board of 228 open reproductions of Jev, 40 of which beat it, with an unfamiliar name at the top.
For RDCO the pricing axis is irrelevant and the dependency axis is the real one. RDCO's Jev volume is a handful of judgments per /ideas run, which sits inside Cloudflare's free daily allowance even on its most expensive classification model, so in absolute terms either choice is a rounding error and cost is not a reason to move. The reasons worth weighing are that Clef's weights are Apache 2.0, which removes single-vendor risk from a primitive the founder made a standing default; that Cloudflare is already RDCO's platform, so Clef is reachable from Workers RDCO already runs and through the AI Gateway rail already configured for Scribble Works; and that Jev is formally "early access" with console-gated rate limits and no service-level agreement. Against all of that sits the finding above: on held-out private tests, which is the regime RDCO operates in, Jev is the better model by five points.
The practical conclusion is that RDCO does not have to choose, and should not pre-commit. Because Cloudflare adopted TypeSafe's interface, jev_judge.py needs two constants changed plus a Cloudflare token, against a 301-line standard-library-only script. That is a flag, not a migration, and it is the test [[2026-09-27-alphasignal-system-one-models-jev-laya-clm8b]] called for and could not run. Until RDCO measures on its own labelled sample, Jev stays the default under [[feedback_jev_for_typed_set_problems]]: it is wired, its price is immaterial at this volume, and it leads on the only independent evidence that is contamination-resistant. Two side findings are worth carrying. First, for the harness work in [[2026-10-22-automode-classifier-channel-text-injection-surface]], Cloudflare argues that feeding a decision model typed values instead of raw text shrinks the prompt-injection surface, because "Clef decides from a five-line summary of typed values instead of a free-form email, which can also be susceptible to prompt injection." That reasoning applies to RDCO's own classifier-gate design whoever wins. Second, the independent accuracy leader is Fastino GLiDE 1.2 at 71.5 and 93ms, eleven points above Jev, and RDCO has never looked at it.
Why this is in the vault
[[2026-10-20-technically-whats-a-decision-model]] explicitly flagged that its source predated Clef and Clairic and could not assess them, and [[feedback_jev_for_typed_set_problems]] makes Jev a standing default for every RDCO skill with a fixed-set decision. This brief supplies the vendor-dependency check that default was missing, and gives /ideas and any future Jev-wired skill a costed decision: parameterize the provider, keep Jev as the default on the strength of the private-test result, and read any Clef-versus-Jev benchmark claim by asking first whether it was scored on public or held-out data.
Test plan (build task, not research)
Pre-registered so the result cannot be retrofitted:
- Add a
--provider {jev,clef,clef-flash,glide}flag tojev_judge.py, changing onlyAPI_URL,MODELand the credential lookup. Confirm Cloudflare's interface-compatibility claim survives contact with RDCO's actual payload, which usesstateplus aquestionsmap. That claim is currently vendor-stated and untested here. - Build the labelled sample first: idea-type assignments and near-duplicate calls from the existing idea bank that the founder or Ray has already judged, with judgments withheld from every model. This sample is private by construction, which is the point.
- Score agreement against the held-out labels, plus wall-clock latency measured from RDCO's own Mac mini, since no vendor pair has been measured on one rig and the two independent sources disagree.
- Report calibration, not just accuracy: whether high-confidence answers are in fact more often right. The two independent sources contradict each other on calibration, so this is the measurement with the most to gain.
- Acceptance: Clef replaces Jev only if it wins on agreement at equal or better measured latency on RDCO's own sample. No published benchmark counts toward this, including the Decision Index.
Open follow-ups
- What does Cloudflare charge for the reinforcement-learning fine-tuning platform, and when does it go self-serve? No rate card, no date (U). That price is the only thing that confirms or refutes the commoditize-the-complement reading, which this brief leaves as inference.
- Who is Fastino, and does GLiDE 1.2 hold up? It leads the independent Decision Index at 71.5 and 93ms, eleven points above Jev, and the vault has nothing on it. If it is real it matters more to RDCO than the Cloudflare question does.
- Is the public-versus-private gap specific to Clef, or general across the 228 reproductions? If most entries lose ground on held-out tests, the finding is about leaderboard hygiene across the category rather than about Cloudflare, and it changes how RDCO should read every decision-model benchmark.
- Did TypeSafe really raise $870M at $7.5B, and what is it buying? Bloomberg-sourced but reached only via secondary blogs with a conflicting $7.6B figure (U). Whether the proceeds fund a platform or just more models decides whether commoditization is survivable for them.
Related
- [[2026-10-20-technically-whats-a-decision-model]] - the vault's conceptual primer; explicitly predates and omits Clef and Clairic, which this brief supplies
- [[2026-09-27-alphasignal-system-one-models-jev-laya-clm8b]] - logged Laya and CLM-8B as open rivals and called for a head-to-head benchmark; this brief answers it from public data and the test plan finally runs it on RDCO's own sample
- [[2026-09-23-every-jev-usage-guide]] - source of the vault's $0.042 figure, re-verified here; its validation recipe is what the test plan re-runs against Clef
- [[2026-09-15-every-typesafe-jev-vibe-check]] - opened the Jev due-diligence thread and the cheap-pre-filter idea a provider flag would let us finally test
- [[2026-10-19-indy-dev-dan-jev-news-curator-agent]] - source of the "sub-half-second" shorthand, which the independent figures support
- [[2026-09-28-indy-dev-dan-10-levels-of-jev]] - the Jev-guard hook pattern, which a provider flag would make vendor-neutral
- [[2026-09-18-security-review-typesafe-skills-plugin]] - the installed TypeSafe skill; its own text defers all numbers to live docs, so it corroborates nothing in the pricing table
- [[2026-10-09-analytics-engineering-roundup-personal-agents-jev-decision-models]] - prior touchpoint using the same "decision models" term
- [[2026-10-22-automode-classifier-channel-text-injection-surface]] - where Cloudflare's typed-values-shrink-the-injection-surface argument applies to RDCO's own gate design
- [[feedback_jev_for_typed_set_problems]] - the standing default this brief stress-tests and leaves in place
- [[reference_cloudflare_clef_clairic]] - the founder's Clef-versus-Clairic correction that framed the question
Sources
TypeSafe (vendor-primary)
- https://typesafe.ai/ - homepage pricing chart ($0.042 / free; "$42 per billion input tokens"); "193.6x Faster, 444.6x Cheaper"; the disclaimer that its zero-hallucination figure "is not an observed error rate from testing"
- https://typesafe.ai/blog/introducing-system-one-models-and-jev - launch post 2026-09-15; "End-to-end response time is 70ms-500ms"; laptop methodology; early access; Reinforcement Learning for Calibrated Decisions (RLCD)
- https://docs.typesafe.ai/models.md -
jev-1.14.0/jev-1.13.0; 64k per request / 32k per question; $42 / $0.042 - https://docs.typesafe.ai/api/system-one.md - "Output tokens are currently free of charge"
- https://docs.typesafe.ai/primitives/noul.md , https://docs.typesafe.ai/concepts/system-one.md
- https://typesafe.ai/enterprise - no platform, fine-tuning, service-level agreement or on-premises offer
- https://typesafe.ai/pricing - HTTP 404; there is no pricing page (verified independently this pass)
- https://typesafe.ai/blog/series-ai - "a really big Series A led by a16z", 2026-10-09
Cloudflare (vendor-primary)
- https://blog.cloudflare.com/clef-decision-models - Clef launch 2026-10-01; "fully Jev-API compatible"; the median/p95 table; Apache 2.0; the "we chose this to be our niche" line; forward-deployed-engineer-first fine-tuning with no published price
- https://developers.cloudflare.com/changelog/post/2026-10-01-clef-workers-ai/ - "Clef is 2.5x faster than Jev at the median, and Clef-flash 13x"; "Built for the hot path"
- https://developers.cloudflare.com/changelog/post/2026-10-09-clef-omni-workers-ai/ - Clef-flash cut to $0.038 "cheaper than Jev"; context cut from 64K to 24K; the 0.24% figure; the Clef re-benchmark at 262/438ms
- https://blog.cloudflare.com/clairic-extraction-model/ - Clairic launch 2026-10-19; the three-way carve; the chaining framing; the 19.3s against 61.7s cascade table; the TSA forecast mini-eval
- https://developers.cloudflare.com/changelog/post/2026-10-19-clairic-workers-ai/ - "Clairic costs $0.10 per million input tokens"
- https://developers.cloudflare.com/workers-ai/platform/pricing/ - the price table; $0.011 per 1,000 neurons; the 10,000 neurons/day allowance
- https://developers.cloudflare.com/workers-ai/models/clef/ , /clef-flash/ , /clef-omni/ , /clairic/ - context windows, preview labels, the
p_nullresponse example - https://huggingface.co/Cloudflare/clef , /clef-flash , /clef-omni - Apache 2.0 weights
Independent measurement
- https://huggingface.co/spaces/multimodalart/jev-decision-index (static view at https://multimodalart-jev-decision-index.static.hf.space/index.html) - Decision Index 0.4.1, updated 2026-10-22; "Not affiliated with TypeSafe AI"; "228 open reproductions"; 110,201 requests, 37 benchmarks, 229 entries, 1x RTX PRO 6000; the FULL SCORE / PUBLIC ONLY toggle; Full score = "0.60 x same-skill private + 0.40 x new domains"; "Anyone can see them, and train on them, so they no longer count toward the Full score"; the tie-band tooltip "tied: within 0.9 points of the first model in its group"; "Jev's latency is a hosted HTTP round trip and is not a controlled comparison with local GPU runs"
- https://arize.com/blog/decision-model-benchmark/ - Jim Bennett, Arize AI, October 2026; the "Where we ran it" column placing Clef 27B on Workers AI and only Kev 9B, Jeeves 9B, Strands Decider 2B and Laya on the MacBook Pro; Jev 1.13 as the version tested; p50 RAGTruth latencies; cost per 1,000; "Clef 27B matched Jev on accuracy in every round"; "Local latencies are from one laptop and aren't comparable with the cloud numbers"
- https://artificialanalysis.ai/evaluations/artificial-analysis-decision-index - "All evaluations are conducted independently by Artificial Analysis"; places Jev above Clef, but exact values are locked in a JavaScript chart and were not read off a table, so the figures are (U) and only the ordering is cited
Press and third-party
- https://www.theregister.com/ai-and-ml/2026/10/01/cloudflare-tries-to-outplay-jev-with-open-weight-clef-models/5300649 - Brandon Vigliarolo, 2026-10-01; "its API is fully Jev compatible, allowing it to serve as a drop-in replacement"; "Clef costs $0.24 per million tokens - nearly six times the price of Jev at $0.042/M"; Michelle Chen on undisclosed training data and the video-memory floors, qualified "assuming single concurrency and a 64k context window"
- https://www.marktechpost.com/2026/10/01/cloudflare-releases-clef-and-clef-flash/ - "Clef uses the same System One API. Switching from Jev means changing the endpoint and model name."; Apache 2.0; the retained Qwen vision encoder; the GPQA Diamond / MMLU-Pro / BBH model-card figures; and "All numbers are vendor-reported, with no independent replication yet"
- https://systemonemodels.org/integrations/cloudflare/ and /compare/jev-vs-clef/ - independent aggregator; corroborates Jev at $0.042 and the Clef family rates; notes a Jev client also needs Cloudflare credentials
- https://community.openai.com/t/decisions-api-is-now-available-in-public-beta/1403877 and /t/decisions-api-is-now-generally-available-with-new-updates/1408934 - OpenAI's own announcements; public beta 2026-10-06, general availability 2026-10-22; $0.10 per 1M input tokens, input only; Predicates / Choices / Scores on
gpt-6-luna
Claims carried without a primary source, flagged (U) in text The TypeSafe funding amount and valuation; the Microsoft-Decision-1 price; the OpenAI DevDay announcement date; the Amazon Strands Decider 2B and Liquid d1-ultra launch dates; the "commoditise in 16 days" framing.
Vault paths cited
- /Users/ray/rdco-vault/06-reference/2026-10-20-technically-whats-a-decision-model.md
- /Users/ray/rdco-vault/06-reference/2026-09-27-alphasignal-system-one-models-jev-laya-clm8b.md
- /Users/ray/rdco-vault/06-reference/2026-09-23-every-jev-usage-guide.md
- /Users/ray/rdco-vault/06-reference/2026-09-15-every-typesafe-jev-vibe-check.md
- /Users/ray/rdco-vault/06-reference/2026-10-19-indy-dev-dan-jev-news-curator-agent.md
- /Users/ray/rdco-vault/06-reference/2026-09-28-indy-dev-dan-10-levels-of-jev.md
- /Users/ray/rdco-vault/06-reference/research/2026-10-22-automode-classifier-channel-text-injection-surface.md
- /Users/ray/rdco-vault/02-sops/2026-09-18-security-review-typesafe-skills-plugin.md
- /Users/ray/.claude/skills/ideas/jev_judge.py - RDCO's live Jev integration; the two constants that make this a flag rather than a migration