"🧠 Inside the pragmatic AI engineer's new secret weapon" — AlphaSignal

Why this is in the vault

Ben Dickson's deep-dive on Thinking Machines Lab's Inkling model crystallizes the "pragmatic engineering" argument for routing most enterprise AI tasks to cost-efficient open-weights models — a framing directly applicable to both RDCO's own agent stack spend and to Ray's phData client discovery conversations.

The core argument

Thinking Machines Lab launched Inkling, a 975-billion parameter sparse Mixture-of-Experts model that activates only 41 billion parameters per token during inference. It supports a 1-million token context window, ships under Apache 2.0, and was available Day-0 across OpenRouter, Together AI, and Fireworks AI.

Architecture highlights:

• Interleaved local sliding-window + global attention to cut memory cost on long-context tasks
• "Thinking Effort" dial lets developers tune reasoning budget per task — crank it up for complex coding, drop it down for high-volume throughput work
• Multimodal (text, image, audio) via a joint token stream; extracts text from images/audio before reasoning, which aids debugging
• Debuted at rank 41 on the Artificial Analysis Intelligence Index — top U.S. open-weights model, behind Chinese models GLM-5.2 and DeepSeek-V4 Pro
• Scored Elo 1238 on GDPval-AA v2 (economically valuable knowledge-work benchmark), ahead of Kimi K2.6 (1190) and DeepSeek V4 Flash (1189)
• Averages only 25,000 output tokens per task vs 37,000–43,000 for Chinese rivals — lower token consumption compounds into real cost savings

Pricing: ~$1.00/M input, $4.05/M output — well below Western closed frontier models ($5–15/M). Slightly more expensive than DeepSeek V4 Pro on raw rates, but lower token consumption likely closes the gap.

Quantization path for self-hosting: NVFP4 format drops VRAM from 2 TB to ~600 GB on Unsloth or llama.cpp. Inkling-Small (276B total, 12B active) is teased but weights not yet released.

Tinker integration: Thinking Machines' managed fine-tuning platform abstracts GPU orchestration and hyperparameter tuning. Enterprises upload proprietary data and run custom eval loops without an ML engineering team — the main enterprise adoption unlock.

"Interaction Models" framework: Thinking Machines trained Inkling across randomized tool sets and schema formats during post-training, preventing overfitting to any one API shape. The goal is a model that plugs into diverse agent harnesses without per-harness adaptation work.

Thesis distillation: Pragmatic engineering means building pipelines that blend model tiers. Frontier models handle edge-case reasoning and long-horizon planning; open-weights handle the 90% of high-volume, domain-specific tasks where cost and latency matter more than ceiling performance.

Mapping against Ray Data Co

The most concrete connection is Ray's phData DSA role: the "pragmatic engineering" cost narrative — frontier for edge cases, open-weights for volume — is exactly the framing a Deal Solutions Architect brings to discovery conversations with enterprise clients currently overspending on closed API tokens. This gives Ray a named model (Inkling), a concrete pricing anchor ($1/M vs $5–15/M), and an enterprise-friendly license (Apache 2.0) to make the argument specific rather than generic.

Second: RDCO's own always-on COO agent currently routes everything through Claude (closed API). The GDPval-AA v2 benchmark is designed for "economically valuable knowledge-work tasks" — the same task profile as an AI COO doing triage, vault writes, and async ops. Inkling's 1238 Elo on that benchmark is a data point worth revisiting when evaluating whether to route lower-stakes RDCO agent tasks to a cheaper open-weights backend.

Related

• [[2026-04-03-oss-future-of-ai]]
• [[2026-06-11-data-engineering-central-agentic-token-tax-opencode-ollama]]