What Will CFOs Do About Token Costs? — Mostly Metrics

CJ's framework for how CFOs should account for and manage LLM token spend as it crosses from a rounding error into a P&L-shaping cost line, drawing on Brex platform-wide AI spend data.

Why this is in the vault

A structured CFO-side framework for splitting, measuring, and gating token spend — directly extends the "token budget as employee cost" and "LLM costs on the P&L" threads already tracked from this sender, and sharpens the human-capital/token-capital framing RDCO already uses internally.

The core argument

Separate product tokens from internal tokens. Token costs embedded in your product belong in COGS; tokens employees burn internally belong in opex. Lumping both into one "AI spend" bucket the way early-stage companies treat rent will blow up the P&L and wreck resource-allocation decisions about what to sell vs. what to build.

COGS: investors want the trend, not the level. Pristine 90% SaaS gross margins now read as "no AI story"; too-thin margins read as "selling a dollar for thirty cents." CJ analogizes to Snowflake's S-1-era margin climb (~50% toward ~70% over 12 quarters) — the improving trajectory signals operating leverage and a Moore's-law cost tailwind, not the absolute number today.

Opex: AI's biggest gains sit in the middle of the distribution, not the top or bottom performers — echoing a sales-quota analogy (top reps don't need coaching, bottom reps won't last; the middle third is where AI-assisted coaching moves the needle). CJ reports most CFOs aren't cutting headcount over AI, just not backfilling attrition.

Three ROI measures, in ascending rigor: (1) revenue per employee — a blunt horizontal metric ($450K median public tech company, ~$1M at the top) with nowhere to hide; (2) functional/departmental metrics, which require bespoke, system-embedded work (tying a sales rep's token usage to actual renewal-size lift means wiring through CRM + ERP + performance systems — "AI 101 to 301 to 501"); (3) free cash flow, the final aggregator — if every company captures the same AI productivity gain, the gains get competed away and it becomes "a race to not lose," not a race to win.

Brex's cross-platform data (cited, not CJ's own): foundation-model/LLM API spend is >80% of all AI spend and grew 5x YoY; 50-70% of companies in most industries already pay for some AI (the gap is spend depth, not adoption); intensity spread across sectors is ~100x, with software services running at 2.29x the Brex median.

"Building a gate": every other major cost line (headcount, software) has an approval checkpoint; token spend still doesn't — no seat counts, no calendar renewal events, nothing hitting POs yet. CJ's prescription: tag cost to the consuming team, then drive to true per-unit numbers (cost per ticket resolved, per document reviewed, per deal cycle) rather than cost-per-employee-per-month, which is directionless without knowing what the tokens were spent on. The company that beats the market's cost-curve decline on those per-unit numbers wins the "right to live and fight another day."

Mapping against Ray Data Co

Directly upgrades how Ray should think about and eventually report token spend inside RDCO: CJ's COGS/opex split maps cleanly onto RDCO's own token usage — Ray's production work (newsletter drafts, vault notes, client-facing CAF deliverables) is COGS-like (product-embedded), while research crons, curiosity loops, and self-review are opex-like (internal tooling cost) — and RDCO doesn't currently separate these in any tracked way. The "gate" concept is the sharper takeaway: RDCO has zero checkpoint on token spend today (Max subscription means no per-call cost visibility per the API-cost-budget-controlled memory), which is exactly the "no seat count, no PO, no renewal event" gap CJ describes — worth flagging to the founder as a future instrumentation need once/if RDCO moves off flat-subscription pricing. The human-capital/token-capital "same plane, optimized frontier" framing also directly echoes the building-an-exo skill's Ch8 data-plane and Nadella "human capital directs token capital" language already in RDCO's model — convergent evidence, not a new idea, but a useful external corroboration point.

⚠️ Sponsorship

Brex — top-of-email image banner plus a full mid-email ad block ("Agentic Finance that automates the receipts... 35,000+ companies, including Mostly Metrics, Anthropic, DoorDash, and Coinbase"). Bias-relevant here beyond the usual placement: the article's evidentiary core — the 80%+ LLM-spend share, the 5x YoY growth, the 50-70% adoption rate, and the 100x sector intensity spread — is all sourced from "the Data team at Brex," i.e., the sponsor's own platform data, not an independent dataset. Treat the specific percentages as Brex-book-of-business statistics (skewed toward Brex's existing customer mix), not a market-representative sample, even though the qualitative argument (gate the cost, split COGS/opex, drive to per-unit) stands independent of the numbers.

Also present: Mostly Talent (CJ's own recruiting arm) in the footer CTA — self-promo, not a third-party sponsor, consistent with the known pattern already flagged in the sender README.

Related

• [[2026-05-10-mostlymetrics-token-budget-as-employee-cost]] — earlier Mostly Metrics piece treating token spend as a headcount-like cost, the direct precursor to this issue's COGS/opex split
• [[2026-07-09-mostly-metrics-llm-costs-pl]] — prior mailbag treatment of where LLM costs land on the P&L, same throughline this issue formalizes into a framework