Why this is in the vault
AlphaSignal's "Sunday Deep Dive" argues a new class of Tabular Foundation Models (TFMs — TabFM, KumoRFM, TabPFN, TabICL) brings zero-shot, in-context prediction to rows-and-columns data the way LLMs did for text, cutting into the traditional XGBoost/feature-engineering pipeline; worth keeping as a marker for when this actually displaces classic ML in production.
Mapping against Ray Data Co
Directly touches the CAF/phData day job: the pain TFMs target — every schema change forcing a fresh ETL pipeline, manual feature engineering, and XGBoost retuning — is the exact slog underlying dbt modeling and client feature work described in [[book-solve-everything-ch6-the-engine-2026-04-13]]. The piece is honest that TFMs currently lose on latency/cost for high-frequency scoring and only pay off when schemas or tasks are unstable — useful calibration against hype, and a concrete "when to reach for this" test to apply if a CAF client ever proposes it for a live pipeline instead of prototyping. Secondarily, Nvidia's June acquisition of Kumo (behind KumoRFM) folded into its Structured Data and Graph Models stack is a data point for the platform-layer-diversification thread in the Nvidia capital-cycle read, not just fab capex.
The core argument
Standard LLM tokenizers destroy tabular structure: tables are two-dimensional and permutation-invariant, but sequential tokenizers shred numeric values into arbitrary tokens and can't preserve column-wise statistical regularities. TFMs instead treat tabular prediction as an in-context-learning task — the dataset itself is the "prompt," and the model does a zero-shot forward pass to predict new rows without per-dataset training. Four models lead: Google's TabFM (open-source, row/column attention, being wired into BigQuery), Nvidia's KumoRFM (via the June Kumo acquisition — models relational data as graphs across multiple tables), Prior Labs' TabPFN (Nature-published, cloud API or local package, license requires attribution), and Inria's TabICL (fastest at scale, handles up to 500K rows/500 features). Tradeoff: TFMs win on iteration speed and cold-start problems but lose on inference latency/cost versus a trained XGBoost model, so the pragmatic pattern is TFMs for prototyping + unstable schemas, classic ML once the problem space and schema are locked for production-scale serving.
Related
- [[2026-06-15-data-engineering-weekly-ai-agents-data-foundations]]
- [[2026-05-26-stratechery-nvidia-earnings-ai-stack-new-reporting]]
- [[book-solve-everything-ch6-the-engine-2026-04-13]]