Where Lakehouse//RT (Reyden) Falls Short, and How to Sequence a Serving-Tier Migration

The question

Which data serving workloads are poor candidates for Databricks Lakehouse//RT (Reyden engine) even at claimed 10ms/12,000 QPS benchmarks, and what is the migration decision tree for teams currently running ClickHouse, Pinot, or Druid alongside Delta Lake? Context: the 2026-06-19 Data Engineering Central Summit review flagged Lakehouse//RT as the most significant attempt yet to collapse the dedicated real-time serving tier into the lakehouse; RDCO designs client pipelines on Databricks and needs to know where to NOT recommend it before advising adoption.

What we already know (from the vault)

• The 2026 Summit review names Reyden as a real-time compute engine serving millisecond-latency queries directly against Delta Lake, explicitly aiming to eliminate the dedicated serving tier (ClickHouse, Pinot, Druid, Redis). Architecture collapse: Delta Lake -> Lakehouse//RT -> Apps + Dashboards + AI Agents, with Unity Catalog governance defined once instead of replicated per serving system. Claimed: 16x faster than existing real-time layers, 10ms latency, 12,000 QPS. [[2026-06-19-data-engineering-central-databricks-summit-2026]]
• Lakehouse//RT is one leg of the broader LTAP (Lake Transactional/Analytical Processing) story: Databricks is unifying the storage layer (one governed Delta/Iceberg copy) while keeping engines specialized: Lakebase for OLTP, Lakehouse for OLAP, Lakehouse//RT for serving. They are explicitly NOT building one engine to do everything. [[2026-06-19-data-engineering-central-databricks-summit-2026]]
• The vault already flags the friction point: the summit reviewer reserved judgment on whether the benchmark numbers survive "messy enterprise workloads" and whether Lakebase handles production OLTP at scale. That caution is the seed of this brief.
• The serving-tier landscape RDCO has been tracking is heterogeneous and already federation-fronted: Eric Sun's Query Proxy fronts StarRocks, VeloDB, RisingWave, ClickHouse, Snowflake, Databricks, and Iceberg/Delta behind one async gRPC API with hot-Postgres + warm-Iceberg federation across a time cutoff. The "hot tier vs warm lake" split is an architectural pattern teams already run on purpose, not an accident to be consolidated away. [[2026-05-18-data-engineering-weekly-issue-270]]

What the web says

• Lakehouse//RT is real and verifiable (it is not RDCO-internal shorthand). Databricks announced it June 16 2026 at the Summit, "now available in Beta for select read-only workloads." The Reyden engine uses a fully asynchronous execution model claiming "sub-100 millisecond latency at 12,000 queries per second" and "response times as low as 10ms on smaller datasets and sub-100ms performance on larger ones." (databricks.com blog)
• Crucial benchmark nuance: the 10ms figure is small-dataset only; large datasets are sub-100ms. The 16x figure compares Reyden against "real-time serving layers" (i.e. ClickHouse/Pinot/Druid class), not against the standard Lakehouse SQL engine — vs SQL Serverless the cited gain is far smaller (one customer, Equilibrium Energy, 3.6x median). (databricks.com blog)
• Stated design target is narrow: "operational analytics, BI and app serving, and observability workloads" — all read workloads. The blog states no claim of sub-10ms single-key point lookups and no write/OLTP claim; Lakebase ("Postgres for data apps and AI agents") is the separate transactional leg. (databricks.com blog) heise frames the whole pitch as making separate real-time databases redundant — but only for the read-serving slice. (heise.de)
• The structural reason serving directly off table formats is hard: "the primary bottleneck in querying Iceberg lies in network I/O latency — fetching Parquet files from remote object stores like S3." Low-latency high-concurrency serving on Iceberg/Delta is "non-trivial" and the market answer is an indexing/caching layer, not raw table-format scans — e.g. StarTree serves 500+ QPS on Iceberg by adding Pinot's indexing on top, not by querying Iceberg natively. (onehouse.ai, startree.ai)
• Dedicated-engine differentiators that do not disappear: ClickHouse for complex ad-hoc SQL/joins and lightning bulk-scan but weak native streaming ingest; Druid for streaming ingest + high-concurrency time-series; Pinot for ultra-low-latency user-facing analytics (sub-100ms at high concurrency). Druid/Pinot carry heavy operational overhead (5-6 node types + ZooKeeper); ClickHouse is the single-binary simplicity option. (tinybird.co, ksolves.com, clickhouse.com)
• The "forced compromise" framing the vendors all use — "accept latency or fragment the stack" — is the marketing core. The honest read: a separate serving layer brings copy-staleness, lock-in, cost, and fragmented governance; consolidation removes those but inherits object-store latency physics unless an in-engine cache/index masks it. (onehouse.ai)

Convergences and contradictions

• Convergence: Vault and web agree on the architecture (collapse the read-serving tier, keep OLTP separate in Lakebase, governance via Unity Catalog) and on the benchmark headline. Both also independently surface the same caution — the numbers are demo-grade until proven on messy production diversity.
• Correction to the vault: the summit note pairs "10ms" loosely with "12,000 QPS." The Databricks blog is precise: 10ms is the small-dataset number; the throughput claim is "sub-100ms at 12,000 QPS." Quote the latter, not "10ms at 12,000 QPS," in any client-facing deck.
• Contradiction worth flagging: the marketing claim of replacing dedicated serving DBs collides with the engineering reality that low-latency high-QPS serving off object storage requires an indexing/caching layer (which is exactly what Pinot/StarTree/ClickHouse are). Reyden almost certainly has such a layer internally — but that means it competes with those engines on the same hard problem, and "Beta, select read-only workloads" signals it has not yet proven it across their full workload diversity.

Synthesis for RDCO

Poor-candidate workloads for Lakehouse//RT today (do not recommend consolidation onto it for these):

1. Write-heavy / OLTP and CDC sinks. Reyden is read-only in Beta; transactional writes belong in Lakebase, not Lakehouse//RT. Anything doing high-rate inserts/updates with read-after-write expectations is out of scope.
2. Sub-5ms / single-digit-ms point lookups and edge caching. The 10ms figure is small-dataset best-case; there is no sub-10ms single-key claim. Workloads currently served by Redis/Aerospike key-value caches at the edge (session state, feature-store online lookups, rate-limiter counters) are not what Reyden is built to replace. Keep the KV cache.
3. Streaming-ingest-driven freshness at the second/sub-second boundary. Druid/Pinot ingest from Kafka/Kinesis and serve within seconds of arrival. Reyden queries the governed Delta/Iceberg copy — freshness is bounded by how fast data lands in the lake. If the SLA is "queryable within 1-2s of the event," validate the lake-landing latency before assuming Reyden meets it.
4. Compliance-zoned / physically-isolated serving. Reyden's value prop is single-copy governance under Unity Catalog. Workloads that mandate a physically separate, separately-credentialed, separately-auditable serving store (regulated data residency, air-gapped tenants) lose that isolation by consolidating. The governance simplification is a liability here, not a feature.
5. Multi-engine / non-Databricks shops. If the serving tier fronts Snowflake + Iceberg + Postgres behind a proxy (the Query Proxy pattern), Reyden only covers the Databricks/Delta slice. Consolidating onto it re-couples you to one vendor you were deliberately federating away from.
6. Cost-sensitive steady-state high-QPS serving. Reyden is Databricks-metered compute. A right-sized self-hosted ClickHouse single binary can be dramatically cheaper at predictable high QPS. Run the unit-economics before consolidating a workload that is already cheap and stable.

Migration decision tree (keep vs consolidate):

Is the workload WRITE-heavy / OLTP?            -> KEEP (route to Lakebase, not //RT)
Need sub-5ms or single-key point lookups?      -> KEEP the KV/edge cache (Redis/Aerospike)
Freshness SLA tighter than lake-landing time?  -> KEEP Druid/Pinot streaming ingest
Compliance demands physical serving isolation? -> KEEP (separate store is the control)
Serving tier spans non-Databricks engines?     -> KEEP federation / partial-consolidate only the Delta slice
Already on Databricks + read-only operational
  analytics/BI/app-serving + freshness OK at
  lake-landing latency + governance-simplification
  is a WIN + Beta risk acceptable?             -> PILOT Lakehouse//RT (shadow, do not cut over)
Steady-state, cheap, stable self-hosted CH?    -> KEEP unless governance/staleness pain is real

How RDCO/phData should advise clients. Lead with the right framing in discovery: the old reflex recommendation to "bolt on ClickHouse/Pinot for real-time" is now a question, not a default — and that repositioning is itself differentiated (most SEs will not know Reyden exists yet). But anchor the recommendation to the workload's bottleneck, not the vendor's narrative. The pattern is shadow-then-cut, never lift-and-shift: stand Lakehouse//RT up beside the incumbent serving DB, mirror the top query shapes, and measure p50/p99 and cost per query on the client's real data before retiring anything — Beta + "select read-only workloads" means the failure mode is workload-diversity, exactly what a benchmark cannot show. The honest one-liner for a client deck: "Reyden can collapse your read-serving tier if your workload is read-only operational analytics on Databricks data and lake-landing freshness meets your SLA; it does not replace your transactional DB, your edge cache, your sub-second streaming-ingest serving, or a compliance-isolated store."

Open follow-ups

• What is the actual lake-landing-to-queryable latency for Lakehouse//RT under Delta with frequent small commits? (The freshness ceiling is the make-or-break for displacing Druid/Pinot.)
• Does Reyden expose an in-engine index/cache config, or is indexing opaque/automatic? Determines tunability vs a hand-tuned Pinot/ClickHouse deployment.
• Cost model: is //RT metered as DBUs on a dedicated warehouse type, and what is cost-per-query at sustained 12k QPS vs a self-hosted ClickHouse equivalent?
• GA timeline and the roadmap for "more capabilities" beyond read-only — when (if ever) do writes/point-lookups enter scope?
• Iceberg parity: the blog says Delta and Iceberg; confirm whether Iceberg serving is at parity or Delta-first.
• Sanity Check angle test: is "the serving tier you were told to add is now the question, not the default" an original re-frame, or does it just restate the summit coverage? (Per the no-derivative-pieces rule, the original re-frame would be the decision-tree, not the announcement.)

Related

• [[2026-06-19-data-engineering-central-databricks-summit-2026]]
• [[2026-05-18-data-engineering-weekly-issue-270]]
• [[2026-04-13-data-engineering-central-lambda-kappa]]
• [[2026-06-07-data-engineering-central-duckdb-unity-catalog-commits]]

Sources

Vault:

• ~/rdco-vault/06-reference/2026-06-19-data-engineering-central-databricks-summit-2026.md
• ~/rdco-vault/06-reference/2026-05-18-data-engineering-weekly-issue-270.md
• ~/rdco-vault/06-reference/2026-04-13-data-engineering-central-lambda-kappa.md
• ~/rdco-vault/06-reference/2026-06-07-data-engineering-central-duckdb-unity-catalog-commits.md

Web:

• Databricks blog — Introducing Lakehouse//RT: https://www.databricks.com/blog/introducing-lakehousert-real-time-performance-unified-lakehouse
• Databricks press release — Lakehouse//RT launch: https://www.databricks.com/company/newsroom/press-releases/databricks-launches-lakehousert-bring-real-time-analytics-directly
• heise online — Lakehouse//RT aims to make separate real-time databases redundant: https://www.heise.de/en/news/Databricks-Lakehouse-RT-aims-to-make-separate-real-time-databases-redundant-11338027.html
• Tinybird — OLAP databases: what's new and what's best in 2026: https://www.tinybird.co/blog/best-database-for-olap
• ksolves — Pinot vs Druid vs ClickHouse: https://www.ksolves.com/blog/big-data/pinot-vs-druid-vs-clickhouse
• ClickHouse — How to choose a database for real-time analytics in 2026: https://clickhouse.com/resources/engineering/how-to-choose-a-database-for-real-time-analytics-in-2026
• StarTree — How StarTree serves 500+ QPS on Iceberg data: https://startree.ai/resources/real-time-analytics-with-iceberg/
• Onehouse — Announcing Onehouse LakeBase (object-store serving latency physics): https://www.onehouse.ai/blog/announcing-onehouse-lakebase-database-speeds-finally-on-the-lakehouse