"Cloudflare as a Data Platform?"
Why this is in the vault
Hands-on evaluation of Cloudflare's emerging data stack (R2, Iceberg, Data Catalog, Pipelines) from a practicing data engineer — directly relevant to RDCO's existing Cloudflare Workers infrastructure.
⚠️ Sponsorship
Cube (agentic analytics/embedded dashboards platform) explicitly sponsors this issue. The editorial is independent of the sponsor — the Cube block is a discrete section, not woven into the Cloudflare evaluation.
The core argument
Cloudflare is positioning itself as a full data platform — "Lake House" style — with four core components: R2 Storage (S3-equivalent), Apache Iceberg on R2, R2 Data Catalog (REST Iceberg catalog with auto-compaction), and R2 SQL (query layer). The author builds a working lake house using Airflow + Polars + pyiceberg against the REST catalog, and it works cleanly.
The critical gap: Cloudflare's native "Pipelines" feature is streaming-only (built on their Arroyo acquisition). There is no native batch pipeline support. For batch workloads — which the author estimates at 80%+ of real pipelines — you must bring your own orchestrator (Airflow, etc.). The workaround is viable but represents a missing piece in Cloudflare's "one-stop shop" pitch.
Additional notes from the hands-on evaluation:
- R2 SQL is CLI-only via Wrangler (no notebook UI), which the author flags as below industry standard
- Pipelines require a paid plan (not available on free tier)
- The R2 + Iceberg + Data Catalog pieces are genuinely simple and well-documented — pyiceberg connects via REST catalog config, Polars writes Iceberg natively, Arrow interoperability is smooth
- Airflow DAG code is clean and minimal against this backend
- Verdict: "it just works" for the storage/catalog layer; Pipelines gap is real if you want native batch
Mapping against Ray Data Co
RDCO already deploys on Cloudflare Workers via Wrangler (hq.raydata.co). The R2 + Iceberg + Data Catalog stack described here could serve as a zero-new-vendor data backend for agent memory, event logging, or lightweight analytics — everything stays on the Cloudflare edge without adding a separate warehouse. The Wrangler CLI pattern for R2 SQL is the same toolchain RDCO already uses for HQ deployment, which lowers the adoption cost. The streaming-only Pipelines limitation is not an immediate constraint for RDCO's current workloads (batch/agent-driven access patterns dominate). The main decision surface: whether R2 + Iceberg is worth evaluating as an alternative to a managed warehouse for RDCO's own data needs.
Related
- [[2026-07-13-data-engineering-weekly-278]]
- [[2026-07-10-data-engineering-weekly-privacy-safe-data-platform]]
- [[2026-06-08-indy-dev-dan-ranked-cloudflare-software-factory-s-tier-tokenomics]]
- [[2026-05-19-cloudflare-cyber-frontier-models]]
- [[2026-06-17-data-engineering-central-lakehouse-delta-lake-rust]]