Why this is in the vault
AWS Lambda MicroVMs are a new AWS compute primitive that targets the gap between Lambda (simple, stateless, 15-min limit) and EC2 (full control, high complexity). The author builds a working implementation, then reverses his initial enthusiasm completely — concluding that MicroVMs combine the worst properties of both Lambda and EC2 for most data engineering workloads. The narrow valid use case is AI/agentic multi-tenant code sandboxing. Filed as a calibration reference for evaluating AWS serverless architecture options in client deal scoping.
⚠️ Sponsorship
Sponsored by Delta Lake — two explicit sponsor blocks at the top. Author states he uses Delta Lake daily. Treat technical endorsements of open table formats within this issue as potentially biased.
Mapping against Ray Data Co
The article's reversal verdict matters directly for phData deal scoping: when a prospect asks whether MicroVMs fit their data pipeline or AI agent workload, the answer is almost certainly no for pipelines and only-maybe for agent sandboxing. The implementation exposes the key friction — there are no native event triggers (no S3, SQS, or EventBridge integration), the lifecycle (build image → run → HTTP trigger → teardown) adds EC2-class operational overhead without EC2's flexibility, and the 8-hour maximum lifetime undercuts the persistent-state value prop. For RDCO's own agent infrastructure, MicroVMs are not a fit: the Mac Mini COO agent runs on-prem and the agentic use case that MicroVMs do solve (isolated multi-tenant execution of user- or AI-generated code) applies only if RDCO were building a SaaS product where end-users run arbitrary code. The one actionable signal for phData engagements: if a client is building an online IDE, SQL playground, or agentic coding assistant requiring per-session VM isolation, MicroVMs become a real candidate — this is the one narrow recommendation window.
Technical specs worth retaining: 32 GB memory ceiling (vs. Lambda's 10 GB), Firecracker-backed VM isolation (same hypervisor powering standard Lambda), suspend/resume state for up to 8 hours, HTTP/2 + gRPC + WebSockets support, Docker-image deployment model via S3.
Related
- [[2026-04-13-data-engineering-central-lambda-kappa.md]] — same newsletter's treatment of Lambda/Kappa architecture; the compute model assumptions there are now complicated by MicroVMs sitting between the two
- [[2026-06-07-data-engineering-central-duckdb-unity-catalog-commits.md]] — same author/newsletter; also tags aws-lambda and covers single-node compute trade-offs for data processing workloads