"Building Agents (and tools) ... simply." — Daniel Beach, Data Engineering Central (Nov 9 2026)
Why this is in the vault
A full, code-level walkthrough of a production internal-support agent (knowledge base, Lakehouse lookup, web search, CSV analysis) built with zero MCP servers, zero vector-DB SaaS, and zero agent frameworks — a direct, well-argued challenge to the "you need an MCP server / Forward Deployed Engineer / expensive platform" framing RDCO's own agent-deployer positioning pushes against.
The core argument
Beach's thesis: nothing about agentic tooling is architecturally new — it's APIs, containers, SQL, CI/CD, and an LLM call sitting behind the simplest part of the stack. He walks a full worked example for a fictional "Acme Widgets" support agent, built entirely without MCP servers or managed vector-DB SaaS:
- LLM calls: plain OpenAI-compatible chat/completions call through Databricks Model Serving — "the easiest part of the entire system."
- Knowledge base: LanceDB on S3, built offline with the same embedding model at index- and query-time; explicitly rejects managed/SaaS vector DBs at this scale.
- Lakehouse lookups: parameterized, pre-written SQL behind a governed service-principal call — no semantic layer.
- Web search: Databricks'
ai_enrichSQL function wired as a plain tool-call function, no MCP wrapper. - File/CSV analysis: DuckDB in-memory, single-SELECT-only guard.
- Deploy: Dockerfile → ECS/Fargate behind an ALB (Terraform) → GitHub Actions build/push/update-service pipeline.
His recurring target is the LinkedIn "AI Engineer" / Forward Deployed Engineer framing that implies this requires specialist consultants — he argues any senior engineer with a systems-design mindset and a few days of docs-reading (aided by Claude) can build and ship it. Tone is deliberately combative/rant-flavored ("rage against the machine," "what a bunch of crap") wrapped around genuinely concrete, runnable pseudocode — not just a pure rant.
Mapping against Ray Data Co
This directly engages RDCO's agent-deployer thesis ([[2026-04-14-levie-agent-deployer-role-jd]]) from the opposite angle: Levie's JD argues the agent-deployer role is valuable precisely because most orgs can't self-serve this integration work; Beach argues the opposite — that any senior engineer can, and the "it's complicated" framing is largely vendor/consultant rent-seeking. Both can be true depending on org size and governance maturity, but this is a real counter-data-point worth holding against RDCO's pitch, not just confirmation. It also reinforces the vault's existing MCP-skepticism thread ([[2026-04-19-indydevdan-ditching-mcp-servers]]: "ditching MCP servers" for raw code-as-tools) with an independent, non-coding-tool-specific data point — Beach's "put the tool call directly in the main Agent loop" is the same architectural instinct as IndyDevDan's raw-code-over-MCP stance, arrived at from a data/platform-engineering angle rather than a coding-agent angle. For RDCO's positioning: if the market consensus among senior practitioners converges on "you don't need MCP, you don't need a platform team, you need systems design," RDCO's sales pitch needs to lean harder on governance/scale/maintenance-at-headcount (the things Beach waves away as "just deploy 5 containers") rather than on raw technical capability, since capability alone is what this piece argues against paying for.
Related
- [[2026-04-14-levie-agent-deployer-role-jd]] — the agent-deployer JD this piece argues against by implication (self-serve vs. specialist-needed framing).
- [[2026-04-19-indydevdan-ditching-mcp-servers]] — independent, earlier confirmation of the same "skip MCP, use raw code/tool calls" instinct from a coding-agent angle rather than data-platform angle.
- [[2026-08-10-data-engineering-central-quasi-agentic-pipelines-databricks-airflow]] — same sender/author's prior piece on bolting agentic steps onto Databricks+Airflow; shares the deterministic-vs-non-deterministic framing this issue's tool-calling pattern assumes.
- [[project_credibility_for_phdata_sales]] — RDCO's "agents in production, measure pipeline, no selling" posture this piece's self-serve argument bears directly on.