A chat app where you ask questions about healthcare claims in plain language, and an AI agent writes and runs the SQL to answer them. I built the agent twice: once on Databricks (MLflow model on Model Serving, SQL warehouse, Unity Catalog) and once fully on open building blocks (FastAPI on Kubernetes, PostgreSQL, Llama via a European AI API). Both use the same model, the same prompt and the same SQL tool, and expose the same API, so the Streamlit front-end switches between them with one radio button. The point: the same solution can run on a commercial platform or on a sovereign open stack, and you can compare them side by side.
Text-to-SQL agent: Databricks or open source, one app
The app
One Streamlit app. The sidebar picks the backend; the example questions are the same for both.
Screenshot of the app with the open-source backend selected. Data: synthetic sample, no real patients.
Same app, two stacks
Switch between the backends: grey layers are shared, coloured layers are swapped. It switches automatically; click to choose.
How the agent answers
A tool-calling loop: the model does not make up numbers, it writes SQL and the database computes the answer.
QuestionFrom the chat, including the earlier conversation.
LLM decidesWith the schema and rules in the system prompt, and
run_sql as available tool.Writes SQLCalls
run_sql with a SELECT on the claims table.Database runs itSQL warehouse or PostgreSQL returns the rows as JSON.
AnswerThe LLM summarises the result in plain language, or queries again.
↺ at most 10 tool rounds per question, then the agent stops
What I learned
Same contract, swappable backendThe open-source service copies the
/invocations interface of Databricks Model Serving. The front-end only changes a URL, so moving between platforms costs nothing in the app.Platform versus building blocksDatabricks gives managed serving, governance and an auto-stopping SQL warehouse out of the box. The open version is a small FastAPI service (128 MiB memory) on my own cluster in the EU, with full control and no platform fee.
Only the SQL dialect differedModel, prompt and tool are identical. The one difference in the prompt: cast amounts to DOUBLE in Databricks SQL, to NUMERIC in PostgreSQL.
Schema in a prompt does not scaleThis first version puts the table schema and rules in the prompt. Fine for one table, but there is no business meaning, and rules in a prompt are not enforcement. That led to the semantic layer in OpenMetadata and a read-only query tool, see my AI principles.