// fundamentals — ai agents on business data

AI Agents on Business Data — Roadmap

Agents that answer questions with your company's data, and get the numbers right: tool use, MCP, text-to-SQL, the semantic layer, permissions and evals. Building projects come at the end.
Assumes SQL and a modeled warehouse. Start with Data Modeling →
Total
32
Done
0
Progress
0%
Left
32
Est. time
~21 h

How a model sees your data

llm 0/4 ▾

Context windows and tokens

All the model knows about your data is what you put in the window. Schemas, rows and docs compete for the same space.
AI Engineering, ch. 2

Models don't compute, they predict

Ask a model to sum a column and it guesses. Every number must come from a query or code it ran, never from memory.

Prompting with schemas and examples

Give it the tables, what each column means and two good answers. That beats any clever wording.
AI Engineering, ch. 5

Structured outputs

Have the model return JSON that matches a schema, so your code can check and use the answer.

Tools & agents

tools 0/5 ▾

Tool use

The model asks to call a function, your code runs it and returns the result. This is how it reaches your data.

Workflows vs agents

A fixed chain of steps is easier to test than a loop that decides for itself. Start with the simplest one that works.
AI Engineering, ch. 6

Designing tools for agents

Few tools with clear names, helpful errors and compact results. A tool is an interface for a model, not an API wrapper.

MCP: one protocol for tools and data

Expose your warehouse once, as an MCP server, and any compatible client can use it.

Build an MCP server

Follow the tutorial end to end and connect your server to a client.

Text-to-SQL

sql 0/5 ▾

What text-to-SQL gets right and wrong

High scores on clean benchmark schemas, much lower on real enterprise warehouses. Know where the gap is before you promise anything.

Schema linking: give it the right tables

Pick the few tables and columns a question needs before writing SQL. Dumping the whole schema makes it worse.

Verified queries as examples

Questions your team already answered, with the SQL they trust. The agent reuses them instead of guessing.

Read-only, limited, timed out

A read-only role, row limits and a statement timeout. The agent should not be able to hurt the warehouse.

Show the query with the answer

Every number comes with the SQL that produced it, so a person can check it.

The semantic layer as the contract

semantic 0/4 ▾

Query metrics, not tables

With a semantic layer the agent asks for revenue by month and the definition does the joins. Fewer ways to be wrong.

The dbt MCP server

Metrics, models and lineage from a dbt project, exposed as tools an agent can call.

Semantic models for Cortex Analyst

The same idea in Snowflake: a YAML file that tells the model what tables, measures and synonyms mean.

Descriptions are prompts

Model and column descriptions are what the agent reads to choose. Write them for that reader.

Retrieval over documents

retrieval 0/3 ▾

Embeddings and vector search

Find the paragraphs that answer a question by meaning, not by keyword.
AI Engineering, ch. 6

Contextual retrieval

Add each chunk's context before you embed it, and combine vector search with keyword search.

Retrieve or query?

Policies and definitions come from documents; numbers come from the warehouse. Route each question to the right one.

Permissions & safety

security 0/4 ▾

The agent acts as the user

It should see only what the person asking can see. Enforce that in the database, not in the prompt.

Masking sensitive columns

Names, IDs and health data masked by policy, so they never reach the model unless they have to.

Prompt injection through data

A row, a ticket or a PDF can carry instructions. Treat what the agent reads as data, and limit what its tools can do.

A person approves every write

Reading is the agent's job. Updating a record, sending an email or running a job waits for a human yes.

Evals & operations

evals 0/4 ▾

A test set of real questions

Collect the questions people really ask, with answers you checked by hand. Run them on every change.
AI Engineering, ch. 4

Grading agents

Compare query results, not SQL text. Use a model as judge only where code can't decide.
AI Engineering, ch. 3

Tracing every step

Log each prompt, tool call, query and result, so you can see why an answer was wrong.

Cost and latency

Cache the long, fixed parts of the prompt and use the smallest model that passes your evals.

// bibliography

The books, courses and practice sites behind this roadmap.

// books
AI Engineering ↗
Chip Huyen · cited in 6 topics
Building applications on foundation models: evaluation, prompts, RAG, agents.

// other resources

Courses for the basics this roadmap doesn't cover.

Anthropic Academy ↗
Anthropic · free
The Claude API, tool use, MCP and agents, in short courses.
AI Agents Course ↗
Hugging Face · free
Agents from zero: tools, the agent loop, frameworks and a final project.
Select Star SQL ↗
Zi Chong Kao · free
SQL by doing: joins, aggregation, subqueries.

Building

building 0/3 ▾

An MCP server over your warehouse

Read-only tools to list tables, describe one and run a limited query, on DuckDB or Postgres.

A metrics agent on the semantic layer

Point the agent at the metrics from the Data Modeling roadmap and have it answer business questions, with the query shown.

An eval suite for your agent

Thirty real questions with checked answers, graded on every change to the prompt, the tools or the model.