Total
32
Done
0
Progress
0%
Left
32
Est. time
~21 h
How a model sees your data
llm 0/4 ▾Context windows and tokens
All the model knows about your data is what you put in the window. Schemas, rows and docs compete for the same space.
AI Engineering, ch. 2
Models don't compute, they predict
Ask a model to sum a column and it guesses. Every number must come from a query or code it ran, never from memory.
Prompting with schemas and examples
Give it the tables, what each column means and two good answers. That beats any clever wording.
AI Engineering, ch. 5
Structured outputs
Have the model return JSON that matches a schema, so your code can check and use the answer.
Tools & agents
tools 0/5 ▾Tool use
The model asks to call a function, your code runs it and returns the result. This is how it reaches your data.
Workflows vs agents
A fixed chain of steps is easier to test than a loop that decides for itself. Start with the simplest one that works.
AI Engineering, ch. 6
Designing tools for agents
Few tools with clear names, helpful errors and compact results. A tool is an interface for a model, not an API wrapper.
MCP: one protocol for tools and data
Expose your warehouse once, as an MCP server, and any compatible client can use it.
Build an MCP server
Follow the tutorial end to end and connect your server to a client.
Text-to-SQL
sql 0/5 ▾What text-to-SQL gets right and wrong
High scores on clean benchmark schemas, much lower on real enterprise warehouses. Know where the gap is before you promise anything.
Schema linking: give it the right tables
Pick the few tables and columns a question needs before writing SQL. Dumping the whole schema makes it worse.
Verified queries as examples
Questions your team already answered, with the SQL they trust. The agent reuses them instead of guessing.
Read-only, limited, timed out
A read-only role, row limits and a statement timeout. The agent should not be able to hurt the warehouse.
Show the query with the answer
Every number comes with the SQL that produced it, so a person can check it.
The semantic layer as the contract
semantic 0/4 ▾Query metrics, not tables
With a semantic layer the agent asks for revenue by month and the definition does the joins. Fewer ways to be wrong.
The dbt MCP server
Metrics, models and lineage from a dbt project, exposed as tools an agent can call.
Semantic models for Cortex Analyst
The same idea in Snowflake: a YAML file that tells the model what tables, measures and synonyms mean.
Descriptions are prompts
Model and column descriptions are what the agent reads to choose. Write them for that reader.
Retrieval over documents
retrieval 0/3 ▾Embeddings and vector search
Find the paragraphs that answer a question by meaning, not by keyword.
AI Engineering, ch. 6
Contextual retrieval
Add each chunk's context before you embed it, and combine vector search with keyword search.
Retrieve or query?
Policies and definitions come from documents; numbers come from the warehouse. Route each question to the right one.
Permissions & safety
security 0/4 ▾The agent acts as the user
It should see only what the person asking can see. Enforce that in the database, not in the prompt.
Masking sensitive columns
Names, IDs and health data masked by policy, so they never reach the model unless they have to.
Prompt injection through data
A row, a ticket or a PDF can carry instructions. Treat what the agent reads as data, and limit what its tools can do.
A person approves every write
Reading is the agent's job. Updating a record, sending an email or running a job waits for a human yes.
Evals & operations
evals 0/4 ▾A test set of real questions
Collect the questions people really ask, with answers you checked by hand. Run them on every change.
AI Engineering, ch. 4
Grading agents
Compare query results, not SQL text. Use a model as judge only where code can't decide.
AI Engineering, ch. 3
Tracing every step
Log each prompt, tool call, query and result, so you can see why an answer was wrong.
Cost and latency
Cache the long, fixed parts of the prompt and use the smallest model that passes your evals.
// bibliography
The books, courses and practice sites behind this roadmap.
// books
AI Engineering ↗
Building applications on foundation models: evaluation, prompts, RAG, agents.
// other resources
Courses for the basics this roadmap doesn't cover.
Anthropic Academy ↗
The Claude API, tool use, MCP and agents, in short courses.
AI Agents Course ↗
Agents from zero: tools, the agent loop, frameworks and a final project.
Select Star SQL ↗
SQL by doing: joins, aggregation, subqueries.
Building
building 0/3 ▾An MCP server over your warehouse
Read-only tools to list tables, describe one and run a limited query, on DuckDB or Postgres.
A metrics agent on the semantic layer
Point the agent at the metrics from the Data Modeling roadmap and have it answer business questions, with the query shown.
An eval suite for your agent
Thirty real questions with checked answers, graded on every change to the prompt, the tools or the model.