The AI Engineer

"AI engineer" job postings ask for evals, agents, RAG and TypeScript in the same breath, and a new graduate reads that list very differently from a backend engineer with ten years behind them. Before any code, you need to know what the job is, what you already bring, and how these 100 days get you there. Today: the map, a baseline score and a study plan, with no code.

At a Glance

• Time: ~2 h core (+ optional Go Deeper) · Depth: intro
• Prerequisites: none. If the terminal or Git is new to you, read A1: Terminal, Git & GitHub before Day 2.
• You'll build: skills_audit.md, a self-scored baseline (20 items, 100 points max) that you re-score on Day 63 and Day 100, plus a weekly study plan
• Stops at: no code
• Fast path: Already work with LLM APIs? Skim the role table in §What AI Engineers Actually Build, read §How to Read a Lesson, then do Exercise 1 (about 30 min).
• Cost: $0. No API calls today.
• New terms: AI engineer, foundation model, eval, agent, RAG

What AI Engineers Actually Build

Search a job board for "AI" and you get four different jobs behind similar titles. Applying to the wrong one wastes months, and studying for the wrong one is worse. Start by separating them.

The plain version: an AI engineer builds products that call a model the way an app calls a payment service. Someone else builds the model. That model is a foundation model: a large model trained by an AI lab on broad data and used by everyone else through an API. You don't train it. You decide what to send it, check what comes back, connect it to your data and tools, measure whether it works, and run it in production at a sensible cost.

Roles blur at small companies. The test: does the job train models or build with them? This course builds with them; training appears only in Phase 10. Three more terms show up constantly in postings:

• RAG (retrieval-augmented generation): look the relevant documents up first, then have the model answer from them. Phase 3 builds one.
• Agent: a loop in which the model picks the next step (call a tool, read a result, decide again) until the task is done. Phase 6 builds one.
• Eval: an automated check of the model's behaviour, run on a fixed set of examples, so you can tell whether a change made things better or worse. You write your first on Day 20: Your First Eval Set.

A Week on the Atlas Team

Throughout the course you build Atlas, an AI assistant for the maintainers of Orbit, a fictional open-source Python library for background jobs. Here is a week as Atlas's AI engineer; each tag says which day teaches the task.

• Monday: the extractor emits bad labels. Atlas turns new Orbit issues into typed records, and overnight some came back with components that don't exist. You tighten the schema and add a bounded re-ask for valid-but-wrong answers. (Day 16, Day 17)
• Tuesday: a wrong citation. Asked about Orbit 3.2 config, Atlas cites the 2.x page on orbit.ini, which 3.0 replaced with orbit.toml. You fix retrieval before touching the prompt. (Day 28)
• Wednesday: is the new prompt better? Instead of eyeballing five outputs of a teammate's new triage prompt, you score it against 30 hand-checked issues and compare with the old one. (Day 20)
• Thursday: the bill doubled. Whole doc pages started going into every request. You measure tokens per call, reorder the prompt so the provider's cache can reuse it, and add an alert. (Day 18)
• Friday: review an agent's diff. A coding agent opened a pull request. You review it like any contributor's: tests, the eval gate, and the repo's AGENTS.md rules. (Day 60)

No equations, no training, no GPU code: most of the week is ordinary engineering around one model component.

What Employers Ask For (2026 Signal)

Course research counted keywords in 320 public AI-engineer, applied-AI and forward-deployed-engineer postings, taken from the job boards of nine companies on 2026-11-06. Read it as a signal, not a census: the sample leans heavily toward AI vendors (companies that sell models or AI tools), so it over-represents their stacks, and a keyword showing up in a posting doesn't tell you how much the job uses it.

Three readings of this table shape the whole course:

1. Evals lead. Proving a feature works is the top ask, so evals arrive early and Phase 5 comes before agents: you can't show an agent improved without a measurement.
2. Patterns beat frameworks. Named frameworks score in single digits or zero. Learn the loop, the retrieval and the measurement; any team's framework is then quick to pick up.
3. Python first, TypeScript useful. A third of postings mention TS/JS, mostly for UI work; the primer is there when you need it.

Two Tracks: Switching Roles vs First Role

The same course serves two kinds of learner, and the advice differs. Pick the track that fits now; you can change later.

Freshers: entry-level loops lean on coding and on explaining a project you built end to end, not on senior system design. DSA practice takes months, so it starts on Day 63 and runs to Day 98.

How This Course Works

The 100 days form 12 phases, and five of them ship a release of Atlas. For the day-by-day list, see the course index.

Atlas, in five releases. Each reuses the one before: one product, not five demos. Unfamiliar terms in this table (eval gate, p95, SLOs) are defined on the days that teach them.

Orbit, the data. Orbit is fictional, written for this course so every lesson uses the same realistic, legally clean corpus (CC0, public domain). datasets/orbit/ holds 45 docs pages (plus messy HTML copies and two poisoned pages for the security days), 240 issues with 504 comments, a SQLite copy of the tracker, and small gold sets for grading. The issues include duplicates, fake personal data, non-English text and injection attempts on purpose, because real trackers do.

Primers are short refreshers outside the 100 days (terminal, Python, async, HTTP, SQL, Docker, testing, model vocabulary and more). Read one when a lesson's At a Glance names it. Before Day 2 you need only A1, A2 and A5; the full reading order is in Go Deeper below.

How to Read a Lesson

Every day has the same layout, so you can decide in two minutes what to read closely and what to skip. The markers at the top are the tools for that decision.

Two habits make the course work: run the code (most AI bugs appear only when the model answers), and try each exercise before opening its solution.

Your Baseline Skills Audit

You can't measure progress without a starting point. The audit below has 20 items in 8 groups that follow the phases. Score each from 0 to 5 using these anchors, total them (100 is the maximum), and save the file as skills_audit.md. You'll re-score the same file on Day 63 and Day 100, so be honest: a low score now makes the progress visible later.

# Skills audit: baseline (Day 1), date: ____
Scores 0-5. Re-score on Day 63 and Day 100.

| # | Group | Skill | D1 | D63 | D100 |
|---|---|---|---|---|---|
| 1 | APIs & prompting | Call an LLM API from code and read its usage and cost | | | |
| 2 | APIs & prompting | Write clear prompts: instructions, examples, output format | | | |
| 3 | APIs & prompting | Reason about tokens, context windows and choosing a model | | | |
| 4 | Structured output | Get typed, schema-valid JSON out of a model | | | |
| 5 | Structured output | Handle valid-but-wrong outputs (validate, re-ask, bound retries) | | | |
| 6 | Retrieval | Embeddings and vector search | | | |
| 7 | Retrieval | Chunking, hybrid search and reranking | | | |
| 8 | Retrieval | A RAG pipeline with citations, and measuring its retrieval | | | |
| 9 | Tools & MCP | Function calling and designing tools a model can use well | | | |
| 10 | Tools & MCP | MCP: use an MCP server and build one | | | |
| 11 | Evals & observability | Build an eval set and run it on every change | | | |
| 12 | Evals & observability | LLM-as-judge, checked against human labels | | | |
| 13 | Evals & observability | Tracing and error analysis (from traces to failure modes) | | | |
| 14 | Agents | An agent loop with stop conditions, budgets and context management | | | |
| 15 | Agents | Durable agents with human approval (e.g. a graph framework) | | | |
| 16 | Agents | Working safely with coding agents (AGENTS.md, reviewing their diffs) | | | |
| 17 | Safety | Prompt injection and threat-modelling an AI feature | | | |
| 18 | Safety | Guardrails, safe output handling and personal data (PII) | | | |
| 19 | Production & ops | Serve an AI feature: API, streaming, background jobs, deploy | | | |
| 20 | Production & ops | Run it: reliability, cost control, monitoring, safe releases | | | |

Total (max 100): D1 ___  D63 ___  D100 ___
Top 3 gaps: 1. ____  2. ____  3. ____

Score what you can do, not what you've read about; when unsure, take the lower number.

The Mindset Shift

Engineers are trained on deterministic systems, where a passing test stays passing. Models break that in three ways, and teams adapt in three matching ways.

• Outputs are probabilistic, so teams measure instead of eyeballing. One good answer proves little; a prompt tweak that fixes one case often breaks two others, so every change is scored.
• Changes are experiments, so teams work eval-first. Hypothesis, change, measure, keep or revert: performance tuning with a benchmark, applied to behaviour.
• Tools churn, so teams learn patterns. Model names, SDKs and frameworks change within months; patterns (structured output, retrieval, tools, the agent loop, evals) last.

When AI isn't the answer. A model is the costly, slow, sometimes-wrong option; use it when input is messy language or the rules are too many to write down.

In production

• Teams wrap every model call in their own module, because model IDs and SDKs change fast.
• Teams track cost per request from the first prototype; token bills grow with traffic and prompt length.
• Teams treat prompts as code (versioned, reviewed, tested) because a one-word edit can change behaviour across thousands of requests.
• Teams gate releases on evals the way they gate on unit tests.
• Teams require human approval before a model acts on users' behalf.

Go Deeper (optional)

Common Pitfalls

• Tool-chasing. Symptom: twenty bookmarked frameworks, no finished project. Fix: learn the pattern once; CrewAI and LlamaIndex appear in 0 of 320 sampled postings (one row each on Day 57).
• Skipping evals because they feel like QA. Symptom: "it looked fine in the demo", with no proof a change helped. Fix: they're the top skill (52%); the habit starts on Day 20.
• Freshers measuring themselves against senior loops. Symptom: anxiety about distributed systems design before the first project works. Fix: follow the First-role track: finished projects plus a DSA plan from Day 63.
• No baseline. Symptom: on Day 63 you can't say what you've learned. Fix: fill in today's audit, even roughly, and commit it.
• Spending before budgeting. Symptom: a surprise bill or unused subscription. Fix: buy nothing until Day 2 sets a spend limit and estimates the course's total cost.

Key Takeaways

• An AI engineer builds products on foundation models through APIs; ML engineers train models, MLOps engineers run the infrastructure, and data scientists answer questions with data.
• Most of the job is software engineering around one probabilistic component: schemas, retrieval, measurement, cost control and review.
• In the 2026 sample, evals (52%), Python (41%) and agents (36%) lead the list, and specific frameworks barely register, so the course teaches patterns first and measures everything.
• Pick a track (Switching roles or First role) and use each lesson's Fast path, Stops at and Where we stop markers to decide what to read closely and what to skip.
• Your 20-item, 100-point baseline audit is the yardstick you re-score on Day 63 and Day 100.

References & Further Reading

• Building effective agents (Anthropic): a clear picture of what teams build with models, and why simple workflows often beat agents.
• LLM evals FAQ (Hamel Husain): a practitioner's view of why evals are central and how teams start.
• OpenAI models overview: the model catalog for the course's main provider; skim it to see how tiers are presented.
• AGENTS.md: the convention for instructing coding agents, which you start using on Day 2.
• CS50's Introduction to Programming with Python: a free course if you need to reach the entry bar for Python.
• The Python Tutorial: the official tutorial, for filling specific gaps quickly.