Prove how good your agent is with numbers you can defend, then make it better: prompts first, then the model itself.
Path 2 of 4, taken after LLM and Agent Foundations. You finish able to measure an agent end to end and improve it with evidence. First you evaluate rigorously: error analysis, code graders, calibrated judges and honest statistics, then multi-turn benchmarks with simulated users and stateful environments that survive an audit. Then you optimise prompts, examples, tools and skills by hand and with DSPy and GEPA, bring people back into the loop with human studies, preference rankings and user signals, and finally train a small open model with fine-tuning and RL, shipping it only if your eval suite says it beats the optimised prompt.
5 courses38 lessons~135 hours
In this path
5
Courses
38
Lessons
~135
Hours
Each course builds on the one before it. Overviews are open to everyone; sign in to start the lessons.
Turn "it seems better" into numbers you can defend: error analysis, code graders, calibrated LLM judges and honest statistics.
2 modules · 7 lessons ~20h 7 quizzes Capstone
Modules
Capstone: A full eval suite for your drafting agent
Evaluate multi-turn, stateful agents with simulated users and real environments, and build benchmarks whose numbers survive an audit.
3 modules · 9 lessons ~32h 9 quizzes Capstone
Modules
Capstone: A benchmark for the account-support agent you'd defend
Treat prompts, examples, tool descriptions and skills as versioned, measured artefacts, and improve them by hand, by optimiser and in production without breaking what already works.
2 modules · 7 lessons ~17h 7 quizzes Capstone
Modules
Capstone: Optimise and ship the questions drafter
Collect, measure and trust the judgements of reviewers and users: study design, rater agreement, preference rankings, real-use signals and quality estimates from a few labels.
3 modules · 7 lessons ~28h 7 quizzes Capstone
Modules
Capstone: Build a feedback system for a support assistant
Train a small open model for one agent step with supervised fine-tuning, preference optimisation and reinforcement learning, and prove with your eval suite whether it beats an optimised prompt.
3 modules · 8 lessons ~38h 8 quizzes Capstone
Modules
Capstone: Train a triage model, then decide whether to ship it