Module 1 · Foundations of evaluation · Lesson 1
Why evals, and their vocabulary
What an agent eval is, the words you need to talk about one precisely, and why a layered set of checks beats any single score.
By the end of this lesson you can
- Define task, trial, grader, transcript, outcome, evaluation harness and agent harness, and map each to eve and Claude Code
- Distinguish capability evals from regression evals and say when a task graduates from one to the other
- Choose between code-based, model-based and human graders for a given success criterion
- Explain the Swiss-cheese model and place evals alongside approvals, monitoring and expert review in a high-stakes workflow