Each lesson in this course built one piece of an evaluation on its own: an environment graded on its end state, a simulated customer, a harness with controls, a noise budget, an audit, an item analysis, a report. This project puts them together into one benchmark, Quillmoor-Bench, for Quillmoor's account-support agent: Claude Code acting through MCP tools on a SQLite backend while a simulated customer (sometimes impatient, sometimes evasive, sometimes lying) talks to it. You'll end with something you could hand to a sceptical reviewer: a versioned task set, a harness anyone can rerun, an audit trail of every task you fixed, and a one-page report card where each number has an interval and a reason to trust it.
It uses every module. Simulated worlds gives you the world and the customer, and the evidence that the customer behaves like one. Running benchmarks packages the tasks for Harbor and runs them reproducibly. Benchmarks you can trust turns you on your own benchmark: re-annotation, an exploit hunt, item analysis and the report. Everything runs on your Claude subscription; an OpenRouter key or a local model is optional. All accounts, invoices and conversations are synthetic. Expect 8 to 12 hours of work, plus a few hours of wall-clock time while trials run.