Module 2 · Operating agents · Lesson 6
Continuous improvement
How a production agent gets better instead of quietly worse. Online signals, drift, eval-gated model and prompt upgrades with paired statistics, canaries and shadow runs, and why A/B tests are slow at low volumes.
By the end of this lesson you can
- Describe the improvement flywheel from production traces to eval cases to gated releases
- Name the kinds of drift that affect a production agent and the signal that detects each
- Compare two agent versions on the same eval tasks with a paired test, and decide whether a difference is real
- Choose between offline gates, shadow runs, canaries and A/B tests given your traffic and risk
- Set up an upgrade gate for a new model in eve or Claude Code, with category-level blocking rules