Module 2 · Changing the model, and governing the spend · Lesson 5
Cost-quality curves, routing and cascades
Measuring the cost-quality frontier with evals, then deciding whether static routing, a learned router or a cheap-first cascade moves you along it, with the arithmetic that shows when a cascade beats the strong model and when its verifier caps it.
By the end of this lesson you can
- Build a cost-quality curve from an eval across models and effort levels, and identify its Pareto frontier
- Derive the cost and accuracy of a two-model cascade from the cheap model's error rate and the verifier's recall and false-alarm rate
- Explain how FrugalGPT, RouteLLM, Hybrid LLM and AutoMix route or cascade, and what their reported savings depend on
- Size an eval large enough to tell two configurations apart, and account for cache and latency effects of routing