Module 3 · Building systems · Lesson 6
Routing on confidence
Turning Jev's probabilities into act, confirm and escalate decisions, with thresholds derived from the cost of each kind of error, per-action gates, intent routing, composite scores, and the research on selective prediction, deferral and conformal sets.
By the end of this lesson you can
- Derive an act-or-escalate threshold from the cost of a wrong action and the cost of review, and extend it to three bands
- Convert between a Choice answer's confidence and its top probability, and explain which one an expected-cost rule needs
- Design a router with a global confidence floor, per-action thresholds and hard rules that no probability can override
- Read a risk–coverage table and choose a threshold that meets a target error rate on covered items
- Explain what learning to defer and conformal prediction sets add to plain thresholding, and compute a conformal set by hand