Module 2 · Scoring that you can trust · Lesson 4
LLM-as-judge
Grade what code can't, such as redundancy, clarity and grounded reasoning, with a judge model and a rubric designed for one failure mode, while controlling for the judge's known biases.
By the end of this lesson you can
- Decide when a criterion needs a judge, and write a binary, single-failure-mode judge prompt with definitions, examples and a way out
- Compare pointwise, pairwise and reference-based judging, and choose one for a given question
- Explain position, verbosity and self-preference bias, and apply a mitigation for each
- Use eve's t.judge (criteria, typed questions, batches, thresholds) and configure its judge model
- Build a subscription-friendly judge with Claude Code headless and record it with t.score