Step-by-step answer
Core 75–90 min7 stepsBuild a Claude Code judge and measure its position bias
A pairwise answerability judge on claude -p, run on 20 synthetic pairs in both orders, with an analysis of first-slot win rate, consistency, accuracy against a hidden truth and length bias, before and after one rubric revision.
Back to the exercise