Calibrate your Claude Code judge against your own labels

Calibration is a loop with two guard rails. The loop: run the judge on dev, read every disagreement next to your critique, change the prompt using train examples only, run dev again. The guard rails are in code, not in your memory: the runner refuses a prompt that quotes a dev or test item, and it runs the test split once and refuses a second time. The decision that matters most is what to do with each disagreement: the three buckets (rubric gap, judge error, label error) lead to different fixes, and this answer uses two of them.