Step-by-step answer
Core 60–90 min8 stepsBuild a paired upgrade gate for Claude Code runs
Thirty synthetic drafting tasks with code graders, a runner that calls claude -p three times per task for each prompt, a comparison that pairs the two configurations on pass^3 with McNemar's test and an exact binomial test, and a gate whose rules were written first. A "keep it to 5 enquiries" prompt ties the baseline on the headline and is blocked for silently dropping cost enquiries; a harmless rewording passes.
Back to the exercise