Step-by-step answer
Core 90 min6 stepsRun an example ablation on the enquiries drafter
A pool of 60 synthetic past enquiries, a separate eval set of 15 contract-pack excerpts with gold issue types, a TF-IDF and MMR selector with a per-type cap, and five arms (zero-shot, static 3 in two orders, retrieved 3, many-shot 30) run twice per pack through claude -p. The scorer adds an anchoring rate, the share of enquiries about a topic the examples showed but the pack doesn't need, which is what separates examples that teach format from examples that teach topics.
Back to the exercise