Module 2 · Optimising and operating prompts · Lesson 4
Automatic prompt optimisation
How optimisers search over prompts and examples - from APE and OPRO to DSPy's MIPROv2 and GEPA's reflective, Pareto-based evolution - what they genuinely buy you, what a run costs, and how they overfit, with a DSPy version of the report summariser.
Setup for the exercises:Install Claude CodePython with uvGet an OpenRouter keyLocal models with Ollama
By the end of this lesson you can
- Describe the shared skeleton of prompt optimisers (candidates, proposer, metric, search) and place APE, ProTeGi, OPRO, TextGrad, MIPROv2 and GEPA on it
- Write a DSPy signature, module and feedback metric for a workflow step, and choose between BootstrapFewShot, MIPROv2 and GEPA
- Explain how GEPA uses reflection on traces and a Pareto front of candidates, and why that is more sample-efficient than scalar-reward search
- Estimate the cost of an optimisation run from call counts and prices, and identify which component dominates it
- Judge an optimised prompt against a held-out set and decide whether to adopt, adapt or reject it