Building user simulators

The tickets in the last lesson were the easy mode. Each one held every fact the agent needed, written down in order, by someone who knew exactly what they wanted. Real customers open with "charged twice??", forget which workspace they mean, answer half a question, push back on a policy, and sometimes have to do part of the work themselves, such as reading a code from their own inbox. Agents get measurably worse in that setting: across more than 200,000 simulated conversations, Laban et al. (2025) found a 39% average drop from single-turn to multi-turn, and observed that "when LLMs take a wrong turn in a conversation, they get lost and do not recover". You can't recruit a person for every trial, so benchmarks like τ-bench put a second model in the loop to play the user. This lesson builds that simulator for Quillmoor's account-support agent.