Module 1 · Decide and prepare · Lesson 2
Training data from traces
Turning agent trajectories, grader verdicts and eval failures into a supervised fine-tuning set, with chat templates and loss masks, rejection sampling and corrected failures, open-weight teachers whose terms allow it, balance by failure mode, and a leakage audit against the frozen test split.
By the end of this lesson you can
- Convert a multi-turn tool-calling trajectory into a conversational training row, and verify which tokens carry loss under the student's own chat template
- Filter trajectories with code graders (rejection sampling, as in STaR and ReST-EM), and decide which failures to correct, drop or keep as negatives
- Generate synthetic inputs and teacher trajectories with an open-weight model whose licence and host terms allow training on its outputs, recording provenance on every example
- Rebalance a filtered set by failure mode, and size it for quality rather than volume
- Prove that a training set is disjoint from the frozen test split with exact and near-duplicate checks, and test the check itself on planted leaks