Fine-Tuning and RL for Agents
Train a small open model for one agent step with supervised fine-tuning, preference optimisation and reinforcement learning, and prove with your eval suite whether it beats an optimised prompt.
3 modules · 8 lessons~38 hoursUpdated 2026-11-01
- Lessons8
- Quizzes8
- Sources54
- Of study~38h
About this course
About this course
Once prompts and optimisers stop paying, the next lever is the model itself. This course is the practical side of post-training for engineers: deciding whether an agent step should be trained at all, turning traces and eval failures into training data without leaking your test set, QLoRA fine-tuning and preference optimisation of a small open-weight model on a free GPU, and reinforcement learning with verifiable and rubric rewards in a multi-turn tool environment. It ends where it should: catching reward hacking and regressions, and comparing the trained model with the optimised prompt on cost and quality before a ship or no-ship call. Claude Code writes, launches and monitors the training code on your subscription, while training runs on free Kaggle or Colab GPUs.
About the running example
The examples follow Quillmoor, a fictional company that sells project-management software, and the triage step of its support agent. For each incoming ticket the step looks up the account and searches the help centre through stub tools, then returns a category, a priority, the account id, whether a human must take over, a short reason and a reply outline. Most fields can be checked by code, and the free-text ones need a rubric judge. Every ticket is synthetic, and training data comes from open-weight models whose terms allow it, never from Claude's outputs.
What you'll learn.
- 1Decide, with a written cost and quality case, whether an agent step should be prompted, prompt-optimised or trained, and name what fine-tuning can't fix
- 2Build supervised, preference and RL datasets from agent traces and eval failures without leaking the frozen test split or training on outputs a provider's terms forbid
- 3Fine-tune a small open-weight model with QLoRA and with DPO or KTO on a free GPU, launched and monitored from Claude Code, and read its training curves
- 4Turn graders and a calibrated judge into verifiable and rubric rewards, train with GRPO-family RL in a multi-turn tool environment, and diagnose stalled groups, length bias and instability
- 5Detect reward hacking and capability or safety regressions with held-out graders and trace review
- 6Compare the trained model with the optimised-prompt baseline on cost per completed task and paired quality with negative flips, and make a ship or no-ship call
3 modules, 8 lessons.
10h 55m of lessons
Module 1 · 2 lessons · 2h 35m
Decide and prepare
Whether training is the right lever, what it costs, and how to turn traces and eval failures into training data without fooling yourself.
Module 2 · 2 lessons · 2h 45m
Supervised and preference fine-tuning
Get a small open model to do the step at all with supervised fine-tuning, then shape its judgement with preference data.
Module 3 · 4 lessons · 5h 35m
Reinforcement learning for agents
Graders become rewards, tool stubs become an environment, and the eval suite decides whether RL helped or just learned to cheat.
Train a triage model, then decide whether to ship it
Take one agent step from an optimised-prompt baseline to a trained small open model and back to a decision. Measure the baseline and fix the bar, build leak-free training data from an open-weight teacher, fine-tune with QLoRA, train with GRPO in a multi-turn tool environment against a priced reward, hunt the hack you plant, and compare paired, with pass^k, intervals, negative flips and cost per completed case, before writing a ship or no-ship memo.
More in Evaluation and Optimisation.
- IntermediateAI agents · Evals
Evaluating and Scoring Agents
Turn "it seems better" into numbers you can defend: error analysis, code graders, calibrated LLM judges and honest statistics.
7 lessons ~20hFree - AdvancedAI agents · Evals
Agent Benchmarks and Simulations
Evaluate multi-turn, stateful agents with simulated users and real environments, and build benchmarks whose numbers survive an audit.
9 lessons ~32hFree - AdvancedPrompt engineering · DSPy and GEPA
Prompt and Context Optimisation
Treat prompts, examples, tool descriptions and skills as versioned, measured artefacts, and improve them by hand, by optimiser and in production without breaking what already works.
7 lessons ~17hFree - AdvancedEvals · Human feedback
Human Feedback at Scale
Collect, measure and trust the judgements of reviewers and users: study design, rater agreement, preference rankings, real-use signals and quality estimates from a few labels.
7 lessons ~28hFree