Coders In Hoods

Courses
Answers
Setup
Bookmarks

Coders In Hoods Academy

Every course is free. Sign in to start, and your progress, quiz scores and bookmarks follow you.

Academy

Courses
Paths
Exercise answers
Setup guides
Bookmarks

Coders In Hoods

The agency
Coaching for teams

Social

LinkedIn
X (Twitter)

Contact

codersinhoods@gmail.com

© Coders In Hoods · since 2019

Free · no card

Coders In Hoods Academy

Sign in to start learning

Free, no card. We use your account to save progress, quiz scores, bookmarks and occasionally send updates from our team.

Course overview

LLM Cost Engineering

Free · sign in to start and track progress

1 · Spending less on the same work

1Token economics of an agent fleet
2Prompt caching as system design
3Trimming and compacting context
4Batch, flex and fast tiers

2 · Changing the model, and governing the spend

5Cost-quality curves, routing and cascades
6Distilling into small models
Exercise answers
- Answer: Distil a clause tagger and measure the gap
7Self-hosting break-even
8Attribution, budgets and alerts

CapstoneCost-engineer an agent pipeline Reference implementation

Answers Lesson 6: Distilling into small models

Step-by-step answer

Core 2–3 hours8 steps

Distil a clause tagger and measure the gap

600 synthetic lease clauses labelled by claude -p in batches of 20 (Sonnet as teacher, Haiku as a second pass), filtered to the labels both agree on, and distilled into two LoRA students on Qwen3-0.6B, with and without the teacher's rationales as a second task. Everyone is scored against a hand-labelled gold subset, and the break-even volume is computed against both the teacher and a small hosted model.

Back to the exercise

Before you start

Before you start:Install Claude Code Python with uv Run GPU code on Modal Run Hugging Face models on CPU

The pipeline is teacher labels → filter → student → evaluation against human gold. Each stage writes a file, so you can stop, inspect and resume. Two decisions matter most. The gold subset is labelled by you and is never trained on, so the student is measured on correctness, not on how well it imitates the teacher. And the break-even is computed against the cheapest model that passes, not against the teacher.

Which outputs are real

The student training and evaluation below are real runs: both LoRA students were trained on Apple-silicon MPS and scored on the gold subset. Their inputs were not real model output, though. The teacher and second-pass labels came from fake/claude, which returns the generator's intended category with seeded, model-specific errors, and the gold labels came from gold.py --auto, which copies the generator's intended labels instead of yours. On your run, both come from Claude and from you.

Previous answerDoes a learned router beat self-confidence?Next answer Build a serving cost calculator

On this page

Before you start
The exercise
1. Set up the project
2. Generate the clauses
3. Label with the teacher
4. Hand-label the gold subset
5. Filter and split
6. Train the two students
7. Run it
8. Write up the deliverable
Common mistakes
All files

The pipeline is teacher labels → filter → student → evaluation against human gold. Each stage writes a file, so you can stop, inspect and resume. Two decisions matter most. The gold subset is labelled by you and is never trained on, so the student is measured on correctness, not on how well it imitates the teacher. And the break-even is computed against the cheapest model that passes, not against the teacher.

Which outputs are real

The student training and evaluation below are real runs: both LoRA students were trained on Apple-silicon MPS and scored on the gold subset. Their inputs were not real model output, though. The teacher and second-pass labels came from fake/claude, which returns the generator's intended category with seeded, model-specific errors, and the gold labels came from gold.py --auto, which copies the generator's intended labels instead of yours. On your run, both come from Claude and from you.

Free · sign in to start

Sign in to keep reading the answer.

Free, no card. We use your account to save progress, quiz scores, bookmarks and occasionally send updates from our team.

8 more sections
Progress, quiz scores and bookmarks