LLM Cost Engineering
Know what each case costs in tokens, then cut that cost with caching, trimming, batching, routing, distillation and self-hosting, without losing quality.
2 modules · 8 lessons~20 hoursUpdated 2026-10-30
- Lessons8
- Quizzes8
- Sources49
- Of study~20h
About this course
About this course
An advanced course on the economics of an agent fleet. You build a per-step cost model of one case, then work through the levers in the order they usually pay: prompt caching designed at the system level, context trimming and compaction, batch and service tiers, model routing and cascades measured against a cost-quality curve, distillation into small open-weight models, and the break-even point for self-hosting on rented GPUs. The course ends with attribution: a per-customer, per-case cost ledger with budgets and alerts. Every lesson recomputes real prices and token counts in Python, cites the papers and docs behind each claim, and gives you exercises to run with Claude.
About the running example
The examples follow a fictional property-law (conveyancing) firm in England and Wales: what one property transaction costs, step by step, across a fleet of agents. It's a case study, chosen because its documents are long, structured and high-stakes. You don't need a legal background, and the same patterns apply to contracts, insurance claims, invoices, support tickets or patient records. In the lessons, a matter is one client's case and a solicitor is the lawyer who signs work off.
What you'll learn.
- 1Build a per-step token and cost model of an agent pipeline, and compute cost per completed case including thinking tokens, retries and failures
- 2Design prompt layouts, cache lifetimes and scheduling so that cache hit rates stay high across a fleet, and decide when trimming or compaction lowers cost
- 3Move the right workloads to batch and service tiers, and price the trade-off in latency and data handling
- 4Measure a cost-quality curve with evals and decide whether a router, a cascade, a distilled model or a single tuned model sits on its frontier
- 5Compute the break-even point for self-hosting an open-weight model on rented H100 or B200 GPUs against serverless and batch prices
- 6Attribute every token to a customer, case, agent and step, reconcile it with the invoice, and enforce budgets and alerts
2 modules, 8 lessons.
9h 40m of lessons
Module 1 · 4 lessons · 4h 40m
Spending less on the same work
Model what a case costs, then apply the levers that leave the model choice unchanged: caching, context discipline and cheaper service tiers.
Module 2 · 4 lessons · 5h
Changing the model, and governing the spend
Measure the cost-quality frontier, then decide whether routing, distillation or self-hosting moves you along it, and put every dollar under attribution and budgets.
Cost-engineer an agent pipeline
Take a simulated agent fleet from an uncached baseline to a measured, attributed and budgeted pipeline, applying every lever in the course and proving each saving with numbers and evals.
More in Production AI at Scale.
- AdvancedAI agents · Infrastructure
Infrastructure for Agents
Run a fleet of agents in production: compute, sandboxes, durable workflows, queues, state, tenant isolation, gateways and safe delivery.
8 lessons ~16hFree - AdvancedAI agents · Security
Production Agent Systems
Take an agent from working demo to a system you can run for months: architecture, security, compliance, observability and safe change.
6 lessons ~18hFree - AdvancedAI · Reliability
Scaling AI Systems
Keep a fleet of agents fast and available under real load: rate limits, flow control, retries, backpressure, caching, failover, latency budgets, load tests and capacity plans.
7 lessons ~18hFree