Coders In Hoods

Courses
Answers
Setup
Bookmarks

Coders In Hoods Academy

Every course is free. Sign in to start, and your progress, quiz scores and bookmarks follow you.

Academy

Courses
Paths
Exercise answers
Setup guides
Bookmarks

Coders In Hoods

The agency
Coaching for teams

Social

LinkedIn
X (Twitter)

Contact

codersinhoods@gmail.com

© Coders In Hoods · since 2019

Free · no card

Coders In Hoods Academy

Sign in to start learning

Free, no card. We use your account to save progress, quiz scores, bookmarks and occasionally send updates from our team.

Course overview

Scaling AI Systems

Free · sign in to start and track progress

1 · Throughput under limits

1Rate limits and quotas
2Concurrency and flow control
3Retries, backoff and overload
4Backpressure, load shedding and batching

2 · Latency, resilience and capacity

5Latency engineering for agents
6Response caching and failover
7Load testing and capacity planning

CapstoneMonth-end readiness for an agent fleet Reference implementation

Capstone project

Scaling AI Systems · after 7 lessons

Month-end readiness for an agent fleet

Build and prove the dispatch layer for three deadline-day agents serving five synthetic customers, against a mock Claude API on two platforms. A token-aware fair limiter, a retry policy with budgets and breakers, a priority admission queue with a batch path, latency and caching work, warm-standby failover, and a month-end load test that ends in a capacity plan.

Reference implementation

The completion-day agents are where scaling problems become client problems. The completion statement reconciliation has to finish before funds move, the post-completion letters have to go out the same day, and the stamp duty return preparation has fourteen days but arrives in the same burst. In this project you build the layer between those agents and the model, the part this course has been about, and prove under a replayed last-Friday-of-the-month load that urgent work makes its deadlines while the provider throttles, overloads and, for one minute, falls over.

Data and costs

Use synthetic data only: invented firms, matters, title numbers and figures. Everything runs locally with free tools: FastAPI mocks of the Claude Messages and Batches APIs (one "primary" and one "fallback" platform), Redis in Docker, Locust and Python. The agents' model calls go to the mocks; where you want a real model for a sanity check, use claude -p on your subscription. Calibrating the mock against a real model is an optional stretch: use claude -p timings, or your own OpenRouter key with a small credit limit.

On this page

What you'll build
Milestones
The reference to compare against
Working with Claude Code
Deliverables

The completion-day agents are where scaling problems become client problems. The completion statement reconciliation has to finish before funds move, the post-completion letters have to go out the same day, and the stamp duty return preparation has fourteen days but arrives in the same burst. In this project you build the layer between those agents and the model, the part this course has been about, and prove under a replayed last-Friday-of-the-month load that urgent work makes its deadlines while the provider throttles, overloads and, for one minute, falls over.

Data and costs

Use synthetic data only: invented firms, matters, title numbers and figures. Everything runs locally with free tools: FastAPI mocks of the Claude Messages and Batches APIs (one "primary" and one "fallback" platform), Redis in Docker, Locust and Python. The agents' model calls go to the mocks; where you want a real model for a sanity check, use claude -p on your subscription. Calibrating the mock against a real model is an optional stretch: use claude -p timings, or your own OpenRouter key with a small credit limit.

Free · sign in to start

Sign in to keep reading the brief.

Free, no card. We use your account to save progress, quiz scores, bookmarks and occasionally send updates from our team.

5 more sections
Progress, quiz scores and bookmarks