Coders In Hoods

Courses
Answers
Setup
Bookmarks

Coders In Hoods Academy

Every course is free. Sign in to start, and your progress, quiz scores and bookmarks follow you.

Academy

Courses
Paths
Exercise answers
Setup guides
Bookmarks

Coders In Hoods

The agency
Coaching for teams

Social

LinkedIn
X (Twitter)

Contact

codersinhoods@gmail.com

© Coders In Hoods · since 2019

Free · no card

Coders In Hoods Academy

Sign in to start learning

Free, no card. We use your account to save progress, quiz scores, bookmarks and occasionally send updates from our team.

Course overview

Infrastructure for Agents

Free · sign in to start and track progress

1 · Compute and execution

1Where agents run
2Sandboxes for agent code
3Durable execution engines
4Queues, schedules and background jobs

2 · State, access and delivery

5State, sessions and checkpoints
6Secrets, identity and tenant isolation
7AI gateways
Exercise answers
- Answer: Simulate fallbacks, correlation and budgets
- Answer: A self-hosted gateway with per-firm keys
8Environments and delivery for agents

CapstoneA production platform slice for one agent Reference implementation

Module 2 · State, access and delivery · Lesson 7

AI gateways

One control point between your agents and every model provider. What Vercel AI Gateway, OpenRouter, LiteLLM and Portkey do (routing, fallbacks, keys, budgets, logging, data controls), the arithmetic of fallbacks, and the ways a gateway can mislead you.

70 min 1 video 3 exercises 7-question quiz Answers

Setup for the exercises:Install Claude Code Python with uv Docker and Docker Compose Postgres with pgvector

By the end of this lesson you can

Describe the functions of an AI gateway and decide which ones your fleet needs
Compare Vercel AI Gateway, OpenRouter, LiteLLM and Portkey on hosting, pricing, routing, keys and budgets
Compute the availability gain from fallbacks, and explain why correlated failures and changed behaviour reduce it
Configure per-tenant keys, budgets and model fallbacks from Python, and handle budget rejections in an agent
Explain the data-handling implications of adding a gateway to the request path

The fleet makes thousands of model calls a day: Claude for the searches review and the completion statement, a cheaper model for classifying inbound email, an open-weight model on Fireworks for bulk extraction. Each agent was built by a different engineer with its own provider key in an environment variable. One afternoon a provider has an incident and three agents stop. At month end nobody can say which firm spent what. A security review asks where prompts containing client data go, and the answer is "several places, depending on the agent". An AI gateway fixes all three by putting one control point between your agents and every provider.

On this page

Intuition: the firm's outgoing post room
What a gateway does
The four products
Fallbacks, and their arithmetic
Keys and budgets per tenant
Code: per-firm key, fallbacks and budget handling
Routing for cost
Data handling
Misconceptions and limits
Connections
Summary
Sources & further reading
Quiz

The fleet makes thousands of model calls a day: Claude for the searches review and the completion statement, a cheaper model for classifying inbound email, an open-weight model on Fireworks for bulk extraction. Each agent was built by a different engineer with its own provider key in an environment variable. One afternoon a provider has an incident and three agents stop. At month end nobody can say which firm spent what. A security review asks where prompts containing client data go, and the answer is "several places, depending on the agent". An AI gateway fixes all three by putting one control point between your agents and every provider.

Free · sign in to start

Sign in to keep reading.

Free, no card. We use your account to save progress, quiz scores, bookmarks and occasionally send updates from our team.

10 more sections
3 hands-on exercises
A 7-question quiz
Progress, quiz scores and bookmarks