← Academy

Step-by-step answers.

What to set up, the complete code built one step at a time, and why each step is there. Try the exercise first: the answer is most useful once you've made your own attempt.

178 exercises · 13 capstones · 13 courses

Answers open after sign-in

191 of 191

ExerciseLinks
Implement and stress-test a byte-level BPE tokenizerLesson 1 · Tokens and embeddingsLLMs for EngineersCore· 60–90 minSetup: Install Claude Code, Python with uv Answer
Probe embedding geometryLesson 1 · Tokens and embeddingsLLMs for EngineersStretch· 45 minSetup: Install Claude Code, Python with uv Answer
Implement attention and verify it against PyTorchLesson 2 · Attention, step by stepLLMs for EngineersCore· 60 minSetup: Install Claude Code, Python with uv, Run Hugging Face models on CPU Answer
Find an induction head in GPT-2Lesson 2 · Attention, step by stepLLMs for EngineersStretch· 90 minSetup: Install Claude Code, Python with uv Answer
Build a block and verify parameter counts against real modelsLesson 3 · The transformer blockLLMs for EngineersCore· 60–90 minSetup: Install Claude Code, Python with uv, Run Hugging Face models on CPU Answer
Prove RoPE is relative, numericallyLesson 3 · The transformer blockLLMs for EngineersStretch· 45 minSetup: Install Claude Code, Python with uv Answer
Fit your own scaling law on a laptopLesson 4 · Pretraining and scaling lawsLLMs for EngineersCore· 60–90 minSetup: Install Claude Code, Python with uv, Run Hugging Face models on CPU Answer
Base model versus assistant on conveyancing promptsLesson 4 · Pretraining and scaling lawsLLMs for EngineersStretch· 45–60 minSetup: Install Claude Code, Python with uv, Run Hugging Face models on CPU Answer
Build a verifiable reward and simulate GRPO advantagesLesson 5 · Post-training an assistantLLMs for EngineersCore· 60 minSetup: Install Claude Code, Python with uv Answer
Measure and catch hallucinations in lease extractionLesson 6 · Sampling and hallucinationLLMs for EngineersCore· 75–90 minSetup: Install Claude Code, Python with uv Answer
Simulate decoding strategiesLesson 6 · Sampling and hallucinationLLMs for EngineersStretch· 45 minSetup: Install Claude Code, Python with uv, Run Hugging Face models on CPU Answer
Watch prompt caching hit and missLesson 7 · Context windows, KV cache and cachingLLMs for EngineersCore· 45–60 minSetup: Install Claude Code, Python with uv Answer
Implement a KV cache and measure the speed-upLesson 7 · Context windows, KV cache and cachingLLMs for EngineersStretch· 60 minSetup: Install Claude Code, Python with uv, Run Hugging Face models on CPU Answer
Run an effort sweep on conveyancing calculationsLesson 8 · Reasoning models and extended thinkingLLMs for EngineersCore· 60–90 minSetup: Install Claude Code, Python with uv Answer
Self-consistency versus more thinkingLesson 8 · Reasoning models and extended thinkingLLMs for EngineersStretch· 45–60 minSetup: Install Claude Code, Python with uv Answer
Run a model bake-off on title-register extractionLesson 9 · Choosing and comparing modelsLLMs for EngineersCore· 90 minSetup: Install Claude Code, Python with uv Answer
Add an open-weight model to the bake-offLesson 9 · Choosing and comparing modelsLLMs for EngineersStretch· 60 minSetup: Install Claude Code, Python with uv, Get an OpenRouter key, OpenRouter or a local model?, Local models with Ollama Answer
Choose a model for document extractionCapstone projectLLMs for EngineersCapstone Answer
Watch the agent loop in Claude CodeLesson 1 · What is an agent?Agent FoundationsCore· 45 minSetup: Install Claude Code, Node.js, pnpm and TypeScript Answer
Run the same task through the Agent SDKLesson 1 · What is an agent?Agent FoundationsStretch· 30 minSetup: Install Claude Code, Node.js, pnpm and TypeScript, Get an OpenRouter key, OpenRouter or a local model? Answer
Build a register tool and give it to Claude CodeLesson 2 · Tools and the agent-computer interfaceAgent FoundationsCore· 75 minSetup: Install Claude Code, Node.js, pnpm and TypeScript Answer
Turn the tool into an MCP serverLesson 2 · Tools and the agent-computer interfaceAgent FoundationsStretch· 60 minSetup: Install Claude Code, Node.js, pnpm and TypeScript, MCP servers in Claude Code Answer
Measure context and move procedure into a skillLesson 3 · Context engineeringAgent FoundationsCore· 75 minSetup: Install Claude Code, Node.js, pnpm and TypeScript Answer
Isolate lease reading in a subagentLesson 3 · Context engineeringAgent FoundationsStretch· 45 minSetup: Install Claude Code, Node.js, pnpm and TypeScript Answer
Produce and verify grounded title reportsLesson 4 · Structured outputs and groundingAgent FoundationsCore· 90 minSetup: Install Claude Code, Node.js, pnpm and TypeScript Answer
Same report through the Agent SDKLesson 4 · Structured outputs and groundingAgent FoundationsStretch· 40 minSetup: Install Claude Code, Node.js, pnpm and TypeScript, Get an OpenRouter key, OpenRouter or a local model?, Local models with Ollama Answer
Build a chain with a code gate, then add routingLesson 5 · Workflow patternsAgent FoundationsCore· 75 minSetup: Install Claude Code, Node.js, pnpm and TypeScript Answer
Orchestrator-workers and an evaluator loop with subagentsLesson 5 · Workflow patternsAgent FoundationsStretch· 60–90 minSetup: Install Claude Code, Node.js, pnpm and TypeScript Answer
Build and run a 10-case evalLesson 6 · Risk and first evalsAgent FoundationsCore· 90 minSetup: Install Claude Code, Node.js, pnpm and TypeScript Answer
Block outbound actions with a hookLesson 6 · Risk and first evalsAgent FoundationsStretch· 45 minSetup: Install Claude Code, Node.js, pnpm and TypeScript Answer
A document review agent in Claude CodeCapstone projectAgent FoundationsCapstone Answer
One task, two modes of the same harnessLesson 1 · Choosing a harnessBuilding Agents with Claude and eveCore· 45 minSetup: Install Claude Code, Node.js, pnpm and TypeScript Answer
Guardrails in Claude Code: tool, permissions, hooksLesson 2 · The Claude Agent SDKBuilding Agents with Claude and eveCore· 60–90 minSetup: Install Claude Code, Node.js, pnpm and TypeScript, MCP servers in Claude Code Answer
Run the title-review agent on the SDKLesson 2 · The Claude Agent SDKBuilding Agents with Claude and eveStretch· 60 minSetup: Install Claude Code, Node.js, pnpm and TypeScript, Get an OpenRouter key, Local models with Ollama, OpenRouter or a local model? Answer
Build and break a contract-pack reader in eveLesson 3 · Agents as files with eveBuilding Agents with Claude and eveCore· 60–90 minSetup: Install Claude Code, Durable agents with eve, Get an OpenRouter key, Local models with Ollama, OpenRouter or a local model? Answer
Make a side-effecting tool safe to replayLesson 3 · Agents as files with eveBuilding Agents with Claude and eveStretch· 45 minSetup: Install Claude Code, Durable agents with eve Answer
An enquiries skill and a search-reviewer subagent in Claude CodeLesson 4 · Skills and subagentsBuilding Agents with Claude and eveCore· 60–90 minSetup: Install Claude Code, Node.js, pnpm and TypeScript Answer
Port the skill and subagent to eveLesson 4 · Skills and subagentsBuilding Agents with Claude and eveStretch· 60 minSetup: Install Claude Code, Node.js, pnpm and TypeScript, Durable agents with eve, Get an OpenRouter key, Local models with Ollama Answer
A solicitor-approved send in eveLesson 5 · Human approval for high-stakes actionsBuilding Agents with Claude and eveCore· 60–90 minSetup: Install Claude Code, Node.js, pnpm and TypeScript, Durable agents with eve, Get an OpenRouter key, Local models with Ollama Answer
The same gate in the Agent SDKLesson 5 · Human approval for high-stakes actionsBuilding Agents with Claude and eveStretch· 60 minSetup: Install Claude Code, Node.js, pnpm and TypeScript, Get an OpenRouter key, Local models with Ollama Answer
Route auth and tenant isolation, checkedLesson 6 · Shipping agentsBuilding Agents with Claude and eveCore· 60–90 minSetup: Install Claude Code, Node.js, pnpm and TypeScript, Durable agents with eve, Get an OpenRouter key, Local models with Ollama Answer
Deploy to Vercel with a deployable modelLesson 6 · Shipping agentsBuilding Agents with Claude and eveStretch· 45–60 minSetup: Install Claude Code, Node.js, pnpm and TypeScript, Durable agents with eve, Get an OpenRouter key, Deploy to Vercel Answer
A drafting agent with human approval in eveCapstone projectBuilding Agents with Claude and eveCapstone Answer
Write an eval charter for one agentLesson 1 · Why evals, and their vocabularyEvaluating and Scoring AgentsCore· 45–60 minSetup: Install Claude Code, Node.js, pnpm and TypeScript Answer
Run your first repeated trials in Claude CodeLesson 1 · Why evals, and their vocabularyEvaluating and Scoring AgentsStretch· 60 minSetup: Install Claude Code, Node.js, pnpm and TypeScript Answer
Build a trace viewer and a counted taxonomyLesson 2 · Error analysis from tracesEvaluating and Scoring AgentsCore· 75–90 minSetup: Install Claude Code, Node.js, pnpm and TypeScript Answer
Generate a balanced 24-task datasetLesson 2 · Error analysis from tracesEvaluating and Scoring AgentsStretch· 60–90 minSetup: Install Claude Code, Node.js, pnpm and TypeScript Answer
Write five eve evals for the enquiries agentLesson 3 · Code-based gradersEvaluating and Scoring AgentsCore· 90 minSetup: Install Claude Code, Durable agents with eve, Get an OpenRouter key, Local models with Ollama Answer
Mutation-test your gradersLesson 3 · Code-based gradersEvaluating and Scoring AgentsStretch· 60 minSetup: Install Claude Code, Node.js, pnpm and TypeScript Answer
Build a Claude Code judge and measure its position biasLesson 4 · LLM-as-judgeEvaluating and Scoring AgentsCore· 75–90 minSetup: Install Claude Code, Node.js, pnpm and TypeScript Answer
Wire a judge into your eve evalsLesson 4 · LLM-as-judgeEvaluating and Scoring AgentsStretch· 60 minSetup: Durable agents with eve, Get an OpenRouter key, Local models with Ollama, Install Claude Code Answer
Calibrate your Claude Code judge against your own labelsLesson 5 · Calibrating judgesEvaluating and Scoring AgentsCore· 90–120 minSetup: Install Claude Code, Node.js, pnpm and TypeScript Answer
Pick a t.judge threshold and correct a reported rateLesson 5 · Calibrating judgesEvaluating and Scoring AgentsStretch· 60 minSetup: Durable agents with eve, Node.js, pnpm and TypeScript, Install Claude Code Answer
Measure flakiness and compare two prompt versionsLesson 6 · Statistics, trials and CI gatesEvaluating and Scoring AgentsCore· 90–120 minSetup: Durable agents with eve, Get an OpenRouter key, Local models with Ollama Answer
Wire a two-tier eval gate into CILesson 6 · Statistics, trials and CI gatesEvaluating and Scoring AgentsStretch· 60–90 minSetup: Durable agents with eve, Get an OpenRouter key, Run evals in GitHub Actions Answer
A full eval suite for your drafting agentCapstone projectEvaluating and Scoring AgentsCapstone Answer
Rewrite the searches summariser and measure itLesson 1 · Prompting current Claude modelsPrompt and Context OptimisationCore· 75 minSetup: Install Claude Code, Python with uv Answer
Effort versus prose instructionsLesson 1 · Prompting current Claude modelsPrompt and Context OptimisationStretch· 45 minSetup: Install Claude Code, Python with uv Answer
Run an example ablation on the enquiries drafterLesson 2 · Choosing few-shot examplesPrompt and Context OptimisationCore· 90 minSetup: Install Claude Code, Python with uv Answer
Three disciplined iterations on the enquiries drafterLesson 3 · Eval-driven prompt iterationPrompt and Context OptimisationCore· 90 minSetup: Install Claude Code, Python with uv Answer
Measure prompt fragilityLesson 3 · Eval-driven prompt iterationPrompt and Context OptimisationStretch· 45 minSetup: Install Claude Code, Python with uv Answer
Build a small reflective optimiser on claude -pLesson 4 · Automatic prompt optimisationPrompt and Context OptimisationCore· 2–3 hoursSetup: Install Claude Code, Python with uv Answer
DSPy: MIPROv2 versus GEPALesson 4 · Automatic prompt optimisationPrompt and Context OptimisationStretch· 2 hoursSetup: Install Claude Code, Python with uv, Get an OpenRouter key, Local models with Ollama Answer
Build a skill-trigger eval in Claude CodeLesson 5 · Optimising tools, instructions and skillsPrompt and Context OptimisationCore· 90 minSetup: Install Claude Code, Python with uv Answer
Transcript-driven tool improvementLesson 5 · Optimising tools, instructions and skillsPrompt and Context OptimisationStretch· 2 hoursSetup: Install Claude Code, Python with uv, MCP servers in Claude Code Answer
Build and break an experiment analysisLesson 6 · Prompt registries and online testsPrompt and Context OptimisationCore· 75 minSetup: Install Claude Code, Python with uv Answer
Wire a registry with labels and trace linkageLesson 6 · Prompt registries and online testsPrompt and Context OptimisationStretch· 90 minSetup: Install Claude Code, Python with uv, Docker and Docker Compose, Trace LLM calls with OpenTelemetry and Langfuse Answer
Migrate the summariser behind a regression suiteLesson 7 · Model migration and regression suitesPrompt and Context OptimisationCore· 2 hoursSetup: Install Claude Code, Python with uv Answer
Gate prompt changes in CILesson 7 · Model migration and regression suitesPrompt and Context OptimisationStretch· 60 minSetup: Install Claude Code, Python with uv, Get an OpenRouter key, Local models with Ollama, Run evals in GitHub Actions Answer
Optimise and ship the questions drafterCapstone projectPrompt and Context OptimisationCapstone Answer
Build and tune a page triageLesson 1 · What a real case file containsDocument Ingestion PipelinesCore· 60–90 minSetup: Install Claude Code, Python with uv Answer
Unpack and deduplicate a matter's emailLesson 1 · What a real case file containsDocument Ingestion PipelinesStretch· 45–60 minSetup: Install Claude Code, Python with uv Answer
Run a four-way parser bake-offLesson 2 · Parsing: text layers to vision modelsDocument Ingestion PipelinesCore· 90 minSetup: Install Claude Code, Python with uv, OCR and PDF parsing: Tesseract and Docling Answer
Self-host an open parsing modelLesson 2 · Parsing: text layers to vision modelsDocument Ingestion PipelinesStretch· 90 minSetup: Install Claude Code, Python with uv, Run GPU code on Modal, Serve a model with vLLM, Get an OpenRouter key, Local models with Ollama Answer
Verify tables and plan references in codeLesson 3 · Tables, plans and handwritingDocument Ingestion PipelinesCore· 75 minSetup: Install Claude Code, Python with uv Answer
Measure two-reader agreement on handwritingLesson 3 · Tables, plans and handwritingDocument Ingestion PipelinesStretch· 60–90 minSetup: Install Claude Code, Python with uv, OCR and PDF parsing: Tesseract and Docling Answer
Extract lease terms with page-verified evidenceLesson 4 · Structured extraction at scaleDocument Ingestion PipelinesCore· 90 minSetup: Install Claude Code, Python with uv Answer
Compare long-document strategiesLesson 4 · Structured extraction at scaleDocument Ingestion PipelinesStretch· 60 minSetup: Install Claude Code, Python with uv Answer
Build a crash-safe ingest workflowLesson 5 · Durable ingest pipelinesDocument Ingestion PipelinesCore· 90–120 minSetup: Install Claude Code, Python with uv, Run Temporal locally Answer
Simulate retry granularityLesson 5 · Durable ingest pipelinesDocument Ingestion PipelinesStretch· 30–45 minSetup: Install Claude Code, Python with uv Answer
Build incremental re-ingestion with early cutoffLesson 6 · Re-ingestion, versioning and backfillsDocument Ingestion PipelinesCore· 90 minSetup: Install Claude Code, Python with uv Answer
Build and measure a hybrid PII detectorLesson 7 · PII detection and redactionDocument Ingestion PipelinesCore· 90–120 minSetup: Install Claude Code, Python with uv Answer
Redact a PDF and prove itLesson 7 · PII detection and redactionDocument Ingestion PipelinesStretch· 30–45 minSetup: Install Claude Code, Python with uv Answer
Build an ingest eval harnessLesson 8 · Ingest evals and the hand-offDocument Ingestion PipelinesCore· 90–120 minSetup: Install Claude Code, Python with uv Answer
Specify and test the retrieval hand-offLesson 8 · Ingest evals and the hand-offDocument Ingestion PipelinesStretch· 45–60 minSetup: Install Claude Code, Python with uv Answer
Ingest a case file end to endCapstone projectDocument Ingestion PipelinesCapstone Answer
Measure truncation and quantisation on legal textLesson 1 · Embeddings in practiceRetrieval and Knowledge SystemsCore· 60–90 minSetup: Python with uv, Get an OpenRouter key, Local models with Ollama Answer
Add a commercial embedder (stretch)Lesson 1 · Embeddings in practiceRetrieval and Knowledge SystemsStretch· 45 minSetup: Python with uv, Get an OpenRouter key, Local models with Ollama Answer
Structure-aware and contextual chunking, measuredLesson 2 · Chunking structured documentsRetrieval and Knowledge SystemsCore· 90 minSetup: Python with uv, Install Claude Code, Get an OpenRouter key, Local models with Ollama Answer
Try late chunking (stretch)Lesson 2 · Chunking structured documentsRetrieval and Knowledge SystemsStretch· 60 minSetup: Python with uv, Run Hugging Face models on CPU Answer
Tune an ANN index against a recall targetLesson 3 · Vector indexes and databasesRetrieval and Knowledge SystemsCore· 60–90 minSetup: Python with uv, Get an OpenRouter key, Local models with Ollama Answer
Red-team tenant isolation (stretch)Lesson 3 · Vector indexes and databasesRetrieval and Knowledge SystemsStretch· 60 minSetup: Python with uv, Docker and Docker Compose, Postgres with pgvector Answer
Build and run a labelled query setLesson 4 · Measuring retrievalRetrieval and Knowledge SystemsCore· 90–120 minSetup: Python with uv, Install Claude Code, Get an OpenRouter key, Local models with Ollama Answer
Verify citations end to end (stretch)Lesson 4 · Measuring retrievalRetrieval and Knowledge SystemsStretch· 60 minSetup: Python with uv, Install Claude Code, Get an OpenRouter key, Local models with Ollama Answer
Build and measure a hybrid pipelineLesson 5 · Hybrid search and rerankingRetrieval and Knowledge SystemsCore· 90 minSetup: Install Claude Code, Python with uv, Get an OpenRouter key, Local models with Ollama Answer
Compare single-shot, grep-only and full agentic retrievalLesson 6 · Agentic retrievalRetrieval and Knowledge SystemsCore· 2 hoursSetup: Install Claude Code, Python with uv, Get an OpenRouter key, Local models with Ollama Answer
Build scoped matter tools with the Agent SDK (stretch)Lesson 6 · Agentic retrievalRetrieval and Knowledge SystemsStretch· 90 minSetup: Install Claude Code, Python with uv, Get an OpenRouter key, Local models with Ollama Answer
Build a graph from verified records, and compare with an LLM-extracted oneLesson 7 · Knowledge graphs and GraphRAGRetrieval and Knowledge SystemsCore· 2 hoursSetup: Install Claude Code, Python with uv Answer
Trial GraphRAG on a global question (stretch)Lesson 7 · Knowledge graphs and GraphRAGRetrieval and Knowledge SystemsStretch· 2–3 hoursSetup: Install Claude Code, Python with uv Answer
Compare long context, RAG and a router on your questionsLesson 8 · Long context versus RAGRetrieval and Knowledge SystemsCore· 2 hoursSetup: Install Claude Code, Python with uv, Get an OpenRouter key, Local models with Ollama Answer
A measured retrieval service for cases and reference documentsCapstone projectRetrieval and Knowledge SystemsCapstone Answer
Inventory the memory of an agent you useLesson 1 · Memory types for agentsAgent MemoryCore· 45–60 minSetup: Install Claude Code, Python with uv Answer
Measure fact drift across repeated compactionLesson 2 · The context window as working memoryAgent MemoryCore· 60–75 minSetup: Install Claude Code, Python with uv Answer
Re-inject matter state after compaction in Claude CodeLesson 2 · The context window as working memoryAgent MemoryStretch· 30–45 minSetup: Install Claude Code, Python with uv Answer
Build and attack a per-matter memory-tool handlerLesson 3 · Files, skills and memory toolsAgent MemoryCore· 60–90 minSetup: Install Claude Code, Python with uv, Get an OpenRouter key, Local models with Ollama Answer
Write a skill that survives compactionLesson 3 · Files, skills and memory toolsAgent MemoryStretch· 45 minSetup: Install Claude Code Answer
Build a miniature knowledge-update benchmarkLesson 4 · Memory frameworks and benchmarksAgent MemoryCore· 75–90 minSetup: Install Claude Code, Python with uv Answer
Do due diligence on a memory frameworkLesson 4 · Memory frameworks and benchmarksAgent MemoryStretch· 45–60 minSetup: Install Claude Code Answer
Build a scoped memory store with provenance-based deletionLesson 5 · Scoping, retention and erasureAgent MemoryCore· 75–90 minSetup: Install Claude Code, Python with uv Answer
Build the write-update-forget pipeline for a 12-week matterLesson 6 · Write, update and forgetAgent MemoryCore· 90 minSetup: Install Claude Code, Python with uv Answer
Design a reviewed consolidation jobLesson 6 · Write, update and forgetAgent MemoryStretch· 60 minSetup: Install Claude Code, Python with uv Answer
Build shared matter state and attack itLesson 7 · Shared state and memory poisoningAgent MemoryCore· 75–90 minSetup: Install Claude Code, Python with uv Answer
Red-team persistence across sessionsLesson 7 · Shared state and memory poisoningAgent MemoryStretch· 60 minSetup: Install Claude Code, Python with uv Answer
Run a memory ablation on synthetic mattersLesson 8 · Evaluating memoryAgent MemoryCore· 90–120 minSetup: Install Claude Code, Python with uv Answer
Model memory cost for your session patternLesson 8 · Evaluating memoryAgent MemoryStretch· 45 minSetup: Install Claude Code, Python with uv Answer
A memory layer for a 12-week caseCapstone projectAgent MemoryCapstone Answer
Build a fleet compute and token cost modelLesson 1 · Where agents runInfrastructure for AgentsCore· 60 minSetup: Install Claude Code, Python with uv Answer
Measure active CPU against wall timeLesson 1 · Where agents runInfrastructure for AgentsStretch· 90 minSetup: Install Claude Code, Python with uv, Deploy to Vercel Answer
Build and attack a local code runnerLesson 2 · Sandboxes for agent codeInfrastructure for AgentsCore· 75 minSetup: Install Claude Code, Python with uv, Docker and Docker Compose Answer
Egress allowlists and credential brokering in a microVMLesson 2 · Sandboxes for agent codeInfrastructure for AgentsStretch· 90 minSetup: Install Claude Code, Python with uv, Deploy to Vercel Answer
Build a toy durable execution engineLesson 3 · Durable execution enginesInfrastructure for AgentsCore· 90 minSetup: Install Claude Code, Python with uv Answer
A completion workflow on Temporal with signals and a deadlineLesson 3 · Durable execution enginesInfrastructure for AgentsStretch· 2 hoursSetup: Install Claude Code, Python with uv, Run Temporal locally Answer
Simulate leases, redelivery and fairnessLesson 4 · Queues, schedules and background jobsInfrastructure for AgentsCore· 75 minSetup: Install Claude Code, Python with uv Answer
An idempotent consumer against a local queueLesson 4 · Queues, schedules and background jobsInfrastructure for AgentsStretch· 60–90 minSetup: Install Claude Code, Python with uv, Deploy to Vercel Answer
Build and test a session store adapterLesson 5 · State, sessions and checkpointsInfrastructure for AgentsCore· 75 minSetup: Install Claude Code, Python with uv, Get an OpenRouter key, OpenRouter or a local model?, Local models with Ollama Answer
Find every copy: an erasure dry runLesson 5 · State, sessions and checkpointsInfrastructure for AgentsStretch· 90 minSetup: Install Claude Code, Python with uv Answer
Row-level security that fails closedLesson 6 · Secrets, identity and tenant isolationInfrastructure for AgentsCore· 75 minSetup: Install Claude Code, Python with uv, Docker and Docker Compose, Postgres with pgvector Answer
A per-firm credential proxyLesson 6 · Secrets, identity and tenant isolationInfrastructure for AgentsStretch· 90 minSetup: Install Claude Code, Python with uv Answer
Simulate fallbacks, correlation and budgetsLesson 7 · AI gatewaysInfrastructure for AgentsCore· 75 minSetup: Install Claude Code, Python with uv Answer
A self-hosted gateway with per-firm keysLesson 7 · AI gatewaysInfrastructure for AgentsStretch· 90 minSetup: Install Claude Code, Python with uv, Docker and Docker Compose, Postgres with pgvector Answer
Build a paired eval gate for CILesson 8 · Environments and delivery for agentsInfrastructure for AgentsCore· 90 minSetup: Install Claude Code, Python with uv, Run evals in GitHub Actions Answer
Roll out a prompt by matter, with shadow runsLesson 8 · Environments and delivery for agentsInfrastructure for AgentsStretch· 2 hoursSetup: Install Claude Code, Python with uv Answer
A production platform slice for one agentCapstone projectInfrastructure for AgentsCapstone Answer
Single agent versus lead plus subagents in Claude CodeLesson 1 · Multi-agent systemsProduction Agent SystemsCore· 75–90 minSetup: Install Claude Code, Node.js, pnpm and TypeScript Answer
Fan out to eve subagents with structured resultsLesson 1 · Multi-agent systemsProduction Agent SystemsStretch· 90 minSetup: Node.js, pnpm and TypeScript, Durable agents with eve, Get an OpenRouter key, Local models with Ollama Answer
Build and break an idempotent outboxLesson 2 · Durable, long-running agentsProduction Agent SystemsCore· 60–90 minSetup: Install Claude Code, Node.js, pnpm and TypeScript Answer
An eve workflow tool that waits for a search resultLesson 2 · Durable, long-running agentsProduction Agent SystemsStretch· 90 minSetup: Node.js, pnpm and TypeScript, Durable agents with eve, Get an OpenRouter key, Local models with Ollama Answer
Red-team and harden a pack-reading agent in Claude CodeLesson 3 · Security and prompt injectionProduction Agent SystemsCore· 60–90 minSetup: Install Claude Code, Node.js, pnpm and TypeScript Answer
Fixed recipients and a locked sandbox in eveLesson 3 · Security and prompt injectionProduction Agent SystemsStretch· 90 minSetup: Node.js, pnpm and TypeScript, Durable agents with eve, Get an OpenRouter key, Local models with Ollama, Docker and Docker Compose Answer
Build a hash-chained audit log and a supervision samplerLesson 4 · Compliance and auditabilityProduction Agent SystemsCore· 60–90 minSetup: Install Claude Code, Node.js, pnpm and TypeScript Answer
Audit hook and retention policy in eveLesson 4 · Compliance and auditabilityProduction Agent SystemsStretch· 90 minSetup: Install Claude Code, Durable agents with eve, Get an OpenRouter key, Local models with Ollama Answer
Trace Claude Code runs and build a per-matter budget reportLesson 5 · Observability and costProduction Agent SystemsCore· 60–90 minSetup: Install Claude Code, Node.js, pnpm and TypeScript, Docker and Docker Compose, Trace LLM calls with OpenTelemetry and Langfuse Answer
Session budgets and redacted exports in eveLesson 5 · Observability and costProduction Agent SystemsStretch· 75 minSetup: Install Claude Code, Durable agents with eve, Get an OpenRouter key, Local models with Ollama, Trace LLM calls with OpenTelemetry and Langfuse Answer
Build a paired upgrade gate for Claude Code runsLesson 6 · Continuous improvementProduction Agent SystemsCore· 60–90 minSetup: Install Claude Code, Node.js, pnpm and TypeScript Answer
An eval-gated model swap in eve CILesson 6 · Continuous improvementProduction Agent SystemsStretch· 90 minSetup: Install Claude Code, Durable agents with eve, Get an OpenRouter key, Local models with Ollama, Run evals in GitHub Actions Answer
Production-readiness review and hardeningCapstone projectProduction Agent SystemsCapstone Answer
Simulate the three buckets and the cache cliffLesson 1 · Rate limits and quotasScaling AI SystemsCore· 60–75 minSetup: Install Claude Code, Python with uv Answer
Log rate-limit headers and cache usageLesson 1 · Rate limits and quotasScaling AI SystemsStretch· 45 minSetup: Install Claude Code, Python with uv, Get an OpenRouter key, OpenRouter or a local model?, Local models with Ollama Answer
Build and test a token-aware, fair limiterLesson 2 · Concurrency and flow controlScaling AI SystemsCore· 75–90 minSetup: Install Claude Code, Python with uv Answer
Adaptive concurrency under a moving limitLesson 2 · Concurrency and flow controlScaling AI SystemsStretch· 60 minSetup: Install Claude Code, Python with uv Answer
Simulate a retry storm and stop itLesson 3 · Retries, backoff and overloadScaling AI SystemsCore· 75 minSetup: Install Claude Code, Python with uv Answer
A circuit breaker with deadlines in front of a flaky mockLesson 3 · Retries, backoff and overloadScaling AI SystemsStretch· 60–75 minSetup: Install Claude Code, Python with uv Answer
Simulate the month-end rushLesson 4 · Backpressure, load shedding and batchingScaling AI SystemsCore· 75–90 minSetup: Install Claude Code, Python with uv Answer
A deferrable-work batch pipeline against a mockLesson 4 · Backpressure, load shedding and batchingScaling AI SystemsStretch· 60–90 minSetup: Install Claude Code, Python with uv, Get an OpenRouter key, OpenRouter or a local model?, Local models with Ollama Answer
Instrument and speed up a real agent turnLesson 5 · Latency engineering for agentsScaling AI SystemsCore· 75–90 minSetup: Install Claude Code, Python with uv, Get an OpenRouter key, Local models with Ollama Answer
When is speculation worth it?Lesson 5 · Latency engineering for agentsScaling AI SystemsStretch· 45–60 minSetup: Install Claude Code, Python with uv Answer
Measure a semantic cache as a classifierLesson 6 · Response caching and failoverScaling AI SystemsCore· 75–90 minSetup: Install Claude Code, Python with uv Answer
Simulate failover with a cold cacheLesson 6 · Response caching and failoverScaling AI SystemsStretch· 60–75 minSetup: Install Claude Code, Python with uv Answer
Load test a fleet slice against a mockLesson 7 · Load testing and capacity planningScaling AI SystemsCore· 90–120 minSetup: Install Claude Code, Python with uv Answer
Write a capacity plan for doubling the fleetLesson 7 · Load testing and capacity planningScaling AI SystemsStretch· 60 minSetup: Install Claude Code, Python with uv Answer
Month-end readiness for an agent fleetCapstone projectScaling AI SystemsCapstone Answer
Build a per-matter cost model from usage logsLesson 1 · Token economics of an agent fleetLLM Cost EngineeringCore· 60–90 minSetup: Install Claude Code, Python with uv Answer
Calibrate the model against real token countsLesson 1 · Token economics of an agent fleetLLM Cost EngineeringStretch· 45 minSetup: Install Claude Code, Python with uv Answer
Simulate cache hits and costs for a matterLesson 2 · Prompt caching as system designLLM Cost EngineeringCore· 75 minSetup: Install Claude Code, Python with uv Answer
Measure warm-then-fan-out on your subscriptionLesson 2 · Prompt caching as system designLLM Cost EngineeringStretch· 45 minSetup: Install Claude Code, Python with uv, Get an OpenRouter key Answer
Simulate context policies with and without cachingLesson 3 · Trimming and compacting contextLLM Cost EngineeringCore· 60–75 minSetup: Install Claude Code, Python with uv Answer
Test what compaction forgetsLesson 3 · Trimming and compacting contextLLM Cost EngineeringStretch· 45–60 minSetup: Install Claude Code, Python with uv Answer
Build a resumable batch extraction jobLesson 4 · Batch, flex and fast tiersLLM Cost EngineeringCore· 75–90 minSetup: Install Claude Code, Python with uv, Get an OpenRouter key Answer
Measure a cost-quality curve and test a cascadeLesson 5 · Cost-quality curves, routing and cascadesLLM Cost EngineeringCore· 90–120 minSetup: Install Claude Code, Python with uv Answer
Does a learned router beat self-confidence?Lesson 5 · Cost-quality curves, routing and cascadesLLM Cost EngineeringStretch· 60 minSetup: Install Claude Code, Python with uv Answer
Distil a clause tagger and measure the gapLesson 6 · Distilling into small modelsLLM Cost EngineeringCore· 2–3 hoursSetup: Install Claude Code, Python with uv, Run GPU code on Modal, Run Hugging Face models on CPU Answer
Build a serving cost calculatorLesson 7 · Self-hosting break-evenLLM Cost EngineeringCore· 60–90 minSetup: Install Claude Code, Python with uv Answer
Benchmark an engine yourselfLesson 7 · Self-hosting break-evenLLM Cost EngineeringStretch· 60–90 minSetup: Install Claude Code, Python with uv, Run GPU code on Modal, Serve a model with vLLM Answer
Build a cost ledger with budgets and alertsLesson 8 · Attribution, budgets and alertsLLM Cost EngineeringCore· 90–120 minSetup: Install Claude Code, Python with uv Answer
Cost-engineer an agent pipelineCapstone projectLLM Cost EngineeringCapstone Answer
Measure what parsing prose costsLesson 1 · What a System One model isSystem One Models with JevCore· 60–75 minSetup: Python with uv, Jev, or its local stand-in, Install Claude Code Answer
Build a calibration toolkit and test it on two modelsLesson 2 · Calibrated probabilitiesSystem One Models with JevCore· 60–90 minSetup: Python with uv, Jev, or its local stand-in, Install Claude Code Answer
Repair an overconfident model with temperature scalingLesson 2 · Calibrated probabilitiesSystem One Models with JevStretch· 45–60 minSetup: Python with uv Answer
Build and test your first requestLesson 3 · Your first Jev requestSystem One Models with JevCore· 60–75 minSetup: Python with uv, Jev, or its local stand-in, Install Claude Code Answer
Log the model that answered, and catch an alias moveLesson 3 · Your first Jev requestSystem One Models with JevStretch· 45–60 minSetup: Python with uv, Jev, or its local stand-in, Install Claude Code Answer
Build an enquiry question set and an option-order checkLesson 4 · Choice, Score and NoulSystem One Models with JevCore· 60–90 minSetup: Python with uv, Jev, or its local stand-in, Install Claude Code Answer
Route through a taxonomy with chained Choices and beam searchLesson 4 · Choice, Score and NoulSystem One Models with JevStretch· 60 minSetup: Python with uv, Jev, or its local stand-in, Install Claude Code Answer
Build a state builder and a reads-then-computes pipelineLesson 5 · State and decompositionSystem One Models with JevCore· 60–90 minSetup: Python with uv, Jev, or its local stand-in, Install Claude Code Answer
Red-team your questions with injected instructionsLesson 5 · State and decompositionSystem One Models with JevStretch· 60 minSetup: Python with uv, Jev, or its local stand-in, Install Claude Code Answer
Build a cost-derived confidence routerLesson 6 · Routing on confidenceSystem One Models with JevCore· 60–90 minSetup: Python with uv, Jev, or its local stand-in, Install Claude Code Answer
Conformal prediction sets for triageLesson 6 · Routing on confidenceSystem One Models with JevStretch· 45–60 minSetup: Python with uv Answer
Build a draft-then-gate reply pipelineLesson 7 · Jev alongside LLMsSystem One Models with JevCore· 75–90 minSetup: Python with uv, Jev, or its local stand-in, Install Claude Code Answer
Sweep a cascade's gate and find the frontierLesson 7 · Jev alongside LLMsSystem One Models with JevStretch· 45–60 minSetup: Python with uv Answer
Build an eval harness for System One answersLesson 8 · Evaluating Jev on your dataSystem One Models with JevCore· 75–90 minSetup: Python with uv, Jev, or its local stand-in, Install Claude Code Answer
Consistency and version regression checksLesson 8 · Evaluating Jev on your dataSystem One Models with JevStretch· 60 minSetup: Python with uv, Jev, or its local stand-in Answer
Triage a busy inbox with JevCapstone projectSystem One Models with JevCapstone Answer

Coders In Hoods Academy

Sign in to start learning

Free, no card. We use your account to save progress, quiz scores, bookmarks and occasionally send updates from our team.