Instrument and speed up a real agent turn

A three-step agent turn (plan, tool round, answer) timed from the token stream of claude -p, or of any OpenAI-compatible endpoint, with three levers you switch on one at a time. Before/after tables at p50 and p95 show which lever moves which term, and why prefetching saves nothing once tools run in parallel.

Back to the exercise

Before you start

Before you start:Install Claude Code Python with uv Get an OpenRouter key Local models with Ollama

This exercise calls a model. Run it either way

Any model through OpenRouterOne key for Claude, GPT, Gemini, Llama and more. Pay per token, with a credit limit.A local open-source modelFree and private with Ollama. Weaker on hard tasks.

Which should I pick? →

The turn is plain Python: the model is asked which tools it needs, Python runs them, and the model answers from their output. That keeps every term you want to measure in your own code: each model call's time to first event, first token and first text comes from the stream, and the tool round is timed around asyncio. The decision that matters most is to time one clock per call, started before the call, and to report time to first visible text for the turn as well as the total, because the levers move them differently.

Answers Lesson 5: Latency engineering for agents

Step-by-step answer

Core 75–90 min8 steps