A deferrable-work batch pipeline against a mock

The deferrable-work path as a small, durable pipeline. P3 items go into SQLite, are flushed as batches every 10 minutes or 5,000 items within the API's size limits, polled, and their results applied through guarded updates, so duplicates are ignored and expired items are resubmitted once and then sent to the synchronous queue. A fake batches service runs it offline; an emulated backend runs it on a real model through OpenRouter or Ollama.

Back to the exercise

Before you start

Before you start:Install Claude Code Python with uv Get an OpenRouter key OpenRouter or a local model?Local models with Ollama

This exercise calls a model. Run it either way

Any model through OpenRouterOne key for Claude, GPT, Gemini, Llama and more. Pay per token, with a credit limit.A local open-source modelFree and private with Ollama. Weaker on hard tasks.

Which should I pick? →

The pipeline is three parts with narrow contracts: a store that owns every state change, a backend with the Message Batches API's shape (create, retrieve, results), and a pipeline whose tick() a scheduler calls once a minute. The decision that matters most is that the store, not the pipeline, enforces "exactly one terminal state": each update names the state and batch it expects to move from, so a duplicate result, a late result from an older batch or a crash between two writes can't apply anything twice.

Answers Lesson 4: Backpressure, load shedding and batching

Step-by-step answer

Stretch 60–90 min6 steps