Self-host an open parsing model

olmOCR 2 (7B, FP8) served by vLLM on a Modal GPU behind an API key, called through the OpenAI SDK as the bake-off's fifth parser. A cost model then separates the GPU's steady-state price per page from the cold start and idle tail you pay on small jobs. The same client runs against OpenRouter, Ollama or a local fake server with no code change.

Back to the exercise

Before you start

Before you start:Install Claude Code Python with uv Run GPU code on Modal Serve a model with vLLM Get an OpenRouter key Local models with Ollama

This exercise calls a model. Run it either way

Any model through OpenRouterOne key for Claude, GPT, Gemini, Llama and more. Pay per token, with a credit limit.A local open-source modelFree and private with Ollama. Weaker on hard tasks.

Which should I pick? →

The self-hosted parser is a server and a client. The server is vllm serve on a rented GPU, which speaks the OpenAI chat API. The client is the OpenAI SDK created with no arguments, so .env alone decides whether a page goes to your GPU, to a hosted model on OpenRouter or to Ollama. The decision that matters is to measure throughput under concurrency and then price whole jobs, not single pages: on a per-second GPU you pay from container start to scale-down, so one matter costs far more per page than a backfill does.

Answers Lesson 2: Parsing: text layers to vision models

Step-by-step answer

Stretch 90 min8 steps