Module 2 · Changing the model, and governing the spend · Lesson 7
Self-hosting break-even
Turning GPU hourly rates and measured throughput into a cost per million tokens, comparing it with serverless and batch prices for the same open-weight model, and finding the utilisation and volume at which running vLLM or SGLang yourself pays.
Setup for the exercises:Install Claude CodePython with uvRun GPU code on ModalServe a model with vLLM
By the end of this lesson you can
- Compute self-hosted cost per million tokens from GPU price, measured throughput and utilisation
- Explain how interactivity (tokens per second per user), sequence shape and engine features change throughput and therefore cost
- Find the break-even utilisation and monthly volume against serverless, cached and batch prices, including operations cost
- Benchmark an engine with vllm bench serve and decide between serverless, scale-to-zero GPUs and dedicated GPUs for a workload