Benchmark an engine yourself

The benchmark runs where the engine runs. One Modal function starts vllm serve, waits for /health, then runs vllm bench serve against localhost for every configuration, so the numbers measure the engine, not the network between your laptop and a GPU. Everything after the GPU run, from parsing to the table, the costs and the prefix comparison, is ordinary Python that runs on a CPU. The decision that matters most is choosing the operating point by a latency target, not by peak throughput: the highest-throughput row is usually one no user would wait for.