Run GPU code on Modal

Modal runs Python functions on cloud GPUs and bills by the second, with nothing to provision: you decorate a function with the GPU you want and run it. Exercises that need an NVIDIA GPU you don't own (serving a model with vLLM, fine-tuning, batch embedding) use it. You'll set a budget first, so a forgotten server can't run up a bill.

Before you start

• macOS, Linux or Windows, with Python and uv.
• A personal email or GitHub account to sign up with.
• The Starter plan is free and includes $30 of compute credit every month, covering GPUs, CPUs and memory (check modal.com/pricing for the current figure). Modal's billing guide says a payment method must be on file to use Modal; usage is charged against the credit first.

GPU prices are per second. At the time of writing a T4 costs $0.000164/s (about $0.59 an hour), an L4 $0.000222/s (about $0.80 an hour) and an H100 $0.001097/s (about $3.95 an hour).

1. Create an account and set a budget

1. Sign up at modal.com with your personal account. This creates a personal workspace.
2. Add a payment method when asked.
3. Open Settings → Usage & Billing (modal.com/settings/usage) and set the workspace budget (shown as the usage limit). It caps total usage in the billing cycle before credits.

Set it to $30. The spend limit, which caps what you pay after credits, defaults to the budget minus your credits, so a $30 budget with $30 of credit leaves a $0 spend limit: Modal stops workloads instead of charging you. Raise it deliberately if an exercise needs more.

The same page shows your usage so far this cycle. Check it after each session.

2. Install the Modal CLI

In a project:

uv init modal-hello
cd modal-hello
uv add modal

Or install it into any environment with pip install modal. With uv, prefix CLI commands with uv run (as below), or activate .venv.

3. Authenticate

uv run modal setup

This opens a browser, asks you to approve a token for your workspace and saves it to ~/.modal.toml. Keep that file out of git and never paste the token into code. If modal setup isn't found, run uv run python -m modal setup.

4. Run a function on a GPU

import modal
 
app = modal.App("hello-gpu")
 
 
@app.function(gpu="T4")
def check_gpu():
    import subprocess
 
    output = subprocess.check_output(["nvidia-smi"], text=True)
    print(output)
    return output
 
 
@app.local_entrypoint()
def main():
    check_gpu.remote()

• @app.function(gpu="T4") asks for the cheapest GPU. Other names include L4, A10, L40S, A100, H100 and H200.
• .remote() runs the function in a Modal container. The NVIDIA driver is preinstalled, so nvidia-smi works with no image setup.
• @app.local_entrypoint() is what modal run calls on your machine.

uv run modal run hello_gpu.py

The first run builds the default image, then prints the nvidia-smi table from a Tesla T4. The whole run uses a few seconds of GPU time, well under a cent.

5. Stop anything still running

Modal has two kinds of app:

• Ephemeral apps come from modal run. They stop when the command exits or Modal sees the client disconnect, so pressing Ctrl+C stops billing. modal run --detach keeps one running after you exit; avoid it unless an exercise asks for it.
• Deployed apps come from modal deploy and keep running until you stop them. A deployed function with no requests scales to zero containers and costs nothing, but a server configured to keep containers warm (min_containers) bills all the time.

List what's running and stop what you don't need:

uv run modal app list
uv run modal app stop hello-gpu

You can also stop apps from the Apps page of the dashboard. When you finish an exercise that deployed something, stop it.

Check it works

uv run modal run hello_gpu.py

Expected: the nvidia-smi table listing one Tesla T4 with its driver and CUDA versions, and afterwards uv run modal app list showing hello-gpu as stopped.

Troubleshooting

An authentication or missing-token error. Run uv run modal setup again. If you have more than one profile, modal profile list shows which is active.

modal: command not found. The CLI is inside the project's .venv. Run it as uv run modal …, or install it with pip install modal into an environment on your PATH.

A function stops with a budget or spend-limit error. You've reached the limit you set. Check usage on the Usage & Billing page; wait for the next cycle or raise the budget.

The first run is slow. Modal builds the container image once and caches it; later runs start much faster. Use the smallest GPU that fits the job, which also tends to be easiest to get.

Going further

• Serve an open-weight model with vLLM on a Modal GPU
• Local models with Ollama, when your own machine is enough
• Python with uv