Trace LLM calls with OpenTelemetry and Langfuse

A trace records what one request did: each model call with its prompt, output, token counts and latency, nested under the steps around it. Exercises on agents, evals and debugging use traces to show what actually happened. OpenTelemetry (OTel) is the vendor-neutral standard for producing traces; Langfuse is an open-source LLM observability tool that receives them and shows them in a UI. Langfuse's Python and TypeScript SDKs are built on OpenTelemetry, so the spans you send here could go to any OTel backend.

Before you start

• Python with uv or Node.js 22 or newer.
• A model backend from OpenRouter or a local model?: an OpenRouter key or Ollama running.
• For self-hosting: Docker and a few GB of free RAM.

1. Get a Langfuse instance

Option A: Langfuse Cloud (quickest)

Sign up with your personal account at cloud.langfuse.com (EU) or us.cloud.langfuse.com (US). The Hobby plan is free, needs no credit card, and includes 50,000 units a month (each trace, observation and score is a unit) with 30 days of data access. Extra usage isn't available on Hobby, so it can't bill you. Create an organization and a project when prompted.

Option B: self-host with Docker Compose

git clone https://github.com/langfuse/langfuse.git
cd langfuse

Before starting, open docker-compose.yml and replace every value marked # CHANGEME with a long random one (openssl rand -hex 32 makes the ENCRYPTION_KEY). Then:

docker compose up -d
docker compose logs -f langfuse-web

Wait 2 to 3 minutes for Ready, open http://localhost:3000 and click Sign up to create the first user (there's no default login). Stop it with docker compose down; docker compose down -v also deletes the data.

2. Create API keys and a .env

In your project, open Settings → API Keys → Create new API keys. You get a public key (pk-lf-…) and a secret key (sk-lf-…). Put them in .env next to the usual model block:

LANGFUSE_PUBLIC_KEY=pk-lf-...
LANGFUSE_SECRET_KEY=sk-lf-...
LANGFUSE_BASE_URL=https://cloud.langfuse.com
 
OPENAI_BASE_URL=https://openrouter.ai/api/v1
OPENAI_API_KEY=sk-or-...
MODEL=anthropic/claude-haiku-5.5

Use https://us.cloud.langfuse.com for the US region and http://localhost:3000 when self-hosting. For Ollama, swap the model block for OPENAI_BASE_URL=http://localhost:11434/v1, OPENAI_API_KEY=ollama, MODEL=qwen3.5. Keep .env in .gitignore: the secret key lets anyone write to your project.

3. Trace a call from Python

uv add langfuse openai

langfuse.openai is a drop-in replacement for the OpenAI client that records every call as a generation:

import os
 
from langfuse import get_client
from langfuse.openai import OpenAI  # drop-in wrapper: every call becomes a trace
 
client = OpenAI()  # reads OPENAI_BASE_URL and OPENAI_API_KEY
 
response = client.chat.completions.create(
    name="hello-trace",
    model=os.environ["MODEL"],
    max_tokens=100,
    messages=[{"role": "user", "content": "Reply with the single word: ready"}],
)
print(response.choices[0].message.content)
 
get_client().flush()  # send buffered spans before the script exits

uv run --env-file .env traced.py

Spans are sent in the background in batches. A long-running server sends them on its own; a short script must flush() or they're lost.

4. Trace a call from TypeScript

pnpm add openai @langfuse/openai @langfuse/otel @opentelemetry/sdk-node

Start the OpenTelemetry Node SDK with Langfuse's span processor before anything else runs:

import { NodeSDK } from "@opentelemetry/sdk-node";
import { LangfuseSpanProcessor } from "@langfuse/otel";
 
export const sdk = new NodeSDK({
  spanProcessors: [new LangfuseSpanProcessor()],
});
 
sdk.start();

import { sdk } from "./instrumentation.ts"; // must be the first import
import OpenAI from "openai";
import { observeOpenAI } from "@langfuse/openai";
 
// reads OPENAI_BASE_URL and OPENAI_API_KEY; the wrapper records each call
const client = observeOpenAI(new OpenAI(), { generationName: "hello-trace" });
 
const response = await client.chat.completions.create({
  model: process.env.MODEL!,
  max_tokens: 100,
  messages: [{ role: "user", content: "Reply with the single word: ready" }],
});
console.log(response.choices[0].message.content);
 
await sdk.shutdown(); // flushes spans before the script exits

node --env-file-if-exists=.env traced.ts

LangfuseSpanProcessor reads the three LANGFUSE_* variables itself. Because it's a normal OTel span processor, spans from other OTel instrumentation in the same process (HTTP, databases) land in the same trace.

5. Send from any OpenTelemetry exporter

Langfuse also accepts plain OTLP over HTTP, so a framework or service that already emits OpenTelemetry can send to it without the Langfuse SDK. Build the Basic-auth string from your keys and point the standard exporter variables at the /api/public/otel endpoint:

AUTH=$(echo -n "pk-lf-...:sk-lf-..." | base64)
export OTEL_EXPORTER_OTLP_ENDPOINT="https://cloud.langfuse.com/api/public/otel"
export OTEL_EXPORTER_OTLP_HEADERS="Authorization=Basic%20$AUTH,x-langfuse-ingestion-version=4"

%20 is the URL-encoded space the OTel header format expects. On Linux, use base64 -w 0 so long keys don't wrap. Only HTTP (JSON or protobuf) is supported, not gRPC. Langfuse maps the OpenTelemetry GenAI attributes (gen_ai.request.model, gen_ai.usage.* and so on) onto its model, token and cost fields.

Check it works

Run traced.py or traced.ts. It prints ready. Then open your project's Tracing page in Langfuse: within a few seconds there's a trace containing a hello-trace generation. Click it to see the input message, the output, the model name, input and output tokens, and the latency.

Troubleshooting

The script runs but no trace appears. The spans never left the process: check the flush() or sdk.shutdown() line ran. Also check that .env is loaded (--env-file) and that LANGFUSE_BASE_URL matches the region your project is in; an EU key doesn't work against the US host.

401 or an authentication warning from Langfuse. The public and secret keys are swapped, belong to another project, or have extra quotes or spaces in .env.

Self-hosted stack won't start, or a port is in use. The compose file binds 3000, plus 5432 (Postgres) and 6379 (Redis) on localhost. Stop any local Postgres or Redis on those ports, or change the host side of the mapping.

Unknown file extension ".ts" or an engine warning. @langfuse/otel and the OTel Node SDK need Node 22 or newer, and running .ts directly needs 22.18+. Upgrade as in Node.js, pnpm and TypeScript.

Going further

• OpenRouter or a local model?, the model configuration these scripts use
• Docker, for the self-hosted stack
• Run Temporal locally and Serve a model with vLLM, both of which you can trace the same way