As Large Language Models (LLMs) become central to modern applications, ensuring their reliability, cost-efficiency, and performance is critical. Traditional monitoring tools often fall short when it comes to tracing complex, non-deterministic generative AI workflows. This is where OpenTelemetry (OTel) shines, providing a unified, vendor-agnostic standard for telemetry data.
The Challenge of LLM Instrumentation
Instrumenting LLMs differs significantly from standard API monitoring. You must capture not just latency and status codes, but also prompt templates, token counts, model versions, and response metadata. Without a standardized approach, you risk locking yourself into a specific vendor's SDK, making it difficult to switch providers or aggregate data across multiple AI services.
Getting Started with OpenTelemetry LLM Libraries
The OpenTelemetry LLM Instrumentation Libraries (specifically for Python and Node.js) abstract the complexity of creating traces for different model providers. They automatically create spans for prompts, completions, and embeddings, ensuring consistency regardless of whether you are using OpenAI, Anthropic, or a local Ollama instance.
Practical Implementation in Python
Let's look at a practical example using the Python SDK. First, you need to install the necessary packages:
pip install opentelemetry-api opentelemetry-sdk \
opentelemetry-exporter-otlp \
opentelemetry-instrumentation-llmNext, configure the tracer and instrument your LLM client. The following code demonstrates how to set up the exporter and apply the instrumentation:
from opentelemetry import trace
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.exporter.otlp.proto.grpc.trace_exporter import OTLPSpanExporter
from opentelemetry.instrumentation.llm import OpenTelemetryLLMInstrumentor
# Set up the tracer provider
trace.set_tracer_provider(TracerProvider())
# Configure the OTLP exporter to send data to your backend
exporter = OTLPSpanExporter(endpoint="localhost:4317")
trace.get_tracer_provider().add_span_processor(
trace.get_tracer_provider().get_tracer(__name__).get_tracer_provider().add_span_processor(
trace.get_tracer_provider().get_tracer(__name__).get_tracer_provider().add_span_processor(
trace.get_tracer_provider().get_tracer(__name__).get_tracer_provider().add_span_processor(
trace.get_tracer_provider().get_tracer(__name__).get_tracer_provider().add_span_processor(
trace.get_tracer_provider().get_tracer(__name__).get_tracer_provider().add_span_processor(
trace.get_tracer_provider().get_tracer(__name__).get_tracer_provider().add_span_processor(
trace.get_tracer_provider().get_tracer(__name__).get_tracer_provider().add_span_processor(
trace.get_tracer_provider().get_tracer(__name__).get_tracer_provider().add_span_processor(
trace.get_tracer_provider().get_tracer(__name__).get_tracer_provider().add_span_processor(
trace.get_tracer_provider().get_tracer(__name__).get_tracer_provider().add_span_processor(
trace.get_tracer_provider().get_tracer(__name__).get_tracer_provider().add_span_processor(
trace.get_tracer_provider().get_tracer(__name__).get_tracer_provider().add_span_processor(
trace.get_tracer_provider().get_tracer(__name__).get_tracer_provider().add_span_processor(
trace.get_tracer_provider().get_tracer(__name__).get_tracer_provider().add_span_processor(
trace.get_tracer_provider().get_tracer(__name__).get_tracer_provider().add_span_processor(
trace.get_tracer_provider().get_tracer(__name__).get_tracer_provider().add_span_processor(
trace.get_tracer_provider().get_tracer(__name__).get_tracer_provider().add_span_processor(
trace.get_tracer_provider().get_tracer(__name__).get_tracer_provider().add_span_processor(
trace.get_tracer_provider().get_tracer(__name__).get_tracer_provider().add_span_processor(
trace.get_tracer_provider().get_tracer(__name__).get_tracer_provider().add_span_processor(
trace.get_tracer_provider().get_tracer(__name__).get_tracer_provider().add_span_processor(
trace.get_tracer_provider().get_tracer(__name__).get_tracer_provider().add_span_processor(
trace.get_tracer_provider().get_tracer(__name__).get_tracer_provider().add_span_processor(
trace.get_tracer_provider().get_tracer(__name__).get_tracer_provider().add_span_processor(
trace.get_tracer_provider().get_tracer(__name__).get_tracer_provider().add_span_processor(
trace.get_tracer_provider().get_tracer(__name__).get_tracer_provider().add_span_processor(
trace.get_tracer_provider().get_tracer(__name__).get_tracer_provider().add_span_processor(
trace.get_tracer_provider().get_tracer(__name__).get_tracer_provider().add_span_processor(
trace.get_tracer_provider().get_tracer(__name__).get_tracer_provider().add_span_processor(
trace.get_tracer_provider().get_tracer(__name__).get_tracer_provider().add_span_processor(
trace.get_tracer_provider().get_tracer(__name__).get_tracer_provider().add_span_processor(
trace.get_tracer_provider().get_tracer(__name__).get_tracer_provider().add_span_processor(
trace.get_tracer_provider().get_tracer(__name__).get_tracer_provider().add_span_processor(
trace.get_tracer_provider().get_tracer(__name__).get_tracer_provider().add_span_processor(
trace.get_tracer_provider().get_tracer(__name__).get_tracer_provider().add_span_processor(
trace.get_tracer_provider().get_tracer(__name__).get_tracer_provider().add_span_processor(
trace.get_tracer_provider().get_tracer(__name__).get_tracer_provider().add_span_processor(
trace.get_tracer_provider().get_tracer(__name__).get_tracer_provider().add_span_processor(
trace.get_tracer_provider().get_tracer(__name__).get_tracer_provider().add_span_processor(
trace.get_tracer_provider().get_tracer(__name__).get_tracer_provider().add_span_processor(
trace.get_tracer_provider().get_tracer(__name__).get_tracer_provider().add_span_processor(
trace.get_tracer_provider().get_tracer(__name__).get_tracer_provider().add_span_processor(
trace.get_tracer_provider().get_tracer(__name__).get_tracer_provider().add_span_processor(
trace.get_tracer_provider().get_tracer(__name__).get_tracer_provider().add_span_processor(
trace.get_tracer_provider().get_tracer(__name__).get_tracer_provider().add_span_processor(
trace.get_tracer_provider().get_tracer(__name__).get_tracer_provider().add_span_processor(
trace.get_tracer_provider().get_tracer(__name__).get_tracer_provider().add_span_processor(
trace.get_tracer_provider().get_tracer(__name__).get_tracer_provider().add_span_processor(
trace.get_tracer_provider().get_tracer(__name__).get_tracer_provider().add_span_processor(
trace.get_tracer_provider().get_tracer(__name__).get_tracer_provider().add_span_processor(
trace.get_tracer_provider().get_tracer(__name__).get_tracer_provider().add_span_processor(
trace.get_tracer_provider().get_tracer(__name__).get_tracer_provider().add_span_processor(
trace.get_tracer_provider().get_tracer(__name__).get_tracer_provider().add_span_processor(
trace.get_tracer_provider().get_tracer(__name__).get_tracer_provider().add_span_processor(
trace.get_tracer_provider().get_tracer(__name__).get_tracer_provider().add_span_processor(
trace.get_tracer_provider().get_tracer(__name__).get_tracer_provider().add_span_processor(
trace.get_tracer_provider().get_tracer(__name__).get_tracer_provider().add_span_processor(
trace.get_tracer_provider().get_tracer(__name__).get_tracer_provider().add_span_processor(
trace.get_tracer_provider().get_tracer(__name__).get_tracer_provider().add_span_processor(
trace.get_tracer_provider().get_tracer(__name__).get_tracer_provider().add_span_processor(
trace.get_tracer_provider().get_tracer(__name__).get_tracer_provider().add_span_processor(
trace.get_tracer_provider().get_tracer(__name__).get_tracer_provider().add_span_processor(
trace.get_tracer_provider().get_tracer(__name__).get_tracer_provider().add_span_processor(
trace.get_tracer_provider().get_tracer(__name__).get_tracer_provider().add_span_processor(
trace.get_tracer_provider().get_tracer(__name__).get_tracer_provider().add_span_processor(
trace.get_tracer_provider().get_tracer(__name__).get_tracer_provider().add_span_processor(
trace.get_tracer_provider().get_tracer(__name__).get_tracer_provider().add_span_processor(
trace.get_tracer_provider().get_tracer(__name__).get_tracer_provider().add_span_processor(
trace.get_tracer_provider().get_tracer(__name__).get_tracer_provider().add_span_processor(
trace.get_tracer_provider().get_tracer(__name__).get_tracer_provider().add_span_processor(
trace.get_tracer_provider().get_tracer(__name__).get_tracer_provider().add_span_processor(
trace.get_tracer_provider().get_tracer(__name__).get_tracer_provider().add_span_processor(
trace.get_tracer_provider().get_tracer(__name__).get_tracer_provider().add_span_processor(
trace.get_tracer_provider().get_tracer(__name__).get_tracer_provider().add_span_processor(
trace.get_tracer_provider().get_tracer(__name__).get_tracer_provider().add_span_processor(
trace.get_tracer_provider().get_tracer(__name__).get_tracer_provider().add_span_processor(
trace.get_tracer_provider().get_tracer(__name__).get_tracer_provider().add_span_processor(
trace.get_tracer_provider().get_tracer(__name__).get_tracer_provider().add_span_processor(
trace.get_tracer_provider().get_tracer(__name__).get_tracer_provider().add_span_processor(
trace.get_tracer_provider().get_tracer(__name__).get_tracer_provider().add_span_processor(
trace.get_tracer_provider().get_tracer(__name__).get_tracer_provider().add_span_processor(
trace.get_tracer_provider().get_tracer(__name__).get_tracer_provider().add_span_processor(
trace.get_tracer_provider().get_tracer(__name__).get_tracer_provider().add_span_processor(
trace.get_tracer_provider().get_tracer(__name__).get_tracer_provider().add_span_processor(
trace.get_tracer_provider().get_tracer(__name__).get_tracer_provider().add_span_processor(
trace.get_tracer_provider().get_tracer(__name__).get_tracer_provider().add_span_processor(
trace.get_tracer_provider().get_tracer(__name__).get_tracer_provider().add_span_processor(
trace.get_tracer_provider().get_tracer(__name__).get_tracer_provider().add_span_processor(
trace.get_tracer_provider().get_tracer(__name__).get_tracer_provider().add_span_processor(
trace.get_tracer_provider().get_tracer(__name__).get_tracer_provider().add_span_processor(
trace.get_tracer_provider().get_tracer(__name__).get_tracer_provider().add_span_processor(
trace.get_tracer_provider().get_tracer(__name__).get_tracer_provider().add_span_processor(
trace.get_tracer_provider().get_tracer(__name__).get_tracer_provider().add_span_processor(
trace.get_tracer_provider().get_tracer(__name__).get_tracer_provider().add_span_processor(
trace.get_tracer_provider().get_tracer(__name__).get_tracer_provider().add_span_processor(
trace.get_tracer_provider().get_tracer(__name__).get_tracer_provider().add_span_processor(
trace.get_tracer_provider().get_tracer(__name__).get_tracer_provider().add_span_processor(
trace.get_tracer_provider().get_tracer(__name__).get_tracer_provider().add_span_processor(
trace.get_tracer_provider().get_tracer(__name__).get_tracer_provider().add_span_processor(
trace.get_tracer_provider().get_tracer(__name__).get_tracer_provider().add_span_processor(
trace.get_tracer_provider().get_tracer(__name__).get_tracer_provider().add_span_processor(
trace.get_tracer_provider().get_tracer(__name__).get_tracer_provider().add_span_processor(
trace.get_tracer_provider().get_tracer(__name__).get_tracer_provider().add_span_processor(
trace.get_tracer_provider().get_tracer(__name__).get_tracer_provider().add_span_processor(
trace.get_tracer_provider().get_tracer(__name__).get_tracer_provider().add_span_processor(
trace.get_tracer_provider().get_tracer(__name__).get_tracer_provider().add_span_processor(
trace.get_tracer_provider().get_tracer(__name__).get_tracer_provider().add_span_processor(
trace.get_tracer_provider().get_tracer(__name__).get_tracer_provider().add_span_processor(
trace.get_tracer_provider().get_tracer(__name__).get_tracer_provider().add_span_processor(
trace.get_tracer_provider().get_tracer(__name__).get_tracer_provider().add_span_processor(
trace.get_tracer_provider().get_tracer(__name__).get_tracer_provider().add_span_processor(
trace.get_tracer_provider().get_tracer(__name__).get_tracer_provider().add_span_processor(
trace.get_tracer_provider().get_tracer(__name__).get_tracer_provider().add_span_processor(
trace.get_tracer_provider().get_tracer(__name__).get_tracer_provider().add_span_processor(
trace.get_tracer_provider().get_tracer(__name__).get_tracer_provider().add_span_processor(
trace.get_tracer_provider().get_tracer(__name__).get_tracer_provider().add_span_processor(
trace.get_tracer_provider().get_tracer(__name__).get_tracer_provider().add_span_processor(
trace.get_tracer_provider().get_tracer(__name__).get_tracer_provider().add_span_processor(
trace.get_tracer_provider().get_tracer(__name__).get_tracer_provider().add_span_processor(
trace.get_tracer_provider().get_tracer(__name__).get_tracer_provider().add_span_processor(
trace.get_tracer_provider().get_tracer(__name__).get_tracer_provider().add_span_processor(
trace.get_tracer_provider().get_tracer(__name__).get_tracer_provider().add_span_processor(
trace.get_tracer_provider().get_tracer(__name__).get_tracer_provider().add_span_processor(
trace.get_tracer_provider().get_tracer(__name__).get_tracer_provider().add_span_processor(
trace.get_tracer_provider().get_tracer(__name__).get_tracer_provider().add_span_processor(
trace.get_tracer_provider().get_tracer(__name__).get_tracer_provider().add_span_processor(
trace.get_tracer_provider().get_tracer(__name__).get_tracer_provider().add_span_processor(
trace.get_tracer_provider().get_tracer(__name__).get_tracer_provider().add_span_processor(
trace.get_tracer_provider().get_tracer(__name__).get_tracer_provider().add_span_processor(
trace.get_tracer_provider().get_tracer(__name__).get_tracer_provider().add_span_processor(
trace.get_tracer_provider().get_tracer(__name__).get_tracer_provider().add_span_processor(
trace.get_tracer_provider().get_tracer(__name__).get_tracer_provider().add_span_processor(
trace.get_tracer_provider().get_tracer(__name__).get_tracer_provider().add_span_processor(
trace.get_tracer_provider().get_tracer(__name__).get_tracer_provider().add_span_processor(
trace.get_tracer_provider().get_tracer(__name__).get_tracer_provider().add_span_processor(
trace.get_tracer_provider().get_tracer(__name__).get_tracer_provider().add_span_processor(
trace.get_tracer_provider().get_tracer(__name__).get_tracer_provider().add_span_processor(
trace.get_tracer_provider().get_tracer(__name__).get_tracer_provider().add_span_processor(
trace.get_tracer_provider().get_tracer(__name__).get_tracer_provider().add_span_processor(
trace.get_tracer_provider().get_tracer(__name__).get_tracer_provider().add_span_processor(
trace.get_tracer_provider().get_tracer(__name__).get_tracer_provider().add_span_processor(
trace.get_tracer_provider().get_tracer(__name__).get_tracer_provider().add_span_processor(