OpenLIT

Open-source AI engineering platform — observability, evaluations, guardrails, prompt management, and GPU monitoring for LLMs and AI applications.

OpenLIT is an open-source AI engineering platform that helps teams build, evaluate, and observe AI applications across the entire lifecycle — from development to production. It provides OpenTelemetry-native auto-instrumentation for 50+ LLM providers, AI frameworks, vector databases, and GPUs with a single line of code.

Deploy OpenLIT

Self-host with Docker Compose or Kubernetes Helm in minutes

Quickstart

Get LLM observability running in under 5 minutes

SDK Reference

Python, TypeScript, and Go SDKs with zero-code instrumentation

Integrations

50+ LLM providers, frameworks, and vector databases

What is OpenLIT?

OpenLIT provides three interconnected OSS tools:

OpenLIT Platform

Full-stack platform for tracing, dashboards, prompt management, evaluations, and secrets management

OpenLIT SDKs

OpenTelemetry-native auto-instrumentation SDKs for Python, TypeScript, and Go

Kubernetes Operator

Automatically inject instrumentation into AI workloads without code changes

Key features

LLM Observability

Distributed tracing and metrics for every LLM call — latency, cost, token usage, and errors

GPU Monitoring

Real-time NVIDIA and AMD GPU utilization, memory, temperature, and power metrics

Evaluations

LLM-as-a-judge and programmatic evaluations to measure response quality in production

Guardrails

Real-time prompt injection detection, topic restrictions, and sensitive data protection

Prompt Hub

Version, manage, and deploy prompts across environments without code changes

Vault

Centrally store LLM API keys — applications retrieve them remotely at runtime

OpenGround

Compare cost, latency, and quality across LLMs side-by-side in a playground

Custom Dashboards

Build dashboards with flexible widgets and custom SQL queries over your telemetry data

How it works

1. Deploy OpenLIT

   Run the OpenLIT stack with Docker Compose — includes the UI, ClickHouse database, and OpenTelemetry Collector.

   git clone https://github.com/openlit/openlit.git
   cd openlit
   docker compose up -d
   
2. Install the SDK

   Install the OpenLIT Python SDK from PyPI.

   pip install openlit
   
3. Instrument your application

   Add two lines to your existing AI application — OpenLIT auto-detects and instruments all supported libraries.

   import openlit
   
   openlit.init(otlp_endpoint="http://127.0.0.1:4318")
   
4. Visualize and optimize

   Open http://127.0.0.1:3000 and log in with the default credentials ([e-mail] / openlituser) to explore traces, metrics, and dashboards.

Supported integrations

OpenLIT auto-instruments 50+ integrations across LLM providers, AI frameworks, and vector databases:

LLM Providers — OpenAI, Anthropic, Cohere, Mistral, Groq, AWS Bedrock, Google Vertex AI, Google AI Studio, Azure OpenAI, Ollama, vLLM, Hugging Face, DeepSeek, and more

AI Frameworks — LangChain, LlamaIndex, CrewAI, OpenAI Agents SDK, Pydantic AI, AutoGen (AG2), Haystack, LiteLLM, Mem0, and more

Vector Databases — Pinecone, ChromaDB, Qdrant, Milvus, Astra DB, PostgreSQL (pgvector)

GPUs — NVIDIA (nvidia-smi), AMD (ROCm)

View all integrations

Browse the full list of supported integrations with setup guides

Open source

OpenLIT is released under the Apache 2.0 license. It follows and maintains OpenTelemetry Semantic Conventions for GenAI observability, consistently updating to align with the latest standards.

GitHub

Star the repo, report issues, and contribute

Slack Community

Join the community for support and discussions