Opik
Open-Source LLM Tracing, Evaluation & Production Monitoring by Comet
What is Opik?
Opik is an open-source, end-to-end platform from Comet that unifies the full LLM lifecycle: tracing every call and span, evaluating outputs with heuristic and LLM-as-judge metrics, and monitoring apps in production. Where DeepEval and Ragas are libraries you run, Opik adds a self-hostable dashboard, dataset/experiment management and online evaluation — bridging offline testing and live observability.
"Trace it, score it, ship it, watch it. Opik closes the loop: the same metrics you use in CI run again on live traffic so you catch quality drift in production."
Three Pillars
Tracing
Log full traces & spans for chains and agents with one decorator. Integrates with OpenAI, LangChain, LlamaIndex and OpenTelemetry.
Evaluation
Built-in metrics for hallucination, answer relevance, context recall/precision, moderation — plus custom LLM-judge and heuristic scorers over datasets.
Monitoring
Online evaluation rules score production traffic continuously, with dashboards for cost, latency and quality drift.
Quick Start
pip install opik# Trace any function
from opik import track
@track
def answer(question: str) -> str:
...
# Evaluate a dataset with a built-in metric
from opik.evaluation import evaluate
from opik.evaluation.metrics import Hallucination
evaluate(
dataset=my_dataset,
task=lambda x: {"output": answer(x["input"])},
scoring_metrics=[Hallucination()],
)Opik vs. Library-Only Tools
| Capability | DeepEval / Ragas | Opik |
|---|---|---|
| Offline eval in CI | ✅ | ✅ |
| Trace dashboard / UI | — | ✅ self-hostable |
| Production online eval | — | ✅ |
| Dataset & experiment versioning | partial | ✅ |