OpenLLMetry (Traceloop)
https://github.com/traceloop/openllmetry
Traceloop's open-source OpenTelemetry auto-instrumentation for LLM workflows — works with any OTel-compatible backend (Datadog, New Relic, Grafana, etc.).
At a glance
- Type
- OTel auto-instrumentation for LLMs
- Tier
- T2
- Created
- 2023
- Latest release
- searched not found
- License
- Apache-2.0
- GitHub
- searched not found
- Pricing
- searched not found
- Funding
- Traceloop raised $6.3M seed 2024 (Sorenson Capital led).
Taxonomy
- storage
- n/a
- retrieval
- n/a
- persistence
- n/a
- update
- n/a
- unit
- n/a
- governance
- n/a
- conflict
- n/a
When to use
Optimised for: searched not found
Anti-fit: searched not found
Pros & cons
Pros
searched not found
Cons
searched not found
Claims & capabilities
searched not found
Technical surface
- API surface
- searched not found
- Backend storage
- not applicable — not a memory product
- Deployment
- searched not found
- Embedding model
- not applicable — not a memory product
- Multi-tenancy
- searched not found
- MCP
- searched not found
- A2A
- searched not found
- OpenTelemetry
- searched not found
Similar systems
Other evaluation & observability platforms in the catalog, ranked by inbound references.
- Arize AI T1
Production ML monitoring + LLM observability (Arize AX) — drift, eval, root-cause. Pre-LLM observability leader; now LLM-pivoted.
- Arize Phoenix T1
Arize AI's open-source LLM observability tool — OpenTelemetry-native LLM tracing, evals, datasets. Arize commercial offers Phoenix + Surveyor.
- Braintrust T1
Eval-first LLM platform — datasets, eval runs, prompt playground, tracing. Founded by ex-Impira (acquired by Dropbox) team. Used by Notion, Brex, AirOps.
- Confident AI / DeepEval T2
Open-source DeepEval framework + Confident AI cloud — pytest-style eval runs, hallucination/relevance/bias metrics, datasets.
- Galileo T1
Galileo Evaluate / Observe / Protect — eval, tracing, guardrails. Big focus on hallucination detection and ChainPoll evaluator metric.
- Helicone T2
Open-source LLM observability with one-line proxy integration — gateway routes through their proxy, captures every LLM call. Sessions + custom properties + evals.