Datadog Agent Observability (LLM Observability)
https://www.datadoghq.com/products/ai/agent-observability/
Datadog's product for evaluating, tracing and monitoring AI agents, combining offline experiments on golden datasets with production tracing and troubleshooting. Its former LLM Observability URL now redirects here.
Active · Verified 2026-10-29 · fresh as of 2026-10-29 · 16 cited facts, 0 with a third-party source · 1 recorded change · Suggest a correction
At a glance
- Tier
- T3
- Created
- searched not found fix
- Latest release
- searched not found fix
- License
- proprietary vendor fix
- GitHub
- not applicable — no GitHub repo fix
- Pricing
- searched not found fix
- Funding
- searched not found fix
- Freshness
- 2026-10-29 fix
- Layer
- agent-infra repo fix
- Licensing
- commercial vendor fix
- SDK languages
- go, java, python, rest, typescript vendor fix
- Protocols
- opentelemetry vendor fix
When to use
Optimised for: searched not found
Anti-fit: searched not found
Pros & cons
Pros
searched not found
Cons
searched not found
Claims & capabilities
The integrated platform for monitoring & security
Technical surface
- API surface
- searched not found
- Backend storage
- searched not found
- Deployment
- searched not found
- Embedding model
- searched not found
- Multi-tenancy
- searched not found
- MCP
- searched not found
- A2A
- searched not found
- OpenTelemetry
- yes
Similar systems
Other evaluation & observability platforms in the catalog, ranked by inbound references.
- Arize Phoenix T1
Arize AI's open-source LLM observability tool — OpenTelemetry-native LLM tracing, evals, datasets. Arize commercial offers Phoenix + Surveyor.
- Arize AI T1
Production ML monitoring + LLM observability (Arize AX) — drift, eval, root-cause. Pre-LLM observability leader; now LLM-pivoted.
- Braintrust T1
Eval-first LLM platform — datasets, eval runs, prompt playground, tracing. Founded by ex-Impira (acquired by Dropbox) team. Used by Notion, Brex, AirOps.
- Confident AI / DeepEval T2
Open-source DeepEval framework + Confident AI cloud — pytest-style eval runs, hallucination/relevance/bias metrics, datasets.
- OpenLLMetry (Traceloop) T2
Traceloop's open-source OpenTelemetry auto-instrumentation for LLM workflows — works with any OTel-compatible backend (Datadog, New Relic, Grafana, etc.).
- Helicone T2
Open-source LLM observability and AI gateway — one-line proxy integration captures every LLM call; sessions, custom properties, evals. Acquired by Mintlify in Mar 2026 and placed in maintenance mode; helicone.ai now states it is no longer part of Mintlify.
Sources (15 cited facts, 0 third-party)
Each fact links to the page it came from; the badge says who published it (how to read sources).
- What it is Product vendor
- What it does & what's distinct Datadog's product for evaluating, tracing and monitoring AI agents, combining offline ex… vendor
- Claims / benefits The integrated platform for monitoring & security vendor
- License proprietary vendor
- OTel yes vendor
- Project & sources https://www.datadoghq.com/products/ai/agent-observability/ vendor
- obs-opentelemetry yes vendor
- obs-datadog yes vendor
- eval-human-loop yes vendor
- licensing-bucket commercial vendor
- taxonomy-three-layer-rollup agent-infra repo
- compliance-certifications ccpa,csa-star,fedramp,gdpr,hipaa,iso27001,iso27017,iso27018,iso27701,iso42001,pci-dss,so… vendor
- sdk-languages go,java,python,rest,typescript vendor
- protocol-support opentelemetry vendor
- maturity ga vendor
What changed
Changes to this record's lifecycle, licence, release, pricing and hosting facts, from the catalog's git history. Subscribe (Atom)
- 2026-10-30 Added to the catalog #347
Related systems
References (16)
- Anthropic Claude (foundation models) integrates with — Datadog LLM Observability auto-instrumentation page lists a Anthropic LLM integration. (verified 2026-10-29)
- Claude Agent SDK integrates with — Datadog LLM Observability auto-instrumentation page lists a Claude Agent SDK LLM integration. (verified 2026-10-29)
- CrewAI integrates with — Datadog LLM Observability auto-instrumentation page lists a CrewAI LLM integration. (verified 2026-10-29)
- Google Agent Development Kit (ADK) integrates with — Datadog LLM Observability auto-instrumentation page lists a Google ADK LLM integration. (verified 2026-10-29)
- Google Gemini 3 family integrates with — Datadog LLM Observability auto-instrumentation page lists a Google GenAI LLM integration. (verified 2026-10-29)
- LangChain (framework) integrates with — Datadog LLM Observability auto-instrumentation page lists a LangChain LLM integration. (verified 2026-10-29)
- LangGraph (orchestration) integrates with — Datadog LLM Observability auto-instrumentation page lists a LangGraph LLM integration. (verified 2026-10-29)
- LiteLLM integrates with — Datadog LLM Observability auto-instrumentation page lists a LiteLLM LLM integration. (verified 2026-10-29)
- Mistral Large 2 / Mixtral family integrates with — Datadog LLM Observability auto-instrumentation page lists a MistralAI LLM integration. (verified 2026-10-29)
- Model Context Protocol (MCP spec) integrates with — Datadog LLM Observability auto-instrumentation page lists a MCP LLM integration. (verified 2026-10-29)
- OpenAI Agents SDK integrates with — Datadog LLM Observability auto-instrumentation page lists a OpenAI Agents LLM integration. (verified 2026-10-29)
- OpenAI GPT family (GPT-5 / GPT-4o / o3 / o4) integrates with — Datadog LLM Observability auto-instrumentation page lists a OpenAI LLM integration. (verified 2026-10-29)
- Pydantic-AI integrates with — Datadog LLM Observability auto-instrumentation page lists a Pydantic AI LLM integration. (verified 2026-10-29)
- Strands Agents (AWS) integrates with — Datadog LLM Observability auto-instrumentation page lists a Strands Agents LLM integration. (verified 2026-10-29)
- Vercel AI SDK integrates with — Datadog LLM Observability auto-instrumentation page lists a Vercel AI SDK LLM integration. (verified 2026-10-29)
- vLLM integrates with — Datadog LLM Observability auto-instrumentation page lists a vLLM LLM integration. (verified 2026-10-29)
Referenced by (4)
- agentgateway integrates with — agentgateway integrations docs have a Datadog observability page. (verified 2026-10-29)
- Bifrost (Maxim AI) integrates with — Bifrost docs have a native Datadog integration page (APM traces, LLM Observability, metrics). (verified 2026-10-29)
- CrewAI integrates with — CrewAI observability docs have a Datadog integration page. (verified 2026-10-29)
- LiteLLM integrates with — LiteLLM docs have a Datadog observability page. (verified 2026-10-29)