Catalog Catalogs Evals & Observability

Evals & Observability

Evaluation platforms, tracing, memory monitoring and the benchmarks behind them. For teams deciding how to test and watch agents in production.

Evaluation and observability platforms, memory observability tools and memory benchmarks, compared on OpenTelemetry support, deployment, pricing, licence and the frameworks and gateways each is documented to trace.

72 systems · 3 sections · 11 acquired, archived, stale or shut down · data as of 2026-10-31

Browse the table → JSON · CSV

Decide

Most referenced

Lowest tier first, then by how many other systems in the catalog reference each one.

  1. Langfuse T1 Acquired Memory ops as named spans + Agent Graphs
  2. Arize Phoenix T1 OSS LLM tracing + eval
  3. LangSmith T1 Memory mutations as distinct span types
  4. Arize AI T1 ML + LLM observability platform
  5. Braintrust T1 Eval + observability for AI
  6. Galileo T1 Acquired Eval + observability + guardrails
  7. Galileo (galileo.ai) (now Splunk Agent Observability) T1 Acquired Real-time intent / belief drift detection
  8. Confident AI / DeepEval T2 LLM eval framework + cloud
  9. OpenLLMetry (Traceloop) T2 OTel auto-instrumentation for LLMs
  10. Helicone T2 Maintenance LLM observability + gateway
  11. Ragas T2 RAG eval metrics
  12. Patronus AI T2 Archived Enterprise eval + guardrails

All 72 in the table →

Sections

Lifecycle

  • Active 58
  • Maintenance 1
  • Research code 2
  • Acquired 4
  • Inactive 1
  • Archived 5
  • Shut down 1

Tiers: T1 7 · T2 16 · T3 43 · T4 6 · T5 0

Recently changed

Everything that changed →

Research

Data and tools

  • JSON · CSV, flattened, one row per system, CC-BY-4.0
  • MCP tools: plan_stack, check_compatibility, search_records
  • CLI: landscape stack · landscape compat <ids…>
  • Data & API · Submit a system