Ragas
https://github.com/vibrantlabsai/ragas
Open-source eval framework for RAG pipelines — faithfulness, answer-relevance, context-precision, context-recall. Widely used as a baseline RAG eval kit.
Active · Verified 2026-10-28 · fresh as of 2026-10-28 · 19 cited facts, 1 with a third-party source · 2 recorded changes · Suggest a correction
At a glance
- Type
- RAG eval metrics repo fix
- Tier
- T2
- Latest release
- v0.4.3 (2026-01-13) — ragas repo fix
- License
- Apache-2.0 repo fix
- Pricing
- searched not found fix
- Funding
- searched not found fix
- Freshness
- 2026-10-28 fix
- Layer
- agent-infra repo fix
- Licensing
- open-source repo fix
- Developer activity
- OpenRank 4.37 (12-mo avg 6.51) · 55 participants — 2026-09 third-party fix
When to use
Optimised for: searched not found
Anti-fit: searched not found
Pros & cons
Pros
searched not found
Cons
searched not found
Claims & capabilities
Apache-2.0 RAG-eval framework by Exploding Gradients; widely cited in 2024 LLM eval literature.
Technical surface
- API surface
- searched not found
- Backend storage
- not applicable — not a memory product
- Deployment
- Self-hosted only — library
- Embedding model
- not applicable — not a memory product
- Multi-tenancy
- searched not found
- MCP
- searched not found
- A2A
- searched not found
- OpenTelemetry
- searched not found
Similar systems
Other evaluation & observability platforms in the catalog, ranked by inbound references.
- Arize Phoenix T1
Arize AI's open-source LLM observability tool — OpenTelemetry-native LLM tracing, evals, datasets. Arize commercial offers Phoenix + Surveyor.
- Arize AI T1
Production ML monitoring + LLM observability (Arize AX) — drift, eval, root-cause. Pre-LLM observability leader; now LLM-pivoted.
- Braintrust T1
Eval-first LLM platform — datasets, eval runs, prompt playground, tracing. Founded by ex-Impira (acquired by Dropbox) team. Used by Notion, Brex, AirOps.
- Confident AI / DeepEval T2
Open-source DeepEval framework + Confident AI cloud — pytest-style eval runs, hallucination/relevance/bias metrics, datasets.
- OpenLLMetry (Traceloop) T2
Traceloop's open-source OpenTelemetry auto-instrumentation for LLM workflows — works with any OTel-compatible backend (Datadog, New Relic, Grafana, etc.).
- Helicone T2
Open-source LLM observability and AI gateway — one-line proxy integration captures every LLM call; sessions, custom properties, evals. Acquired by Mintlify in Mar 2026 and placed in maintenance mode; helicone.ai now states it is no longer part of Mintlify.
Sources (18 cited facts, 1 third-party)
Each fact links to the page it came from; the badge says who published it (how to read sources).
- What it is RAG eval metrics repo
- What it does & what's distinct Open-source eval framework for RAG pipelines — faithfulness, answer-relevance, context-p… repo
- Claims / benefits Apache-2.0 RAG-eval framework by Exploding Gradients; widely cited in 2024 LLM eval lite… repo
- Created 2023 repo
- Latest release v0.4.3 (2026-01-13) — ragas repo
- License Apache-2.0 repo
- GitHub vibrantlabsai/ragas — ~16.2k stars, last commit 2026-02 repo
- Deployment Self-hosted only — library repo
- Founders / pedigree Shahul ES, Jithin James; founded 2023. repo
- Code/weights release Active — last commit 2026-02 · 13,887★ repo
- commit-trajectory [{"ym":"2023-11","cum":189},{"ym":"2023-12","cum":213},{"ym":"2024-01","cum":266},{"ym":… repo
- download-trajectory [{"ym":"2025-11","monthly":369085,"cum":369085},{"ym":"2025-12","monthly":715809,"cum":1… registry
- architecture-deployment-locus local repo
- licensing-bucket open-source repo
- positioning-platform-vs-product platform repo
- taxonomy-three-layer-rollup agent-infra repo
- developer-activity-opendigger {"activity":91.7,"as_of":"2026-09","openrank_12mo":6.51,"openrank_current":4.37,"partici… third-party
- sdk-languages python repo
What changed
Changes to this record's lifecycle, licence, release, pricing and hosting facts, from the catalog's git history. Subscribe (Atom)
Related systems
Referenced by (8)
- Arize Phoenix integrates with — Arize Phoenix docs have a Ragas integration page. (verified 2026-10-29)
- Comet ML integrates with — Comet's Opik docs have a Ragas integration page. (verified 2026-10-29)
- Haystack (framework) integrates with — Haystack integrations directory has a Ragas integration page ("Use the Ragas evaluation framework to calculate model-based metrics"). (verified 2026-10-29)
- Helicone integrates with — Helicone docs have a Ragas integration page. (verified 2026-10-29)
- Langfuse integrates with — Langfuse's integrations directory has a Ragas integration page. (verified 2026-10-29)
- Milvus integrates with — Milvus's integrations overview lists a Milvus + Ragas integration tutorial. (verified 2026-10-29)
- Pydantic-AI integrates with — Pydantic Evals third-party integrations docs have a Ragas section. (verified 2026-10-29)
- Weaviate integrates with — Weaviate's integrations / model-provider docs list Ragas as an integration. (verified 2026-10-29)