LiteLLM
https://github.com/BerriAI/litellm
BerriAI's open-source LLM gateway — unified OpenAI-format API for 100+ providers; LiteLLM Proxy adds budgets, fallbacks, observability, rate-limiting.
Active · Verified 2026-10-27 · fresh as of 2026-10-27 · 27 cited facts, 4 with a third-party source · 1 recorded change · Suggest a correction
At a glance
- Type
- Universal LLM API + Proxy repo fix
- Tier
- T1
- Section
- Inference platforms & gateways
- Latest release
- v1.106.4 (2026-10-25) registry fix
- Funding
- BerriAI raised $1.6M seed 2023 (Y Combinator W23); commercial steward of LiteLLM OSS. third-party fix
- Freshness
- 2026-10-27 fix
- Layer
- model-infra repo fix
- SDK languages
- python, rest repo fix
- Protocols
- a2a, mcp-client, mcp-server, openai-compatible, opentelemetry vendor fix
- Compliance
- soc2-type2 vendor fix
- Developer activity
- OpenRank 255.53 (12-mo avg 203.13) · 1327 participants — 2026-09 third-party fix
When to use
Optimised for: searched not found
Anti-fit: searched not found
Pros & cons
Pros
searched not found
Cons
searched not found
Claims & capabilities
100+ LLM providers behind OpenAI-compatible API; widely used as proxy/gateway.
Technical surface
- API surface
- Python SDK, OpenAI-compatible proxy REST API
- Backend storage
- not applicable — not a memory product
- Deployment
- Self-hosted only — SDK and proxy; Enterprise self-hosted or air-gapped
- Embedding model
- not applicable — not a memory product
- Multi-tenancy
- searched not found
- MCP
- yes — MCP Gateway in LiteLLM Proxy (Streamable HTTP, SSE, stdio) with per-key/team access control
- A2A
- searched not found
- OpenTelemetry
- searched not found
Compare LiteLLM with…
Similar systems
Other inference platforms & gateways in the catalog, ranked by inbound references.
- OpenRouter T1
Unified API + marketplace for 300+ models — pay-as-you-go credits, automatic fallback, model arena. Used as default LLM router by many JS/TS agents.
- Together AI T1
Cloud platform for OSS LLM inference and fine-tuning — serverless inference, dedicated endpoints, fine-tuning API, training cluster.
- Fireworks AI T1
Fast inference platform for OSS LLMs — custom Cuda kernels, speculative decoding, multi-LoRA serving. Targets latency-sensitive production use cases.
- vLLM T1
Open-source LLM inference engine with PagedAttention — high-throughput batching, paged KV-cache. UC Berkeley origin; de-facto OSS inference stack.
- Portkey AI T2
AI gateway + observability — load balancing across 200+ LLMs, retries, caching, eval, prompts. Open-source Gateway + commercial Portkey Cloud. Acquired by Palo Alto Networks (closed about 2026-06); now sold as Prisma AIRS AI Gateway (GA 2026-07-16).
- Baseten T1
Inference platform for ML models — Truss package format, Chains for multi-model workflows, autoscaling GPU/CPU serving. Targeted at production teams shipping LLM/diffusion endpoints.
Sources (26 cited facts, 4 third-party)
Each fact links to the page it came from; the badge says who published it (how to read sources).
- What it is Universal LLM API + Proxy repo
- What it does & what's distinct BerriAI's open-source LLM gateway — unified OpenAI-format API for 100+ providers; LiteLL… repo
- Claims / benefits 100+ LLM providers behind OpenAI-compatible API; widely used as proxy/gateway. repo
- Created 2023 repo
- Latest release v1.106.4 (2026-10-25) registry
- License MIT (except enterprise/ directory, commercial license) repo
- GitHub BerriAI/litellm — ~62.1k stars, last commit 2026-10 repo
- Funding BerriAI raised $1.6M seed 2023 (Y Combinator W23); commercial steward of LiteLLM OSS. third-party
- Pricing OSS free; Enterprise custom pricing (30-day self-hosted trial key) vendor
- Deployment Self-hosted only — SDK and proxy; Enterprise self-hosted or air-gapped vendor
- Founders / pedigree Ishaan Jaffer, Krrish Dholakia; founded 2023 (YC W23). third-party
- Code/weights release Active — last commit 2026-05 · 46,707★ repo
- API surface Python SDK, OpenAI-compatible proxy REST API repo
- MCP support yes — MCP Gateway in LiteLLM Proxy (Streamable HTTP, SSE, stdio) with per-key/team acces… vendor
- commit-trajectory [{"ym":"2026-04","cum":38451},{"ym":"2026-05","cum":39238}] repo
- download-trajectory [{"ym":"2025-11","monthly":20498704,"cum":20498704},{"ym":"2025-12","monthly":63551549,"… registry
- adoption-accelerator yc-w23 third-party
- architecture-deployment-locus local repo
- licensing-bucket open-core repo
- positioning-platform-vs-product platform repo
- taxonomy-three-layer-rollup model-infra repo
- developer-activity-opendigger {"activity":3482.77,"as_of":"2026-09","openrank_12mo":203.13,"openrank_current":255.53,"… third-party
- compliance-certifications soc2-type2 vendor
- sdk-languages python,rest repo
- protocol-support a2a,mcp-client,mcp-server,openai-compatible,opentelemetry vendor
- maturity ga repo
What changed
Changes to this record's lifecycle, licence, release, pricing and hosting facts, from the catalog's git history. Subscribe (Atom)
- 2026-10-27
- License “MIT” → “MIT (except enterprise/ directory, commercial license)” source
- Latest release first recorded as “v1.106.4 (2026-10-25)” source
- Pricing first recorded as “OSS free; Enterprise custom pricing (30-day self-hosted trial key)” source
- Deployment first recorded as “Self-hosted only — SDK and proxy; Enterprise self-hosted or air-gapped” source
- MCP support first recorded as “yes — MCP Gateway in LiteLLM Proxy (Streamable HTTP, SSE, stdio) with per-key/team access control” source
Related systems
References (59)
- Agent2Agent Protocol (A2A) integrates with — LiteLLM docs describe an Agent Gateway for invoking agents over the A2A protocol. (verified 2026-10-29)
- AgentOps integrates with — LiteLLM docs have an AgentOps logging/observability integration page. (verified 2026-10-29)
- Amazon Bedrock Guardrails integrates with — LiteLLM docs have a Bedrock Guardrails page. (verified 2026-10-29)
- Amazon Nova family integrates with — LiteLLM docs have an Amazon Nova provider page. (verified 2026-10-29)
- Anthropic Claude (foundation models) integrates with — LiteLLM docs have an Anthropic provider page. (verified 2026-10-29)
- Anyscale integrates with — LiteLLM docs have an Anyscale provider page. (verified 2026-10-29)
- Argilla integrates with — LiteLLM docs have an Argilla logging/observability integration page. (verified 2026-10-29)
- Arize AI integrates with — LiteLLM docs have an Arize AI logging/observability integration page. (verified 2026-10-29)
- Arize Phoenix integrates with — LiteLLM docs have an Arize Phoenix logging/observability integration page. (verified 2026-10-29)
- AWS SageMaker integrates with — LiteLLM docs have an AWS SageMaker provider page. (verified 2026-10-29)
- Azure AI Content Safety Prompt Shields integrates with — LiteLLM docs have a Azure Content Safety guardrail page. (verified 2026-10-29)
- Baseten integrates with — LiteLLM docs have a Baseten provider page. (verified 2026-10-29)
- Bedrock AgentCore (AWS) integrates with — LiteLLM docs have a Bedrock AgentCore provider page. (verified 2026-10-29)
- Braintrust integrates with — LiteLLM docs have a Braintrust logging/observability integration page. (verified 2026-10-29)
- Cohere Command R+ / Command A integrates with — LiteLLM docs have a Cohere provider page. (verified 2026-10-29)
- Comet ML integrates with — LiteLLM docs have a Comet Opik logging/observability integration page. (verified 2026-10-29)
- Confident AI / DeepEval integrates with — LiteLLM docs have a DeepEval logging/observability integration page. (verified 2026-10-29)
- CrowdStrike Falcon Guardian (formerly Pangea AI Guard) integrates with — LiteLLM docs have a CrowdStrike AIDR guardrail page. (verified 2026-10-29)
- Datadog Agent Observability (LLM Observability) integrates with — LiteLLM docs have a Datadog observability page. (verified 2026-10-29)
- DeepSeek R1 / V3 family integrates with — LiteLLM docs have a DeepSeek provider page. (verified 2026-10-29)
- Enkrypt AI integrates with — LiteLLM proxy docs have an Enkrypt AI guardrail integration page. (verified 2026-10-29)
- Fireworks AI integrates with — LiteLLM docs have a Fireworks AI provider page. (verified 2026-10-29)
- Google Cloud Model Armor integrates with — LiteLLM docs have a Google Cloud Model Armor guardrail page. (verified 2026-10-29)
- Google Gemini 3 family integrates with — LiteLLM docs have a Gemini (Google AI Studio) provider page. (verified 2026-10-29)
- Guardrails AI integrates with — LiteLLM docs have a Guardrails AI guardrail page. (verified 2026-10-29)
- Helicone integrates with — LiteLLM docs have a Helicone provider page. (verified 2026-10-29)
- HiddenLayer AISec Platform 2.0 integrates with — LiteLLM proxy docs have a HiddenLayer guardrail integration page. (verified 2026-10-29)
- Humanloop integrates with — LiteLLM docs have a Humanloop logging/observability integration page. (verified 2026-10-29)
- Jamba (AI21 Labs) integrates with — LiteLLM docs have an AI21 provider page. (verified 2026-10-29)
- Jina AI Embeddings integrates with — LiteLLM docs have a Jina AI provider page. (verified 2026-10-29)
- Lakera Guard / Lakera Red integrates with — LiteLLM proxy docs have a Lakera AI guardrail integration page. (verified 2026-10-29)
- Langfuse integrates with — LiteLLM docs have a Langfuse logging/observability integration page. (verified 2026-10-29)
- LangGraph (orchestration) integrates with — LiteLLM docs have a LangGraph provider page. (verified 2026-10-29)
- LangSmith integrates with — LiteLLM docs have a LangSmith logging/observability integration page. (verified 2026-10-29)
- Lasso Security integrates with — LiteLLM proxy docs have a Lasso Security guardrail integration page. (verified 2026-10-29)
- Manus AI integrates with — LiteLLM docs have a Manus provider page. (verified 2026-10-29)
- Meta Llama 4 family integrates with — LiteLLM docs have a Meta Llama API provider page. (verified 2026-10-29)
- Milvus integrates with — LiteLLM docs have a Milvus vector stores provider page. (verified 2026-10-29)
- Mistral Large 2 / Mixtral family integrates with — LiteLLM docs have a Mistral AI provider page. (verified 2026-10-29)
- MLflow integrates with — LiteLLM docs have an MLflow logging/observability integration page. (verified 2026-10-29)
- Model Context Protocol (MCP spec) integrates with — LiteLLM docs describe its MCP gateway for exposing and calling MCP servers. (verified 2026-10-29)
- MongoDB Atlas Vector Search integrates with — LiteLLM docs have a MongoDB vector stores provider page. (verified 2026-10-29)
- OpenAI GPT family (GPT-5 / GPT-4o / o3 / o4) integrates with · depends on at runtime — 100+ LLM providers behind OpenAI-compatible API
- OpenRouter integrates with — LiteLLM docs have an OpenRouter provider page. (verified 2026-10-29)
- Pillar Security integrates with — LiteLLM proxy docs have a Pillar Security guardrail integration page. (verified 2026-10-29)
- Presidio integrates with — LiteLLM docs have a Presidio PII/PHI masking guardrail page. (verified 2026-10-29)
- Prompt Security (SentinelOne) integrates with — LiteLLM docs have a Prompt Security guardrail page. (verified 2026-10-29)
- PromptLayer integrates with — LiteLLM docs have a PromptLayer logging/observability integration page. (verified 2026-10-29)
- Protect AI integrates with — LiteLLM proxy docs have a Palo Alto Networks Prisma AIRS guardrail integration page. (verified 2026-10-29)
- Pydantic-AI integrates with — LiteLLM docs have a Pydantic AI Agents provider page. (verified 2026-10-29)
- Pydantic Logfire integrates with — LiteLLM docs have a Logfire observability page. (verified 2026-10-29)
- Reka Core / Flash / Edge integrates with — LiteLLM docs have a Reka provider page. (verified 2026-10-29)
- Replicate integrates with — LiteLLM docs have a Replicate provider page. (verified 2026-10-29)
- Together AI integrates with — LiteLLM docs have a Together AI provider page. (verified 2026-10-29)
- v0 (Vercel) integrates with — LiteLLM docs have a v0 provider page. (verified 2026-10-29)
- vLLM integrates with — LiteLLM docs have a vLLM provider page. (verified 2026-10-29)
- Voyage AI (MongoDB) integrates with — LiteLLM docs have a Voyage AI provider page. (verified 2026-10-29)
- Weights & Biases integrates with — LiteLLM docs have a Weights & Biases logging/observability integration page. (verified 2026-10-29)
- xAI Grok 4 integrates with — LiteLLM docs have an xAI provider page. (verified 2026-10-29)
Referenced by (37)
- agentgateway integrates with — agentgateway standalone docs have a "Migrate from LiteLLM" config-import page. (verified 2026-10-29)
- AgentOps integrates with — AgentOps docs have a LiteLLM integration page. (verified 2026-10-29)
- Aider (harness) depends on at runtime — uses any LiteLLM-compatible model. Memory via CONVENTIONS.md and .aider.conf.yml — author-edited markdown.
- Arize Phoenix integrates with — Arize Phoenix docs have a LiteLLM integration page. (verified 2026-10-29)
- Bifrost (Maxim AI) integrates with — Bifrost SDK integrations include a drop-in proxy page for the LiteLLM SDK. (verified 2026-10-29)
- Braintrust integrates with — Braintrust docs have a LiteLLM integration page. (verified 2026-10-29)
- Comet ML integrates with — Comet's Opik docs have a LiteLLM integration page. (verified 2026-10-29)
- Datadog Agent Observability (LLM Observability) integrates with — Datadog LLM Observability auto-instrumentation page lists a LiteLLM LLM integration. (verified 2026-10-29)
- DSPy integrates with — DSPy language model docs describe an optional LiteLLM compatibility engine (dspy[litellm]) for other providers. (verified 2026-10-29)
- Fireworks AI integrates with — Fireworks AI docs integration: LiteLLM (LLM Gateways; native fireworks_ai provider). (verified 2026-10-29)
- Google Agent Development Kit (ADK) integrates with — ADK docs have a LiteLLM model page for ADK agents. (verified 2026-10-29)
- Goose (Block) depends on at runtime — adjacent-infrastructure cell: LiteLLM for multi-model routing; MCP for tool integrations; Block-internal extensions (square/cash app workflows)
- Haystack (framework) integrates with — Haystack integrations directory has a LiteLLM integration page ("Use any of 100+ LLM providers with Haystack through LiteLLM"). (verified 2026-10-29)
- Helicone integrates with — Helicone docs have a LiteLLM integration page. (verified 2026-10-29)
- HoneyHive integrates with — HoneyHive integrations docs have a LiteLLM tracing/integration page. (verified 2026-10-29)
- Inspect AI integrates with — Inspect AI model-providers page documents a LiteLLM Proxy provider. (verified 2026-10-29)
- Invariant Guardrails integrates with — Invariant Gateway README has a LiteLLM integration section. (verified 2026-10-29)
- Laminar integrates with — Laminar integrations docs have a LiteLLM observability page. (verified 2026-10-29)
- LangChain (framework) integrates with — LangChain docs have a LiteLLM integration provider page (Unified interface for 100+ LLM APIs). (verified 2026-10-29)
- Langflow integrates with — Langflow components bundle: LiteLLM component. (verified 2026-10-29)
- Langfuse integrates with — Langfuse's integrations directory has a LiteLLM integration page. (verified 2026-10-29)
- LangSmith integrates with — LangSmith docs have a tracing integration page for LiteLLM. (verified 2026-10-29)
- LlamaIndex (framework) integrates with — LlamaIndex has an official LLM integration package llama-index-llms-litellm in its repo. (verified 2026-10-29)
- Mem0 integrates with — Mem0 docs have a LiteLLM LLM provider page for Mem0 open source. (verified 2026-10-29)
- OpenAI Agents SDK integrates with — OpenAI Agents SDK docs have a page on using any model via the LiteLLM integration (LitellmModel). (verified 2026-10-29)
- OpenInference integrates with — The OpenInference repo README lists an OpenInference instrumentation package for LiteLLM. (verified 2026-10-29)
- OpenLLMetry (Traceloop) integrates with — OpenLLMetry's What's Supported page lists LiteLLM as instrumented out of the box. (verified 2026-10-29)
- Phidata (now Agno) integrates with — Agno ships a litellm model provider module (agno.models.litellm). (verified 2026-10-29)
- Presidio integrates with — Presidio samples include "PII Masking LLM calls using LiteLLM". (verified 2026-10-29)
- Pydantic-AI integrates with — Pydantic AI "Other compatible APIs" docs have a LiteLLM section. (verified 2026-10-29)
- Pydantic Logfire integrates with — Pydantic Logfire integrations docs have a LiteLLM page. (verified 2026-10-29)
- Restate integrates with — Restate AI docs integration page: Restate & LiteLLM. (verified 2026-10-29)
- smolagents (Hugging Face) integrates with — smolagents models reference documents LiteLLMModel and LiteLLMRouterModel. (verified 2026-10-29)
- Strands Agents (AWS) integrates with — Strands Agents docs have a LiteLLM model provider page. (verified 2026-10-29)
- Together AI integrates with — Together AI docs third-party/agent integrations: LiteLLM (incl. Claude Code on Together via LiteLLM proxy). (verified 2026-10-29)
- Vercel AI Gateway integrates with — Vercel AI Gateway docs have a LiteLLM framework integration page. (verified 2026-10-29)
- Weights & Biases integrates with — W&B Weave docs have a LiteLLM integration page. (verified 2026-10-29)