LiteLLM
https://github.com/BerriAI/litellm
BerriAI's open-source LLM gateway — unified OpenAI-format API for 100+ providers; LiteLLM Proxy adds budgets, fallbacks, observability, rate-limiting.
At a glance
- Type
- Universal LLM API + Proxy
- Tier
- T1
- Section
- Inference platforms & gateways
- Created
- 2023
- Latest release
- searched not found
- License
- MIT
- GitHub
- searched not found
- Pricing
- searched not found
- Funding
- BerriAI raised $1.6M seed 2023 (Y Combinator W23); commercial steward of LiteLLM OSS.
Taxonomy
- storage
- n/a
- retrieval
- n/a
- persistence
- n/a
- update
- n/a
- unit
- n/a
- governance
- n/a
- conflict
- n/a
When to use
Optimised for: searched not found
Anti-fit: searched not found
Pros & cons
Pros
searched not found
Cons
searched not found
Claims & capabilities
~14k stars; 100+ LLM providers behind OpenAI-compatible API; widely used as proxy/gateway.
Technical surface
- API surface
- searched not found
- Backend storage
- not applicable — not a memory product
- Deployment
- searched not found
- Embedding model
- not applicable — not a memory product
- Multi-tenancy
- searched not found
- MCP
- searched not found
- A2A
- searched not found
- OpenTelemetry
- searched not found
Compare LiteLLM with…
Similar systems
Other inference platforms & gateways in the catalog, ranked by inbound references.
- vLLM T1
Open-source LLM inference engine with PagedAttention — high-throughput batching, paged KV-cache. UC Berkeley origin; de-facto OSS inference stack.
- Anyscale T1
Commercial platform built on Ray — distributed training, fine-tuning, serving. Anyscale Endpoints provides hosted OSS inference; ray cluster is the substrate underneath OpenAI's training stack.
- Baseten T1
Inference platform for ML models — Truss package format, Chains for multi-model workflows, autoscaling GPU/CPU serving. Targeted at production teams shipping LLM/diffusion endpoints.
- Fireworks AI T1
Fast inference platform for OSS LLMs — custom Cuda kernels, speculative decoding, multi-LoRA serving. Targets latency-sensitive production use cases.
- Modal T1
Serverless cloud for AI/ML — Python-decorator workflow defines GPU/CPU functions deployed to managed infra; widely used for training jobs, batch inference, and agent backends.
- OctoAI (now NVIDIA) T2
Inference cloud spun out of OctoML — acquired by NVIDIA in 2024. Originally TVM-compiler heritage; offered API for OSS LLM/diffusion serving.
Related systems
References (1)
- OpenAI GPT family (GPT-5 / GPT-4o / o3 / o4) depends on at runtime — ~14k stars; 100+ LLM providers behind OpenAI-compatible API; widely used as proxy/gateway.
Referenced by (2)
- Aider (harness) depends on at runtime — po-map for context; uses any LiteLLM-compatible model. Memory via CONVENTIONS.md and .aider.conf.yml — author-edited mark
- Goose (Block) depends on at runtime — adjacent-infrastructure cell: LiteLLM for multi-model routing; MCP for tool integrations; Block-internal extensions (square/cash app workflows)