vLLM
https://github.com/vllm-project/vllm
Open-source LLM inference engine with PagedAttention — high-throughput batching, paged KV-cache. UC Berkeley origin; de-facto OSS inference stack.
Active · Verified 2026-10-26 · fresh as of 2026-10-26 · 20 cited facts, 2 with a third-party source · 1 recorded change · Suggest a correction
At a glance
- Type
- Fast LLM inference engine repo fix
- Tier
- T1
- Section
- Inference platforms & gateways
- Latest release
- v0.32.0 (2026-10-18) repo fix
- License
- Apache-2.0 repo fix
- GitHub
- vllm-project/vllm — ~95.4k stars, last commit 2026-10 third-party fix
- Pricing
- Free (open source, Apache-2.0) repo fix
- Funding
- searched not found fix
- Freshness
- 2026-10-26 fix
- Layer
- model-infra repo fix
- Licensing
- open-source repo fix
- Protocols
- openai-compatible, opentelemetry vendor fix
- Developer activity
- OpenRank 508.69 (12-mo avg 464.49) · 2235 participants — 2026-09 third-party fix
When to use
Optimised for: searched not found
Anti-fit: searched not found
Pros & cons
Pros
searched not found
Cons
searched not found
Claims & capabilities
UC Berkeley Sky Computing Lab; de-facto OSS inference engine for LLMs; commercial backers include Anyscale, Neural Magic (acquired by Red Hat 2024).
Technical surface
- API surface
- searched not found
- Backend storage
- not applicable — not a memory product
- Deployment
- Self-hosted only — pip / Docker inference server with OpenAI-compatible API; managed hosting only via third-party clouds
- Embedding model
- not applicable — not a memory product
- Multi-tenancy
- searched not found
- MCP
- searched not found
- A2A
- searched not found
- OpenTelemetry
- searched not found
Compare vLLM with…
Similar systems
Other inference platforms & gateways in the catalog, ranked by inbound references.
- LiteLLM T1
BerriAI's open-source LLM gateway — unified OpenAI-format API for 100+ providers; LiteLLM Proxy adds budgets, fallbacks, observability, rate-limiting.
- OpenRouter T1
Unified API + marketplace for 300+ models — pay-as-you-go credits, automatic fallback, model arena. Used as default LLM router by many JS/TS agents.
- Together AI T1
Cloud platform for OSS LLM inference and fine-tuning — serverless inference, dedicated endpoints, fine-tuning API, training cluster.
- Fireworks AI T1
Fast inference platform for OSS LLMs — custom Cuda kernels, speculative decoding, multi-LoRA serving. Targets latency-sensitive production use cases.
- Portkey AI T2
AI gateway + observability — load balancing across 200+ LLMs, retries, caching, eval, prompts. Open-source Gateway + commercial Portkey Cloud. Acquired by Palo Alto Networks (closed about 2026-06); now sold as Prisma AIRS AI Gateway (GA 2026-07-16).
- Baseten T1
Inference platform for ML models — Truss package format, Chains for multi-model workflows, autoscaling GPU/CPU serving. Targeted at production teams shipping LLM/diffusion endpoints.
Sources (19 cited facts, 2 third-party)
Each fact links to the page it came from; the badge says who published it (how to read sources).
- What it is Fast LLM inference engine repo
- What it does & what's distinct Open-source LLM inference engine with PagedAttention — high-throughput batching, paged K… repo
- Claims / benefits UC Berkeley Sky Computing Lab; de-facto OSS inference engine for LLMs; commercial backer… repo
- Created 2023 repo
- Latest release v0.32.0 (2026-10-18) repo
- License Apache-2.0 repo
- GitHub vllm-project/vllm — ~95.4k stars, last commit 2026-10 third-party
- Pricing Free (open source, Apache-2.0) repo
- Deployment Self-hosted only — pip / Docker inference server with OpenAI-compatible API; managed hos… repo
- Founders / pedigree Woosuk Kwon, Zhuohan Li, Cody Yu et al; UC Berkeley; project launched 2023-06. repo
- Code/weights release Active — last commit 2026-05 · 79,799★ repo
- commit-trajectory [{"ym":"2026-04","cum":16229},{"ym":"2026-05","cum":16641}] repo
- architecture-deployment-locus local repo
- licensing-bucket open-source repo
- positioning-platform-vs-product platform repo
- taxonomy-three-layer-rollup model-infra repo
- developer-activity-opendigger {"activity":5771.09,"as_of":"2026-09","openrank_12mo":464.49,"openrank_current":508.69,"… third-party
- sdk-languages python repo
- protocol-support openai-compatible,opentelemetry vendor
What changed
Changes to this record's lifecycle, licence, release, pricing and hosting facts, from the catalog's git history. Subscribe (Atom)
- 2026-10-26
- Latest release first recorded as “v0.32.0 (2026-10-18)” source
- Pricing first recorded as “Free (open source, Apache-2.0)” source
- Deployment first recorded as “Self-hosted only — pip / Docker inference server with OpenAI-compatible API; managed hosting only via third-party clouds” source
Related systems
Referenced by (23)
- Agent Router (formerly Envoy AI Gateway) integrates with — Agent Router supported-providers page lists self-hosted vLLM. (verified 2026-10-29)
- AutoGen / AG2 integrates with — AG2 model configuration docs show configuring a self-hosted vLLM OpenAI-compatible endpoint. (verified 2026-10-29)
- Bifrost (Maxim AI) integrates with — Bifrost supported-providers docs have a provider page for vLLM. (verified 2026-10-29)
- Datadog Agent Observability (LLM Observability) integrates with — Datadog LLM Observability auto-instrumentation page lists a vLLM LLM integration. (verified 2026-10-29)
- Daytona integrates with — Daytona docs have a guide for running vLLM with Daytona sandboxes. (verified 2026-10-29)
- Google Agent Development Kit (ADK) integrates with — ADK docs have a vLLM model page for ADK agents. (verified 2026-10-29)
- Haystack (framework) integrates with — Haystack integrations directory has a vLLM integration page ("Use the vLLM inference engine with Haystack"). (verified 2026-10-29)
- Inspect AI integrates with — Inspect AI model-providers page documents a vLLM provider. (verified 2026-10-29)
- Kong AI Gateway integrates with — Kong AI Gateway docs have a vLLM AI-provider reference page. (verified 2026-10-29)
- LangChain (framework) integrates with — LangChain docs have a vLLM chat model integration page. (verified 2026-10-29)
- Langflow integrates with — Langflow components bundle: vLLM language model component. (verified 2026-10-29)
- Langfuse integrates with — Langfuse's integrations directory has a vLLM integration page. (verified 2026-10-29)
- LiteLLM integrates with — LiteLLM docs have a vLLM provider page. (verified 2026-10-29)
- LlamaIndex (framework) integrates with — LlamaIndex has an official LLM integration package llama-index-llms-vllm in its repo. (verified 2026-10-29)
- Mem0 integrates with — Mem0 docs have a vLLM LLM provider page for Mem0 open source. (verified 2026-10-29)
- Milvus integrates with — Milvus's integrations overview lists a Milvus + vLLM integration tutorial. (verified 2026-10-29)
- Modal integrates with — Modal's examples include deploying an OpenAI-compatible LLM service with vLLM on Modal. (verified 2026-10-29)
- Phidata (now Agno) integrates with — Agno ships a vllm model provider module (agno.models.vllm). (verified 2026-10-29)
- Portkey AI integrates with — Portkey docs have a vLLM LLM provider integration page. (verified 2026-10-29)
- Pydantic-AI integrates with — Pydantic AI "Other compatible APIs" docs have a vLLM section. (verified 2026-10-29)
- SGLang competes with — competitor to vLLM for OSS inference engines
- smolagents (Hugging Face) integrates with — smolagents models reference documents VLLMModel. (verified 2026-10-29)
- Strands Agents (AWS) integrates with — Strands Agents docs have a community vLLM model provider integration page. (verified 2026-10-29)