SGLang
https://github.com/sgl-project/sglang
Fast LLM serving framework with RadixAttention (cache reuse across requests with shared prefixes) + structured programming primitives. UC Berkeley.
Active · Verified 2026-10-27 · fresh as of 2026-10-27 · 20 cited facts, 1 with a third-party source · 1 recorded change · Suggest a correction
At a glance
- Type
- Structured LLM-program runtime repo fix
- Tier
- T2
- Section
- Inference platforms & gateways
- Latest release
- v0.5.22 (2026-10-16) — sglang registry fix
- License
- Apache-2.0 repo fix
- Pricing
- searched not found fix
- Funding
- searched not found fix
- Freshness
- 2026-10-27 fix
- Layer
- model-infra repo fix
- Licensing
- open-source repo fix
- Protocols
- openai-compatible vendor fix
- Developer activity
- OpenRank 447.67 (12-mo avg 364.63) · 1456 participants — 2026-09 third-party fix
When to use
Optimised for: searched not found
Anti-fit: searched not found
Pros & cons
Pros
searched not found
Cons
searched not found
Claims & capabilities
LMSYS-affiliated; competitor to vLLM for OSS inference engines; Apache-2.0.
Technical surface
- API surface
- searched not found
- Backend storage
- not applicable — not a memory product
- Deployment
- Self-hosted only — LLM serving framework
- Embedding model
- not applicable — not a memory product
- Multi-tenancy
- searched not found
- MCP
- searched not found
- A2A
- searched not found
- OpenTelemetry
- searched not found
Compare SGLang with…
Similar systems
Other inference platforms & gateways in the catalog, ranked by inbound references.
- LiteLLM T1
BerriAI's open-source LLM gateway — unified OpenAI-format API for 100+ providers; LiteLLM Proxy adds budgets, fallbacks, observability, rate-limiting.
- OpenRouter T1
Unified API + marketplace for 300+ models — pay-as-you-go credits, automatic fallback, model arena. Used as default LLM router by many JS/TS agents.
- Together AI T1
Cloud platform for OSS LLM inference and fine-tuning — serverless inference, dedicated endpoints, fine-tuning API, training cluster.
- Fireworks AI T1
Fast inference platform for OSS LLMs — custom Cuda kernels, speculative decoding, multi-LoRA serving. Targets latency-sensitive production use cases.
- vLLM T1
Open-source LLM inference engine with PagedAttention — high-throughput batching, paged KV-cache. UC Berkeley origin; de-facto OSS inference stack.
- Portkey AI T2
AI gateway + observability — load balancing across 200+ LLMs, retries, caching, eval, prompts. Open-source Gateway + commercial Portkey Cloud. Acquired by Palo Alto Networks (closed about 2026-06); now sold as Prisma AIRS AI Gateway (GA 2026-07-16).
Sources (19 cited facts, 1 third-party)
Each fact links to the page it came from; the badge says who published it (how to read sources).
- What it is Structured LLM-program runtime repo
- What it does & what's distinct Fast LLM serving framework with RadixAttention (cache reuse across requests with shared … repo
- Claims / benefits LMSYS-affiliated; competitor to vLLM for OSS inference engines; Apache-2.0. repo
- Created 2024 repo
- Latest release v0.5.22 (2026-10-16) — sglang registry
- License Apache-2.0 repo
- GitHub sgl-project/sglang — ~38.3k stars, last commit 2026-10 repo
- Deployment Self-hosted only — LLM serving framework registry
- Founders / pedigree Lianmin Zheng, Liangsheng Yin et al; UC Berkeley LMSYS; launched 2024-01. repo
- Code/weights release Active — last commit 2026-05 · 27,715★ repo
- commit-trajectory [{"ym":"2026-04","cum":12163},{"ym":"2026-05","cum":12660}] repo
- download-trajectory [{"ym":"2025-11","monthly":1088380,"cum":1088380},{"ym":"2025-12","monthly":10106235,"cu… registry
- architecture-deployment-locus local repo
- licensing-bucket open-source repo
- positioning-platform-vs-product platform repo
- taxonomy-three-layer-rollup model-infra repo
- developer-activity-opendigger {"activity":4261.56,"as_of":"2026-09","openrank_12mo":364.63,"openrank_current":447.67,"… third-party
- sdk-languages python repo
- protocol-support openai-compatible vendor
What changed
Changes to this record's lifecycle, licence, release, pricing and hosting facts, from the catalog's git history. Subscribe (Atom)
Related systems
References (1)
- vLLM competes with — competitor to vLLM for OSS inference engines
Referenced by (7)
- Bifrost (Maxim AI) integrates with — Bifrost supported-providers docs have a provider page for SGLang. (verified 2026-10-29)
- Daytona integrates with — Daytona docs have a guide for running SGLang with Daytona sandboxes. (verified 2026-10-29)
- DSPy integrates with — DSPy language model docs show serving local LMs on a GPU server with SGLang. (verified 2026-10-29)
- Inspect AI integrates with — Inspect AI model-providers page documents a SGLang provider. (verified 2026-10-29)
- LlamaIndex (framework) integrates with — LlamaIndex has an official LLM integration package llama-index-llms-sglang in its repo. (verified 2026-10-29)
- Modal integrates with — Modal's examples include serving models with SGLang on Modal. (verified 2026-10-29)
- Strands Agents (AWS) integrates with — Strands Agents docs have a community SGLang model provider integration page. (verified 2026-10-29)