Catalog Systems Inference platforms & gateways Text Generation Inference (TGI)

Text Generation Inference (TGI)

https://github.com/huggingface/text-generation-inference

Hugging Face's LLM inference server (continuous batching, FlashAttention, quantization). Entered maintenance mode, then archived 2026-03-21; HF recommends vLLM, SGLang, llama.cpp or MLX instead.

Archived. repository archived (read-only) Source

Archived · Verified 2026-10-29 · fresh as of 2026-10-29 · 18 cited facts, 2 with a third-party source · 2 recorded changes · Suggest a correction

At a glance

Type
HF inference server repo fix
Tier
T2
Section
Inference platforms & gateways
Created
2022 repo fix
Latest release
v3.3.7 (2025-12-19) repo fix
License
Apache-2.0 repo fix
GitHub
huggingface/text-generation-inference — ~10.9k stars, archived 2026-03-21, last commit 2026-03 repo fix
Pricing
searched not found fix
Funding
searched not found fix
Freshness
2026-10-29 fix
Layer
model-infra repo fix
Licensing
open-source repo fix
Deployment locus
local repo fix
Platform vs product
platform repo fix
Accelerator
betaworks-camp-2016 third-party fix
Developer activity
OpenRank 0.45 (12-mo avg 9.37) · 5 participants — 2026-03 third-party fix
Maintainer health
0 issues in 90 days · 1 release in 12 months (latest 2025-12-19) · bus factor 2 — as of 2026-11-01 · concern: repository archived repo

When to use

Optimised for: searched not found

Anti-fit: searched not found

Pros & cons

Pros

searched not found

Cons

searched not found

Claims & capabilities

Apache-2.0 (was BSL-1.1 briefly 2023-07 then reverted to Apache); HF's production inference server.

Technical surface

API surface
searched not found
Backend storage
not applicable — not a memory product
Deployment
Self-hosted only — inference server (Docker)
Embedding model
not applicable — not a memory product
Multi-tenancy
searched not found
MCP
searched not found
A2A
searched not found
OpenTelemetry
searched not found

Similar systems

Other inference platforms & gateways in the catalog, ranked by inbound references.

  • LiteLLM T1

    BerriAI's open-source LLM gateway — unified OpenAI-format API for 100+ providers; LiteLLM Proxy adds budgets, fallbacks, observability, rate-limiting.

  • OpenRouter T1

    Unified API + marketplace for 300+ models — pay-as-you-go credits, automatic fallback, model arena. Used as default LLM router by many JS/TS agents.

  • Together AI T1

    Cloud platform for OSS LLM inference and fine-tuning — serverless inference, dedicated endpoints, fine-tuning API, training cluster.

  • Fireworks AI T1

    Fast inference platform for OSS LLMs — custom Cuda kernels, speculative decoding, multi-LoRA serving. Targets latency-sensitive production use cases.

  • vLLM T1

    Open-source LLM inference engine with PagedAttention — high-throughput batching, paged KV-cache. UC Berkeley origin; de-facto OSS inference stack.

  • Portkey AI T2

    AI gateway + observability — load balancing across 200+ LLMs, retries, caching, eval, prompts. Open-source Gateway + commercial Portkey Cloud. Acquired by Palo Alto Networks (closed about 2026-06); now sold as Prisma AIRS AI Gateway (GA 2026-07-16).

Sources (17 cited facts, 2 third-party)

Each fact links to the page it came from; the badge says who published it (how to read sources).

What changed

Changes to this record's lifecycle, licence, release, pricing and hosting facts, from the catalog's git history. Subscribe (Atom)

  1. 2026-10-27
    • Latest release first recorded as “v3.3.7 (2025-12-19)” source
    • Deployment first recorded as “Self-hosted only — inference server (Docker)” source
    #272
  2. 2026-05-16
    • Lifecycle first recorded as “archived”
    #56

Related systems

Referenced by (1)

  • LlamaIndex (framework) integrates with — LlamaIndex has an official LLM integration package llama-index-llms-text-generation-inference in its repo. (verified 2026-10-29)