OctoAI (now NVIDIA)
Inference cloud spun out of OctoML — acquired by NVIDIA in 2024. Originally TVM-compiler heritage; offered API for OSS LLM/diffusion serving.
Acquired · Verified 2026-10-29 · fresh as of 2026-10-29 · 10 cited facts, 0 with a third-party source · 1 recorded change · Suggest a correction
At a glance
- Type
- GenAI inference (acquired) vendor fix
- Tier
- T2
- Section
- Inference platforms & gateways
- Created
- 2019 (as OctoML); rebrand 2023 vendor fix
- Latest release
- octoai Python SDK 1.11.0 (2024-09-18) — final release before shutdown registry fix
- License
- searched not found fix
- GitHub
- searched not found vendor fix
- Pricing
- searched not found fix
- Funding
- Acquired by NVIDIA in 2024-10 (terms undisclosed; reportedly ~$250M); previously raised $135M total through Series C 2023-12 (Tiger Global led). vendor fix
- Freshness
- 2026-10-29 fix
- Layer
- model-infra repo fix
When to use
Optimised for: searched not found
Anti-fit: searched not found
Pros & cons
Pros
searched not found
Cons
searched not found
Claims & capabilities
Acquired by NVIDIA 2024-10.
Technical surface
- API surface
- searched not found
- Backend storage
- not applicable — not a memory product
- Deployment
- searched not found
- Embedding model
- not applicable — not a memory product
- Multi-tenancy
- searched not found
- MCP
- searched not found
- A2A
- searched not found
- OpenTelemetry
- searched not found
Similar systems
Other inference platforms & gateways in the catalog, ranked by inbound references.
- LiteLLM T1
BerriAI's open-source LLM gateway — unified OpenAI-format API for 100+ providers; LiteLLM Proxy adds budgets, fallbacks, observability, rate-limiting.
- OpenRouter T1
Unified API + marketplace for 300+ models — pay-as-you-go credits, automatic fallback, model arena. Used as default LLM router by many JS/TS agents.
- Together AI T1
Cloud platform for OSS LLM inference and fine-tuning — serverless inference, dedicated endpoints, fine-tuning API, training cluster.
- Fireworks AI T1
Fast inference platform for OSS LLMs — custom Cuda kernels, speculative decoding, multi-LoRA serving. Targets latency-sensitive production use cases.
- vLLM T1
Open-source LLM inference engine with PagedAttention — high-throughput batching, paged KV-cache. UC Berkeley origin; de-facto OSS inference stack.
- Portkey AI T2
AI gateway + observability — load balancing across 200+ LLMs, retries, caching, eval, prompts. Open-source Gateway + commercial Portkey Cloud. Acquired by Palo Alto Networks (closed about 2026-06); now sold as Prisma AIRS AI Gateway (GA 2026-07-16).
Sources (9 cited facts, 0 third-party)
Each fact links to the page it came from; the badge says who published it (how to read sources).
- What it is GenAI inference (acquired) vendor
- What it does & what's distinct Inference cloud spun out of OctoML — acquired by NVIDIA in 2024. Originally TVM-compiler… vendor
- Claims / benefits Acquired by NVIDIA 2024-10. vendor
- Created 2019 (as OctoML); rebrand 2023 vendor
- Latest release octoai Python SDK 1.11.0 (2024-09-18) — final release before shutdown registry
- Funding Acquired by NVIDIA in 2024-10 (terms undisclosed; reportedly ~$250M); previously raised … vendor
- HQ US vendor
- Founders / pedigree Luis Ceze (CEO; UWash + creator of Apache TVM), Tianqi Chen (XGBoost / TVM); founded 201… vendor
- taxonomy-three-layer-rollup model-infra repo
What changed
Changes to this record's lifecycle, licence, release, pricing and hosting facts, from the catalog's git history. Subscribe (Atom)
Related systems
Referenced by (1)
- LlamaIndex (framework) integrates with — LlamaIndex has an official LLM integration package llama-index-llms-octoai in its repo. (verified 2026-10-29)