OpenAI Realtime API
https://platform.openai.com/docs/guides/realtime
OpenAI's Realtime API for native speech-to-speech multimodal interaction with GPT-4o — launched Oct 2024 (DevDay). Skips the STT → LLM → TTS pipeline by passing speech directly to GPT-4o. The platform substrate that Vapi/Retell/Bland integrate as one of their LLM choices.
At a glance
- Type
- Speech-to-speech LLM API (OpenAI Realtime)
- Tier
- T1
- Section
- Voice agent platforms
- Created
- 2024-10
- Latest release
- not applicable — not OSS
- License
- not applicable — not OSS
- Pricing
- Audio token-based: $40/$80 per 1M input/output audio tokens (Oct-2024 launch)
- Funding
- OpenAI parent — $40B round Mar-2025 ($300B val)
Taxonomy
- storage
- kv
- retrieval
- injection
- persistence
- session
- update
- agent-controlled
- unit
- turn
- governance
- opaque
- conflict
- stateless
When to use
Optimised for: Native speech-to-speech with GPT-4o; lowest-latency LLM voice tier
Anti-fit: Stateless platform — caller builds orchestration, memory, telephony adapter
Pros & cons
Pros
Native speech-to-speech (skips STT/TTS); OpenAI brand + price/perf; WebRTC support.
Cons
Audio token pricing is expensive at scale; stateless (caller builds memory/orchestration); OpenAI-only model choice.
Claims & capabilities
Launched DevDay Oct-2024; GPT-4o speech-to-speech native; WebSocket + WebRTC support
Technical surface
- API surface
- WebSocket + WebRTC + REST
- Backend storage
- Stateless session API; caller manages durability
- Deployment
- API (Cloud); no self-host
- Embedding model
- not applicable — not a memory product
- Multi-tenancy
- API multi-tenant
- MCP
- searched not found
- A2A
- searched not found
- OpenTelemetry
- searched not found
Similar systems
Other voice agent platforms in the catalog, ranked by inbound references.
- Cartesia Sonic T1
Cartesia's Sonic TTS — state-space-model-based voice generation, lowest published TTS latency (~40ms). Cartesia raised $64M Series A Q1 2025 (Index + Lightspeed); founded by Mamba authors. Agent platform layer ('Cartesia Line') wraps Sonic with telephony + LLM.
- Vapi T1
Voice-AI developer platform — STT + LLM + TTS pipelines wired to telephony (Twilio / SIP). Used to build outbound + inbound phone agents at scale. Raised $20M Series A from Bessemer + Y Combinator (Dec 2024) at $130M valuation; revenue reported >$10M ARR by Q1 2025.
- Bland AI T1
Voice-AI platform with custom STT + LLM + TTS stack ('the whole pipeline'). Raised $40M Series A Apr-2024 (Lightspeed + Anthology). Distinguished by 'pathway' graphs (deterministic call flows) and aggressive outbound-calling positioning.
- CallRail Conversation Intelligence T2
CallRail's AI-driven conversation intelligence layer atop their call-tracking product. Marketing-vertical voice analytics — auto-tagging, sentiment, lead scoring. CallRail is profitable / private (Sageview-backed); not a voice-agent runtime but the BI / analytics adjacent layer.
- ElevenLabs Conversational AI T1
ElevenLabs launched Conversational AI Nov-2024 — STT + LLM + their own market-leading TTS in one pipeline. Combines vendor's voice cloning + multilingual TTS strengths with hosted call orchestration. ElevenLabs raised $180M Series C Jan-2025 at $3.3B val.
- Hume EVI T1
Hume's Empathic Voice Interface (EVI) — voice agent platform with vocal-affect understanding. Hume raised $50M Series B Mar-2024 (EQT). Distinguished from pure LLM voice agents by emotion/prosody modelling; uses Hume's proprietary speech-prosody models.