ElevenLabs on Tuesday launched Eleven Realtime, two speech-to-speech voice agent models that, for the first time, put best-in-class spoken-reasoning accuracy and sub-second response time in the same product — and did it by absorbing the entire voice agent stack into a single managed service that developers can no longer individually inspect, configure, or replace.

The launch matters for any developer currently assembling a voice agent from separate speech-recognition, language-model, and text-to-speech components. What ElevenLabs shipped today simplifies that assembly significantly. What it takes away is the ability to swap out any of those pieces — and the text transcript that most enterprise safety and compliance tooling is built around.

Breaking an Old Accuracy-Latency Tradeoff in Voice AI

Building a voice agent that sounds natural has historically required accepting one of two bad bargains: high accuracy with long latency, or fast response with degraded reasoning. The fastest models start talking before they have thought very hard; the most accurate ones think so hard that the pause is noticeable.

Eleven Realtime v1 Max starts speaking 0.57 seconds after receiving a question on average, with v1 Mini following at 0.62 seconds. Both numbers clear the 600-millisecond floor that voice AI researchers and developers widely use as the threshold below which an interaction begins to feel conversational rather than transactional.

On Big Bench Audio — a 1,000-question spoken-reasoning evaluation created by Artificial Analysis, the independent AI evaluation firm — v1 Max scored 98.8 percent. Artificial Analysis benchmarks speech-to-speech models across spoken reasoning, conversational dynamics, and agentic performance. V1 Mini scored 75.9 percent. The benchmark works by posing questions in audio form and scoring the spoken answer; it tests whether a model can understand and reason over what is said, not merely transcribe it.

The significance of those two numbers together — 98.8 percent accuracy and 570-millisecond latency — is that ElevenLabs claims no model currently on the Artificial Analysis Speech-to-Speech leaderboard is simultaneously faster and more accurate. Independent analysis confirmed the evaluation was conducted by Artificial Analysis, not ElevenLabs. The Pareto frontier claim is time-bounded — the voice AI competitive landscape shifts quickly — but it was accurate against the published leaderboard at launch.

For conversational dynamics, the results split between the two models. On Full-Duplex-Bench — which tests how a model handles pauses, interruptions, and backchannels, those "mm-hmm" and "right" vocalizations that signal a human is still listening without indicating they have finished speaking — v1 Mini scored 95.8 percent against v1 Max's 94.1 percent. For most natural-sounding AI assistants, tutoring applications, and interactive characters, v1 Mini's conversational fluency advantage likely outweighs the reasoning gap. For customer support, lead qualification, and multi-step tool-calling workflows — where the agent has to execute a sequence and reason through steps — v1 Max's 87.1 percent tool-use task completion rate against v1 Mini's 60.3 percent makes the choice straightforward.

What "Speech-to-Speech" Actually Means Here — and What It Doesn't

The launch framing describes these as end-to-end speech-to-speech models, which implies a single neural network that takes audio in and returns audio out. The technical reality is more specific. ElevenLabs itself confirms these models use a cascaded architecture internally — speech recognition, voice activity detection, turn-taking logic, a language model, and text-to-speech are integrated components, tuned to work together, but they are not a single unified model.

The distinction matters practically. What ElevenLabs has done is make the curation decisions — which components, how they are configured, how they interact — and packaged the result as a single API surface. Developers no longer have to make those decisions themselves. But developers also can no longer unmake them. The company offers a "Custom mode" in ElevenAgents where developers can select underlying language and speech models individually, but selecting a Realtime model means choosing the bundle. The realtime models documentation outlines what settings ElevenLabs manages and what settings remain configurable.

That matters for three reasons that the launch framing does not address.

First, there is no transcript. In the traditional pipeline — speech-to-text, then language model, then text-to-speech — the text representation at the middle step is available to the developer. That transcript is what most enterprise content moderation, compliance logging, and safety tooling operates on. An end-to-end or tightly bundled managed pipeline produces audio output without that intermediate text layer. Safety tooling built for text has to be rebuilt for audio. For developers deploying regulated enterprise workflows — financial services, healthcare, government — that is not a footnote.

Second, when something goes wrong in a voice agent call, developers need enough diagnostic information to identify which component failed. With Realtime's managed architecture, that visibility is limited to what ElevenLabs exposes through its monitoring tools. How much diagnostic information the API surfaces for production troubleshooting remains to be established in practice.

Third, there are meaningful capability gaps between the two Realtime models that do not appear in the benchmark tables. V1 Mini does not support Zero Retention Mode or HIPAA-eligible configurations. V1 Max supports both, but only in US and EU data residency environments — it is not available in India or Singapore. For global enterprises with data localization requirements, this is a hard geographic constraint.

How to Get There From OpenAI's Realtime API

The go-to-market argument ElevenLabs is making to developers is straightforward: if you already have code built against OpenAI's Realtime API, the migration cost is a single endpoint URL and a model name. The Eleven Realtime API runs over WebSocket and follows the OpenAI Realtime specification, which means OpenAI's own SDK works without modification.

That framing is accurate for WebSocket applications. For applications using WebRTC — which is common in browser-based voice experiences because WebRTC handles audio capture and playback natively — the migration requires changing the transport layer first, then handling 24kHz PCM16 audio capture and playback independently. It is not a free swap. ElevenLabs has published a migration guide covering these differences, and the company has stated that no WebRTC-compatible version of the Realtime API is available at launch.

The API also operates separately from ElevenAgents, the company's full managed agent platform. Phone and WhatsApp deployment channels, knowledge bases, guardrails, and production monitoring all live on the ElevenAgents platform side. Developers who start with the bare API and later want those features will need to rebuild on ElevenAgents.

For applications where the API is the right path, the two models support more than 90 languages and can use any of ElevenLabs' library of more than 17,000 voices.

Is a Market Running at 15 Million Conversations a Week Ready for This Shift?

The architectural question lands inside a market moving at considerable speed. ElevenLabs' ElevenAgents platform was handling more than 15 million conversations per week globally at time of the launch announcement — a figure the company said was three times the volume recorded in February 2026. Enterprise customers account for 55 percent of ElevenLabs' revenue, with its technology deployed in the daily operations of five of the world's ten largest insurers, according to the company.

The broader market ElevenLabs is competing in reached roughly $22.5 billion in 2026 and is projected to approach $61.7 billion by 2031, according to Mordor Intelligence. Venture investors poured more than $7 billion into voice AI startups in the first quarter of 2026 alone — far more than in any prior full year — according to data cited in Financial Times reporting.

Three weeks before the Realtime launch, ElevenLabs closed a $300 million employee tender offer that valued the company at $22 billion — double the $11 billion valuation set in February when it closed a $500 million Series D led by Sequoia Capital. Wellington Management and T. Rowe Price led the September transaction, joined by new investors including EQT, Goldman Sachs, GIC, Ontario Teachers' Pension Plan, and Sapphire Ventures. Among its Series D backers are Andreessen Horowitz, BlackRock, NVIDIA, and Salesforce, whose participation was announced in a May 2026 third close after the company surpassed $500 million in annual recurring revenue. The $22 billion figure places ElevenLabs among the ten most valuable AI companies outside of China.

How Does It Compare to Competitors?

The voice AI inference market Eleven Realtime enters has three visible competitors.

OpenAI's GPT-4o Realtime API established the dominant WebSocket protocol convention for speech-to-speech inference, which is precisely why ElevenLabs built API compatibility with it rather than a proprietary protocol. OpenAI prices audio at $32 to $64 per million input and output tokens.

Cartesia, a voice inference startup built on state-space model architecture rather than transformers, publishes 40-millisecond latency claims for its text-to-speech component. Independent benchmarking by Coval measured production median latency closer to 188 milliseconds. Cartesia does not yet publish comparable spoken-reasoning accuracy scores on Big Bench Audio.

Deepgram, which closed a $130 million Series C in January 2026 at a $1.3 billion valuation, focuses on real-time speech-to-text as infrastructure for developer-assembled voice agent stacks rather than end-to-end speech-to-speech models. It is not a direct competitor to Eleven Realtime but represents the alternative architecture: developers who prefer to source their own speech-to-text, language model, and text-to-speech components separately and manage the pipeline themselves.

ElevenLabs' differentiation case rests on the combination of benchmark accuracy, sub-600-millisecond latency, and platform breadth. Competing providers that score comparably on spoken-reasoning benchmarks currently exhibit longer latency; those with comparable latency score lower. Whether that Pareto advantage holds as competitors update their models will be the defining technical question of this product generation.

Is Voice the Next Interface?

The Eleven Realtime launch advances ElevenLabs' strategic move away from its origins as a text-to-speech component provider. The ElevenAgents platform, the September Eleven v4 and v4 Turbo text-to-speech release targeting roughly 100-millisecond inference for synthesis alone, and now Eleven Realtime together describe a company moving up the stack from voice synthesis component to full conversational AI platform.

Co-founder and CEO Mati Staniszewski has framed the ambition plainly: voice should be how AI interacts with people the way people interact with each other. In a February 2026 Al Jazeera interview, he also acknowledged the technology can be abused — for fraud, disinformation, and what he called psychological operations — a dual-use tension the company has addressed through audio tracing, voice cloning verification requirements, and its AI Speech Classifier, which detects ElevenLabs-generated audio.

For now, the immediate task for any developer considering Eleven Realtime is the question the benchmarks alone cannot answer: does the accuracy-latency performance hold up under production load, and does the platform surface enough diagnostic information to track down failures when it does not?

Frequently Asked Questions

What is Eleven Realtime and how does it differ from building a voice agent from components?

Eleven Realtime bundles speech recognition, voice activity detection, turn-taking logic, a language model, and text-to-speech synthesis into a single managed API surface. Developers select v1 Mini or v1 Max and get a configured voice agent from one model choice, without assembling or tuning individual components. The tradeoff is that developers cannot replace, inspect, or independently configure the underlying components — and there is no text transcript, which means existing text-based safety and compliance tooling does not apply directly.

How does ElevenLabs Realtime compare to OpenAI's Realtime API?

The APIs are protocol-compatible over WebSocket, so teams already using OpenAI's Realtime API can try Eleven Realtime by changing the endpoint URL and model name. ElevenLabs claims v1 Max outperforms OpenAI's Realtime models on Big Bench Audio spoken-reasoning accuracy while maintaining comparable latency, though direct head-to-head independent benchmarking is not yet published. The key practical difference: Eleven Realtime offers access to ElevenLabs' library of more than 17,000 voices and tighter integration with the ElevenAgents platform, while OpenAI's Realtime API offers broader ecosystem familiarity.

What do enterprises need to know about compliance and data residency before adopting Eleven Realtime?

V1 Mini does not support Zero Retention Mode or HIPAA-eligible configurations. V1 Max supports both, but only in US and EU data residency environments — it is not available in India or Singapore. The absence of a real-time text transcript means that standard text-layer content moderation, compliance logging, and audit trail tooling does not apply directly; enterprises regulated under HIPAA, GDPR, or sector-specific voice data rules should evaluate whether the platform's native monitoring tools satisfy their requirements before deploying at scale.

When does simplifying the voice agent stack create new risks for developers?

Bundling the voice agent pipeline eliminates integration complexity — but also eliminates visibility into which component caused a failure. If v1 Max gives an unexpected response on a production call, developers cannot inspect the speech-recognition output, the language model reasoning step, or the synthesis output independently. The platform's monitoring tools will need to surface enough diagnostic data for developers to troubleshoot failures without that component-level access. This is the most significant operational risk of the managed-stack approach, and it is worth testing in staging before production deployment.