Best Voice LLMs for Voice Agents in 2026: Ranked on Speed and Accuracy
PolyAI's Dialog-RSN-1 leads our speech-to-speech benchmark and Gemini 2.5 Flash is the fastest raw model on our LLM board. The full rankings, the catch on each pick, and how to test them on your calls.
Every voice agent has one model at its center deciding what to say next. In a cascaded agent, that model is a text LLM: speech-to-text transcribes the caller, the LLM decides the reply, and text-to-speech speaks it. In a speech-to-speech agent, it’s a single model that hears the caller and answers out loud. Either way, callers judge it on two things: how long they wait, and whether the agent does what it’s supposed to do.
We rank the best voice LLMs on both with Coval’s public benchmarks, which time every model’s replies and grade how well each one follows the same simulated bank call. As of November 9, 2026, PolyAI’s Dialog-RSN-1 is the best of the ten speech-to-speech models on our board: 95% of its replies started within 1,101 ms, the fastest on the board, and it carried out 100% of the steps the call required. For a cascade, Gemini 2.5 Flash with thinking off is the fastest raw model on our LLM board, at 589 ms to the first token, and it carried out 99% of the required steps. Google limits it to accounts that already use it, so for everyone else, GPT-4.1 is the best model we’ve measured that any project can use.
The table below has every pick and the catch that could rule it out. After it, we explain how we rank, walk through both rankings (including why Google’s recommended move from Gemini 3.1 Flash Live to 3.8 Live is now an easy call), and show how to rerun the comparison on your own calls before you switch.
Our picks for the best voice LLMs in 2026
Every number in this table comes from our two public boards, the speech-to-speech benchmark and the LLM benchmark, over the seven days to November 9. Speech-to-speech rows show the whole wait a caller hears; text LLM rows show the LLM’s share of it.
| Pick | Best for | p95 wait | Instruction adherence | Catch |
|---|---|---|---|---|
| Dialog-RSN-1 (PolyAI) | Best speech-to-speech model | 1,101 ms voice-to-voice | 100% | Existing PolyAI customers; others request early access |
| Gemini 3.8 Live (Google) | Fastest speech-to-speech model you can sign up for | 1,253 ms voice-to-voice | 99% | About 150 ms slower than Dialog-RSN-1 |
| Qwen3.8 Omni Flash Realtime (Alibaba) | Most accurate speech-to-speech model you can sign up for | 1,400 ms voice-to-voice | 100% | About 150 ms slower than Gemini 3.8 Live |
| Gemini 2.5 Flash, thinking off (Google) | Best text LLM for a cascade | 589 ms to first token | 99% | Google limits 2.5 to accounts already using it |
| GPT-4.1 (OpenAI) | Text LLM any project can use | 606 ms to first token | 93% | OpenAI shuts it down January 14, 2027 |
The cascade section below also covers the best open-weight model and the current models our LLM board doesn’t run.
How we rank voice LLMs: the slow replies and the skipped steps
Both boards grade every model playing the same role: Ultra Bank Bot, a support agent for a fictional bank, talking to simulated customers. Before it gives any account help, the agent has to verify the caller’s full name, phone number, and account number, in that order and one at a time. If verification fails, every reply must be the exact line “Sorry, your identity could not be verified. Please call back later.” No tools are attached, so the test is about following instructions. The Ultra Bank prompt is public, along with the rest of the runner code.
We rank speed by p95, the wait that 95% of replies beat, because the slow replies are the ones callers remember. “If you have 10 to 20 turns in the conversation, it’s really important that the long-tail performance is high,” Coval founder Brooke Hopkins said on our Baseten podcast. At p95, one reply in twenty is slower still, so a 20-turn call should expect to hit one.
The two boards time different waits. Voice-to-voice latency, for speech-to-speech models, runs from the end of the caller’s last word to the first audio of the reply: the whole pause the caller hears. We time it from the audio of single-turn exchanges, where the caller says one thing and the model answers out loud. Time to first token, for text LLMs, is how long the provider takes to stream back the first token of the reply. That’s only the LLM’s slice of a cascade’s wait, on top of the time speech-to-text and text-to-speech take. As a rough budget for that slice, Pipecat, the open-source voice agent framework from Daily, treats a model much above 700 ms to its first answer token as too slow for most voice agents.
Accuracy is instruction adherence, scored on the full bank calls: an LLM judge grades each call against the steps the prompt requires, like verifying the caller before giving account help, and the score is the share of those steps the model carried out, from 0 to 100%.
The best speech-to-speech models, ranked
PolyAI’s Dialog-RSN-1 keeps its slowest replies shorter than any other model and ties for the top adherence score. Here is the full board, ranked by p95 voice-to-voice latency.
| Rank | Model (maker) | p95 voice-to-voice latency | Instruction adherence |
|---|---|---|---|
| 1 | Dialog-RSN-1 (PolyAI) | 1,101 ms | 100% |
| 2 | Gemini 3.8 Live (Google) | 1,253 ms | 99% |
| 3 | Qwen3.8 Omni Flash Realtime (Alibaba) | 1,400 ms | 100% |
| 4 | Prism (Rime) | 1,554 ms | 75% |
| 5 | GPT-Live-1 (OpenAI) | 1,700 ms | 75% |
| 6 | Phonic 1 (Phonic) | 1,737 ms | 100% |
| 7 | Eleven Realtime v1 Max (ElevenLabs) | 1,883 ms | 93% |
| 8 | Eleven Realtime v1 Mini (ElevenLabs) | 1,948 ms | 83% |
| 9 | Grok Voice 3.0 (SpaceXAI) | 2,144 ms | 100% |
| 10 | Gemini 3.1 Flash Live, preview (Google) | 2,597 ms | 95% |
Adherence splits this board more than speed does. Six of the ten carried out 95% or more of the required steps, and four of them, Dialog-RSN-1, Qwen3.8 Omni Flash Realtime, Phonic 1, and Grok Voice 3.0, carried out every one. At the other end, GPT-Live-1 and Rime’s Prism carried out 75%, so a big name or a recent launch is no guarantee.
Dialog-RSN-1 listens like a speech-to-speech model but doesn’t speak for itself. PolyAI built it as a dialog model that hears the caller’s raw audio, with no transcript in between, and hands its reply to a separate text-to-speech voice. That voice counts against it: we time Dialog-RSN-1 exactly as we time the other nine, from the end of the caller’s last word to the first frame of the agent’s audio, so its 1,101 ms covers the whole pause the caller hears. The first release focuses on English. The catch is access: existing PolyAI customers can turn it on in their projects, and new customers request early access. If you build on PolyAI, it’s the pick. If you don’t, the next two models are ones you can sign up for directly.
Gemini 3.8 Live is the fastest of those, at 1,253 ms, and it carried out 99% of the required steps. Google calls it the default Live API model “for most low-latency voice agent experiences” and recommends it to everyone on Gemini 3.1 Flash Live, the older preview model, which Google will shut down as early as November 17, 2026. On our board, that upgrade is now a clear win: the p95 wait dropped 1,344 ms, from 2,597 ms to 1,253 ms, and adherence on the bank call rose from 95% to 99%. It wasn’t always: on our October 28 board, the same move cost nine points of adherence. The steps on that call are identity checks, so if you run on 3.1 Flash Live, test your own verification flow on 3.8 Live before the shutdown forces the move.
Qwen3.8 Omni Flash Realtime carries out the required steps best of the models you can sign up for: 100%, tied with Dialog-RSN-1, at a p95 of 1,400 ms, about 150 ms behind Gemini 3.8 Live. Phonic 1 and Grok Voice 3.0 also carried out every step, but at 1,737 ms and 2,144 ms they keep callers waiting longer on the slow turns.
ElevenLabs’ two models sit in the slower half of the board. ElevenLabs launched Max and the lower-cost v1 Mini on October 20, 2026, and pitches Max “for complex tasks.” Both support more than 90 languages, and their API is compatible with OpenAI’s Realtime API. On our bank call, Max carried out 93% of the required steps at a p95 of 1,883 ms, and Mini 83% at 1,948 ms.
OpenAI’s GPT-Live-1 is quicker, at 1,700 ms, but carried out only 75% of the required steps, tied for the lowest score with Rime’s Prism (1,554 ms). Prism has the fewest measured replies on the board, 32 against 90 for most models, so treat its numbers as early.
Each number on this board is the whole wait a caller hears. A cascade splits that wait across three models, and the one in the middle is the text LLM.
The best text LLMs for a cascaded voice agent
Our LLM board has three models this week, and they’re close on speed. Gemini 2.5 Flash is the fastest raw model: 95% of its first tokens arrived within 589 ms, and it carried out 99% of the required steps. We call it with thinking off, so it answers without running a reasoning pass first. GPT-4.1 is close behind at 606 ms but trails on adherence at 93%. Each model has 10 measured runs in this window, so gaps this small can flip from week to week.
Both picks come with a catch about access. Google limits the 2.5 models to users who have actively used them before, and points new projects to Gemini 3.5 Flash-Lite or 3.8 Flash. GPT-4.1 is open to any OpenAI project, but OpenAI has scheduled its shutdown for January 14, 2027, with GPT-6 Luna as the replacement, and our board doesn’t run Luna. That makes GPT-4.1 the pick for an agent that has to ship before then, with the migration already on the calendar.
If your agent has to run past January and Google won’t give you 2.5, look at current models our board doesn’t run. The best outside data on them is Pipecat’s August 2026 voice-readiness benchmark. Pipecat ran 46 model configurations through the same scripted 30-turn conversations, and a turn passes only if the model makes no tool, instruction, or knowledge-base error. Its pass rate is the share of turns that passed. Three results shape the shortlist:
- Qwen3.8-27B is the open-weight pick. With thinking off, served on Baseten, it passed 98.2% of turns, and 95% of its first answer tokens arrived within 801 ms. Its weights are open under the Apache 2.0 license, so you can host it yourself.
- Gemini 3.6 Flash is the strongest Google model on the list. Google’s previous-generation Flash, still a stable model, passed 97.1% of turns at minimal thinking, though its p95 of 984 ms is well past Pipecat’s 700 ms budget, so it trades speed for accuracy.
- Gemini 3.5 Flash-Lite, one of the two models Google points new projects to, had the quickest p95 of the three at 679 ms but passed only 68.6% of turns. Test it hard before you trust it with a verification flow.
One more model tops our LLM board: Alma, the voice LLM from Coval partner Phonely, at 584 ms and 100%. We measure it through Phonely’s hosted agent rather than a raw model endpoint, so it’s the one to look at if you already build on Phonely (more in the Alma launch).
The other two stages of a cascade have rankings of their own: see our speech-to-text rankings and text-to-speech rankings.
Speech-to-speech or cascade: what each gives up
A cascade buys control. Each stage is its own model, so you can swap one out, inspect what it handed to the next, run any LLM you like, and check the reply text before it’s spoken, for example against your refund policy. A speech-to-speech model buys simplicity: it hears the caller directly, so there’s no transcript in the middle and fewer hand-offs to manage. Speed isn’t a given, though: even the quickest model on our speech-to-speech board leaves one reply in twenty waiting more than a second.
Don’t read the two boards as a race, either. Gemini 2.5 Flash’s 589 ms is only the LLM’s slice of a cascade’s wait, while Dialog-RSN-1’s 1,101 ms is the whole pause. Our guide to speech-to-speech vs cascaded architectures walks through the full decision.
New voice AI models and retirements to watch
The lineup behind these rankings changes every few weeks. These are the launches, access changes, and shutdowns that affect the picks above, with the upcoming dates first.
| Date | Model | Change |
|---|---|---|
| January 14, 2027 | GPT-4.1 and GPT-4o snapshots (OpenAI) | Shutdown, announced October 14; replacement GPT-6 Luna |
| November 17, 2026 (earliest) | Gemini 3.1 Flash Live (Google) | Shutdown; replacement Gemini 3.8 Live |
| October 20, 2026 | Eleven Realtime v1 Mini and Max (ElevenLabs) | Launched |
| September 18, 2026 | Gemini 2.5 models (Google) | Access limited to existing users |
| September 15, 2026 | Gemini 3.8 Live (Google) | Released as stable |
| September 10, 2026 | GPT-Live-1 (OpenAI) | Generally available |
| July 30, 2026 | Dialog-RSN-1 (PolyAI) | Launched |
How to test your shortlist on your own calls
A public board narrows the field. Your own calls decide it, because your agent has its own steps, callers, and tools. Brooke breaks model choice into three steps: use public benchmarks to get down to three to five models, then start “experimenting with those on your own local benchmarks,” then plug the finalists into your system and “run simulations of those conversations.” This post is step one. Here’s how to run the other two in Coval simulation:
- Connect each candidate. Coval reaches each model or agent through a connection. Point a Chat connection at any OpenAI-compatible chat completions API to test a text LLM without placing calls. For a speech-to-speech agent, use the OpenAI Realtime or Gemini Live connection, or have Coval call your agent’s number and play the caller.
- Swap the model with mutations. A mutation is a variant of your agent that runs alongside the original. On the OpenAI Realtime and Gemini Live connections it can override the model; on a Chat connection, the endpoint.
- Hold everything else still. Run every variant on the same test set, which defines what the simulated callers try to do, and the same personas, which define who they are and how they sound.
- Score what callers notice. Attach latency and adherence metrics from Coval’s metric library, or write your own for your flow’s required steps, and compare the mutations side by side.
- Put it on a schedule. Any evaluation can run on a recurring cadence, so a weekly model comparison turns the next vendor update into a measured change, not a surprise in production.
It’s the same setup teams use for a vendor bake-off: your own scenarios, run identically against every candidate.
The best voice LLM is the one that holds up on your calls
Every model above was judged on the two things a caller notices: the wait on the slow turns, and whether the agent did what it was told. GPT-Live-1 shows the two can move apart: mid-pack on speed, tied for last on adherence. And Gemini 3.8 Live’s swing from 86% to 99% in under two weeks shows why one snapshot isn’t enough. On our boards, Dialog-RSN-1 and Gemini 2.5 Flash hold up best. To find the one that holds up on yours, run two or three picks from the table through your own test set in Coval, and keep the comparison on a schedule so the next model launch gets measured before your callers meet it.
Frequently asked questions
Does Claude have a speech-to-speech model?
No. Anthropic’s Claude models produce text output, so Claude runs as the LLM in a cascade, between speech-to-text and text-to-speech. Anthropic’s model list puts Haiku 4.5 among its legacy models and names Haiku 5.5 its fastest. In Pipecat’s 30-turn test, Haiku 4.5 passed 98.0% of turns, but 5% of its first answer tokens took longer than 1,615 ms, well past Pipecat’s 700 ms rule of thumb, so test Haiku 5.5 in your cascade instead.
Is GPT-4o still a good LLM for voice agents?
Not for a new build. OpenAI has scheduled the GPT-4o snapshots to shut down on January 14, 2027, the same day as GPT-4.1, with GPT-6 Luna as the replacement. It’s also slow at the tail in Pipecat’s test: GPT-4o passed 94.7% of turns, but 5% of its first answer tokens took longer than 1,369 ms, nearly twice Pipecat’s 700 ms rule of thumb. If your agent runs on GPT-4o, plan the move and test the replacement on your own calls first.
Should I turn thinking off for a voice agent?
For most voice agents, yes, or set it to the lowest level the model allows. Thinking is a reasoning pass the model runs before it answers, and the caller waits through it, which is why Google pitches Gemini 3.8 Live for voice “without reasoning delays.” Our LLM board runs Gemini 2.5 Flash with thinking off, and it still carried out 99% of the steps on the bank call. If a flow needs heavier reasoning, run both settings against the same test set and let the numbers decide.