The Lowest Latency Voice AI Stack in 2026: STT, LLM, and TTS Picks From Our Benchmarks

On our benchmarks, the fastest accurate cascade's three models add up to about 940 ms at p95, and the LLM is 800 ms of it. Here are the picks and what that sum leaves out.

If you’re building a cascaded voice agent (speech-to-text, or STT, to transcribe the caller, an LLM to decide the reply, and text-to-speech, or TTS, to speak it), every provider will tell you its piece is the fastest. What matters is which combination of the three is fastest without trading away accuracy, and how much of the caller’s wait it accounts for.

Quick answer: As of November 4, 2026, the lowest latency voice AI stack on our benchmarks that doesn’t trade away accuracy is Qwen3 ASR Fast (served by Nari Labs) for speech-to-text, Gemini 2.5 Flash for the LLM, and Fluxions vui for text-to-speech. Their p95 times add up to about 940 ms, and the LLM’s first token is 800 ms of that, so choose the LLM first. Google has closed Gemini 2.5 Flash to new API users and routes it to Gemini 3.5 Flash-Lite from December 1, so keep the two speech picks and choose a current LLM by timing its first token on your own prompt.

That 940 ms is what the three models take, not what a caller waits. Turn detection, the handoff from the LLM to the voice, and the phone network all sit outside it. Below are three stacks, each best for something different, then why the LLM decides your budget, what our boards leave out, how the fastest speech-to-speech model compares, and how to measure your own agent end to end.

The lowest latency voice AI stacks on our benchmarks

Every number here comes from Coval’s public benchmarks as of November 4, 2026: speech-to-text and text-to-speech from each model’s 30-day page, and the LLMs from our LLM board. Each is a p95, the time 95% of requests beat. Speech-to-text is timed from the end of the caller’s speech to the final transcript, the LLM from the request to its first token, and text-to-speech from the synthesis request to the first sound a caller would hear.

All three stacks clear our starting bar for a live phone agent: under 5% word error rate (the share of words that come out wrong) for speech-to-text, and 2% or lower for text-to-speech, which we score by transcribing the synthesized audio. We compare them on model time, our name for the three p95s added together.

Best forSpeech-to-textLLM (first token)Text-to-speechModel time
Fastest without trading accuracyQwen3 ASR Fast 71 msGemini 2.5 Flash 800 msvui 69 ms940 ms
Lowest model timeQwen3 ASR Fast 71 msGPT-4.1 630 msvui 69 ms770 ms
Most accurate transcriptionUniversal 3.6 Pro 190 msGemini 2.5 Flash 800 msvui 69 ms1,059 ms

The first row is our pick. The second swaps in GPT-4.1, a faster LLM that follows its instructions less reliably (more on that trade below). The third swaps in AssemblyAI’s Universal 3.6 Pro, which has the lowest word error rate on our speech-to-text board, 2.2% against Qwen3 ASR Fast’s 2.6%, for 119 ms more.

The fastest speech-to-text and text-to-speech picks each have close alternates that clear the same bar. For speech-to-text, Soniox STT Rt v6 returns its final transcript in 98 ms and Inworld STT-1 in 109 ms. For text-to-speech, Qwen3 TTS Fast (99 ms), Inworld TTS Flash 2 (102 ms), and Gradium (106 ms) all start speaking within 40 ms of vui. Our guides to the best STT providers and best TTS providers cover the full lineups.

Read the model time for what it is: three p95s added together. It isn’t a measured turn. And because one model’s slow requests rarely land on the same turn as another’s, it isn’t the p95 of a real turn either. It’s a budget for the models’ share of the wait, and inside that budget one model dominates.

The LLM decides most of your latency budget

In the fastest stack, the LLM’s first token takes 800 of the 940 ms. Speech-to-text and text-to-speech together take 140. If a stack guide has told you to shop hardest for the voice, our boards say otherwise: the good speech models are already fast enough that swapping between them barely moves the total.

Look at what each swap costs. Trading vui for Gradium adds 37 ms at p95. Trading Qwen3 ASR Fast for the most accurate transcription on the board adds 119. Changing the LLM moves the total by 170, more than either speech swap.

Stacked bar chart of p95 model time for three stacks. Fastest without trading accuracy (Qwen3 ASR Fast, Gemini 2.5 Flash, vui): 71 ms speech-to-text, 800 ms LLM first token and 69 ms text-to-speech, 940 ms total. Lowest model time (Qwen3 ASR Fast, GPT-4.1, vui): 71, 630 and 69, 770 ms total. Most accurate transcription (Universal 3.6 Pro, Gemini 2.5 Flash, vui): 190, 800 and 69, 1,059 ms total. Three models added up, not a measured turn.

The LLM’s first token is most of each stack’s model time. p95 per model from Coval’s benchmarks as of November 4, 2026 (speech-to-text and text-to-speech over 30 days). Model time is the sum of the three models, not a measured turn.

That 170 ms is the GPT-4.1 trade. Our LLM board scores each model on instruction adherence: the share of a test case’s expected behaviors that an LLM judge finds the model performed, scored across the whole conversation. On the board’s test, a multi-turn banking conversation, GPT-4.1 reaches its first token at 630 ms but performs 93% of the expected behaviors. Gemini 2.5 Flash takes 800 ms and performs 98%. For an agent that verifies identities or moves money, five points of adherence are worth far more than 170 ms of silence, so we’d pay the 170 ms.

Phonely’s Alma, a model built for phone calls, tops the board at 502 ms and 99% adherence. Our board reaches it through Phonely’s own hosted call API rather than the shared prompt GPT-4.1 and Gemini 2.5 Flash get, though, so it isn’t a like-for-like pick for a stack you assemble yourself.

Gemini 2.5 Flash and GPT-4.1 are also on their way out, so neither is the model to start a new build on. Google has closed gemini-2.5-flash to new API users, and from December 1, 2026, its requests route automatically to gemini-3.5-flash-lite. OpenAI lists GPT-4.1 as deprecated. What carries over to whichever current model you choose is the shape of the budget: on our board, the three LLMs’ first tokens take about 500 to 800 ms at p95, several times the 140 ms both speech models take together. So shortlist current LLMs, don’t give up instruction adherence for speed, and time each one’s first token on your own prompt, from where your agent runs. Coval reports that number on your agent’s own calls as the LLM Time to First Byte trace metric.

What the 940 ms leaves out

“With voice AI, you’re fighting for latency at every step of the way,” Coval founder and CEO Brooke Hopkins said in a conversation with Baseten. Our boards time each model on its own, so each clock starts and stops at a clean edge. A real turn has steps before, between, and after those edges.

Deciding the caller has finished. Our speech-to-text clock starts at one shared end-of-speech point that a voice activity model marks for every provider. Your agent’s turn detection, the logic that decides the caller is done talking rather than pausing, sits outside that clock. Its defaults aren’t small: LiveKit’s turn detector, which its agents use by default, waits at least 0.3 seconds after the last speech before it ends a turn (0.5 seconds without it), and the example configuration in OpenAI’s voice activity detection guide waits for 500 ms of silence. Those waits equal a third to more than half of the 940 ms, which is why testing turn detection deserves its own pass.

Handing the reply to the voice. Our text-to-speech clock starts at the synthesis request. In a real pipeline, something decides when the LLM’s text is ready to send. Pipecat, for example, buffers it until a sentence ends by default, which its own documentation says adds about 200–300 ms per sentence. Its token mode streams text to the voice as it arrives instead, with some risk to speech quality depending on the provider.

The phone network. Our boards call each provider’s API from us-east-1, with no phone line involved. A caller on a mobile network adds delay in each direction, and the ITU’s telephone network planning standard puts the mobile leg alone at 80–110 ms one way. Our guide to measure voice AI latency covers the phone leg and how to time a full turn.

Cold connections and distance. Both speech clocks leave out connection setup (the TCP, TLS, and protocol handshakes), which Coval’s benchmark methodology puts at about 50–200 ms for a WebSocket text-to-speech connection. An agent that opens a fresh connection mid-call pays that on the turn where it happens, and one that runs far from its providers’ servers pays a longer round trip than our us-east-1 runs show.

Tool calls. A turn that looks something up makes two model requests: one that asks for the tool, and a second, after your code runs it, that writes the reply. Our LLM clock stops at the first token of one request, so the second request and the tool’s own run time come on top.

LiveKit’s documentation describes adding up per-stage metrics as a way to “approximate total latency,” and approximate is the right word. The boards tell you what each model costs. The turn your caller sits through belongs to your agent.

How the cascade compares with speech-to-speech on our boards

Our speech-to-speech board measures a different kind of agent: one where the provider takes in the caller’s audio and sends back the spoken reply, with no chain for you to wire together. It runs the same multi-turn simulated calls against every model and reads voice-to-voice latency from the call recording, from the moment the caller stops speaking to the first audible reply. That clock covers the whole turn, including deciding the caller has finished.

In its newest data, from November 3, the fastest model on that board, PolyAI’s Dialog Rsn 1, started its spoken reply 921 ms after the caller stopped talking, at p95. That’s a whole turn, measured, in about the 940 ms our fastest accurate cascade budgets for its three models alone, before turn detection or the handoff is counted.

So on speed, our boards favor speech-to-speech. The case for a cascade is control. If something has to happen between the agent deciding what to say and the caller hearing it, like checking the reply against your refund policy, a cascade hands you the text to check first. We weigh that trade in speech-to-speech vs cascaded.

Test your shortlist end to end before you commit

Component boards tell you where to start. Where you land depends on your prompt, your turn detection settings, your tools, and the phone path to your callers, and only a test of the assembled agent captures all of them. Coval’s benchmarking team put it simply in our benchmarking AMA: “Measure what reaches the user, not only what leaves the model.”

Here’s how to run that test on the stacks above in Coval:

  • Call your agent the way customers do. A simulated caller dials your agent’s phone number or SIP address, and the call routes over the public phone network. A persona sets that caller’s voice, language, audio environment, and behavior, so the test hears accents and background noise, not studio audio.
  • Run each candidate as a mutation. A mutation is a configuration variant of your agent that Coval runs alongside the base agent, with results grouped by variant. Give each candidate its own variant, the way you’d compare TTS voices or providers, and all of them run against the same test set.
  • Compare the wait, then find the stage. The Latency metric reports the average silence between the caller finishing a turn and the agent starting to respond, so you can compare stacks on what callers actually sit through. Trace metrics then split that wait by stage, with time to first byte for speech-to-text, the LLM, and text-to-speech, read from the OpenTelemetry traces your agent sends to Coval. One command, npx @coval/wizard, instruments a Pipecat, LiveKit, or Vapi agent to send them.

Start from the boards, then measure your agent

On our boards, the lowest latency voice AI stack that doesn’t trade away accuracy is Qwen3 ASR Fast, Gemini 2.5 Flash, and vui, at about 940 ms of model time at p95, and the LLM is 800 ms of it. The speech picks are the easy part of the budget. The LLM is the real decision, and with Gemini 2.5 Flash closed to new users, you’ll make it on your own numbers. Our STT, LLM, and TTS boards are where to start. Then put the assembled agent on realistic calls in Coval and measure what your callers actually wait.

Get deployment-ready.