Whisper Large v3 Turbo Streaming Providers in 2026: Who Streams It and Why the Host Matters

Which hosts stream Whisper Large v3 Turbo, which serve it for files, and what happened when we ran the same Whisper model on a dedicated host and a shared one.

Part of the appeal of an open-weight model like Whisper, one whose weights anyone can download and run, is that you get to choose where it runs. Several inference hosts serve Whisper Large v3 Turbo, OpenAI’s faster Whisper variant, but the aggregator pages that list them show prices, not latency, and rarely say whether you can stream. So which providers actually stream Turbo to a live voice agent, and does the host you pick change what you get?

Quick answer: Whisper doesn’t stream on its own, so any streaming Whisper is a wrapper someone built around the model. Baseten documents a streaming Whisper Large v3 Turbo deployment on dedicated GPUs, Together AI streams Whisper Large v3 (not Turbo) on its shared API, and Groq, Cloudflare Workers AI, and DeepInfra serve Turbo for file transcription. The host you pick changes the results. Our benchmark runs Whisper Large v3 on both Baseten and Together, and over the 30 days to November 4, 2026, Baseten’s endpoint got 4.5% of words wrong to Together’s 7.2%. It also returned 95% of its final transcripts within 248 ms of the caller finishing, against 494 ms on Together.

Same models can give different results, because the host is part of the model you’re choosing. Below, we cover who serves Turbo and how, why the host matters so much for Whisper, our two-host numbers, when dedicated hosting beats shared and when it doesn’t, the open models built to stream, and how to test the exact endpoint your agent will call.

Whisper Large v3 Turbo streaming providers, and which ones actually stream

Here’s what each host’s own pages say it serves. Streaming means the host takes audio over a WebSocket while the caller is still talking and sends partial transcripts back as it goes; the others take a finished recording. A shared endpoint serves many customers on the same infrastructure, and a dedicated one runs on hardware reserved for a single customer.

HostWhisper it servesStreamingHow it’s hosted
BasetenLarge v3 TurboYes: WebSocket, partial transcriptsDedicated GPUs you deploy
Together AILarge v3 (no Turbo listed)Yes: WebSocket, partial transcriptsShared API, or a dedicated endpoint
GroqLarge v3 Turbo and Large v3File upload or URLAPI billed by audio length
Cloudflare Workers AILarge v3 TurboBatchAPI billed by audio length
DeepInfraLarge v3 TurboFile uploadAPI billed by audio length

For a live voice agent, our pick is a dedicated streaming deployment like Baseten’s. You get Turbo with real streaming, you control the hardware and how many calls share it, and on Whisper Large v3 our dedicated Baseten endpoint beat Together’s shared API on accuracy and on the slowest turns. If you’d rather not manage GPUs, Together’s streaming Large v3 is a shared API you can call without deploying anything, and it matched Baseten on a typical turn. Groq, Cloudflare, and DeepInfra suit recordings, not live calls.

Turbo is a faster Large v3, and the host supplies the streaming

Turbo is Large v3 with most of its decoder taken out. OpenAI cut the decoder, the part of the model that writes out the text, from 32 layers to 4 and fine-tuned the result. That shrinks the model from 1,550 million parameters to 809 million and, by OpenAI’s count, makes it about 8 times faster. The cost, in OpenAI’s words, is “a minor quality degradation”: Whisper’s maintainer at OpenAI puts Turbo at about the level of the older Large v2, with larger drops in languages such as Thai and Cantonese, and Turbo isn’t trained for translation. Groq’s model table shows the trade on its own platform: a 12% word error rate for Turbo and 10.3% for Large v3.

Neither version streams. OpenAI’s model card says Whisper models “cannot be used for real-time transcription out of the box,” and the model works on 30-second windows of audio. A host that streams Whisper builds the rest: voice activity detection to tell when the caller has stopped talking, chunking to feed audio in as it arrives, and the partial and final transcripts. Baseten’s streaming Turbo example ships its own detection settings, such as a 300 ms minimum silence before it ends a segment, and Together’s server runs its own voice activity detection. Choices like these decide how fast the final transcript lands, which is why one model can behave differently from host to host.

The numbers that follow are for Whisper Large v3, the version our benchmark runs on both Baseten and Together. They answer the question behind “which host is fastest”: how much does the host change what one model delivers?

On our benchmark, the host changed Whisper’s accuracy and its slowest turns

On a typical turn, the host made no difference. On accuracy and on the slow turns, it made a big one.

Our Whisper Large v3 page tracks the model on Baseten’s dedicated streaming endpoint and Together AI’s shared API, more than 13,000 samples each over the 30 days to November 4, 2026, on fixed public audio that ranges from clean speech to accented, noisy, far-field, and phone-codec recordings. Latency is time to final segment (TTFS): how long after the caller stops talking the final transcript arrives, with every host’s timer started at the same end-of-speech point, marked by one shared voice activity model. Word error rate (WER) is the share of words transcribed wrong, so lower is better.

Whisper Large v3Baseten (dedicated)Together AI (shared)
Average TTFS126 ms173 ms
p50 TTFS111 ms111 ms
p95 TTFS248 ms494 ms
Average WER4.5%7.2%

Half of all turns came back within 111 ms on both hosts, but the slowest ones split: Together’s p95 was twice Baseten’s. Accented speech showed the widest gap, averaging 87 ms on Baseten and 180 ms on Together. Together’s endpoint also made about 60% more word errors with the same model.

Bar chart of Whisper Large v3 time to final transcript in milliseconds after the caller stops talking. At p50, Baseten dedicated and Together AI shared are both 111 ms. At p95, Baseten dedicated is 248 ms and Together AI shared is 494 ms. Source: Coval benchmark, 30 days to November 4, 2026.

Same model, two hosts: identical p50, twice the p95. Whisper Large v3 time to final transcript on Coval’s benchmark, 30 days to November 4, 2026.

Why would one model behave so differently? Our model page names the levers: “Quantization, hardware, batching and decoding parameters can change speed and accuracy for an open-weight model.” Those settings belong to the endpoint, not the model, and they sit on top of the streaming wrapper. A shared API also carries other customers’ traffic, and Together’s docs say its shared response times “can vary” with the model and current demand. That’s why we measure each hosted endpoint separately.

Dedicated vs shared inference: neither wins by default

Whisper’s result makes dedicated hosting look like the safe default. Our Qwen3 numbers say it isn’t automatic. We measure two models from Alibaba’s Qwen team, one for speech-to-text and one for text-to-speech, on our own Baseten dedicated deployment and on Nari Labs’ shared API, which serves versions it calls Qwen3 ASR Fast and Qwen3 TTS Fast. That gives us the same model family on two kinds of host.

For speech synthesis, the shared host was faster. Qwen3 TTS Fast on Nari Labs averaged 69 ms time to first audio, the wait from sending text until the first audible sound comes back, against 104 ms for Qwen3 TTS 1.7B on Baseten. For transcription, each host won a different measure: Qwen3 ASR 1.7B on Baseten averaged 35 ms TTFS but reached 110 ms at p95, while Qwen3 ASR Fast on Nari Labs averaged 51 ms with a p95 of 71 ms.

So dedicated hosting doesn’t buy speed by default. It buys control, and control has running costs.

Idle GPUs cost money, so Baseten scales deployments to zero by default. A request that arrives with no replica running waits while one boots, and Baseten bills the startup minutes. In our runs, a Baseten replica took about four minutes to boot from zero, so we warm dedicated endpoints before timing starts. A phone line needs the same: set Baseten’s min_replica to 1 or more so no caller waits on a boot.

Billing differs too. Together bills shared speech models per second of audio. Baseten bills dedicated GPUs per minute while a replica runs, calls or no calls, and nothing at zero replicas. Our pricing directory lists the published rates by endpoint.

On dedicated hardware you also set how many streams share a GPU, and Baseten is plain about the trade: “Higher concurrency can significantly reduce cost per audio hour, with a corresponding tradeoff in latency.” Region is the other lever. Together’s shared tier can keep processing in the United States for select models, and for any other region its docs point you to a dedicated endpoint pinned there.

Shared hosting spares you servers to run and bills by the second of audio. Dedicated buys you the hardware, the concurrency, the region, and warm capacity. Our conversation with Baseten covers when dedicated inference pays off.

Open-weight speech models built to stream, and where they run

Whisper needs a host’s wrapper to stream, but some newer open-weight models are built to transcribe audio as it arrives. Here are two of them, plus NVIDIA’s Parakeet for comparison, with where each runs and, for the two built to stream, our word error rates from the same 30 days.

ModelMaker and licenseHow it streamsWhere it runs (our WER)
Qwen3 ASR 1.7BQwen (Alibaba), Apache 2.0One model handles streaming and offline; streaming runs on the vLLM engineBaseten dedicated (2.9%); Nari Labs shared, as Qwen3 ASR Fast (2.6%)
Nemotron 3.5 ASR Streaming 0.6BNVIDIA, OpenMDW-1.1Built to stream; a chunk size from 80 ms to 1.12 s trades latency for accuracyTogether AI shared (13.0%)
Parakeet TDT 0.6B v3NVIDIA, CC-BY-4.0Through NVIDIA’s chunked-inference script, a wrapper like Whisper’sTogether AI, shared or dedicated

Built to stream isn’t the same as accurate: Nemotron’s 13.0% WER is more than four times Qwen3 ASR’s on either host.

If you’re choosing between Turbo and Parakeet, Parakeet is the smaller model (600 million parameters to Turbo’s 809 million) and covers 25 European languages, and like Turbo it needs a chunking wrapper to stream. When streaming is the point, Qwen3 ASR is built for it, and it was more accurate in our runs too: under 3% WER on both of its hosts, against Whisper Large v3’s 4.5% and 7.2%. For managed and open options side by side, see our guide to the best speech-to-text providers.

Test the exact endpoint your agent will call

Our numbers give you a shortlist; your agent decides. It brings its own callers, audio path, traffic, and region, and the endpoint you pick has to hold up under all of them. That’s what Coval tests.

Start with simulated callers who sound like your real ones. A persona sets the simulated caller’s voice, language, accent, and background noise, so you can test across accents and noisy rooms before a customer does. Then run the same test set (the scenarios your simulated callers work through) against two deployments of your agent, one wired to each host. In Coval, one variant of the run can swap in the second deployment’s WebSocket address, and both sets of results come back in the same run, side by side.

Score the speech-to-text step on its own, too. If your agent sends OpenTelemetry traces to Coval, the STT Word Error Rate metric compares the transcript recorded in your agent’s speech-to-text spans with Coval’s transcript of what the simulated caller said, and STT Time to First Byte reads how long your speech-to-text service took to return its first result. Both come from your agent’s own traces, so they measure the endpoint you actually call, under your load.

Then keep scoring after launch, because a host can change hardware, batching, or capacity behind the same model name. With production monitoring, every production call you send to Coval is scored against your metrics, and Coval learns your baselines and flags drift the moment it starts, so a slower or less accurate endpoint shows up as an alert instead of a customer complaint.

Measure the endpoint, not the model name

Picking Whisper Large v3 Turbo is half the decision. The host is the other half, and as our Large v3 numbers show, it can move accuracy and the slowest turns while the model stays the same. Shortlist hosts on measured endpoint numbers like ours, then put your own callers through the endpoint you plan to ship.

Get deployment-ready.