Time to First Audio, Measured Right: How a Benchmark Change Made TTS Faster

Key Takeaways

• The Coval TTS benchmark changed how it measures latency: perceived TTFA now counts leading silence before the first audible sample, not just network arrival time.
• The change moved Gradium’s reported number from 171.9 ms to 429.6 ms with no model or infrastructure change on Gradium’s side. The gap was measurement, not regression.
• Gradium used the new signal to retrain its default model to emit audible audio in its first 10 ms frame, cutting its perceived TTFA to 214 ms.
• Rows recorded before the methodology change are not comparable with rows recorded after it. The metric kept its name, so current comparisons exclude everything recorded before the change.
• The benchmark runner, datasets, and methodology are open source (Apache 2.0) for independent verification.

On June 3, 2026, Gradium’s time to first audio on the Coval TTS leaderboard moved from 171.9 ms to 429.6 ms. No model had shipped and no serving infrastructure had changed. Gradium’s own instrumentation still reported 148 ms of network and synthesis latency, unchanged from the week before.

What changed was the measurement. The leaderboard had started counting the silence at the start of the stream.

Why time to first audio should include the whole stack, not just the model

Most latency benchmarks try to isolate the model from the servers around it. Coval’s benchmark takes the opposite position: a benchmark should reflect how the whole stack feels to a user, because that is what the customer experiences. The latency number includes the network round trip (Coval’s earlier write-up on TTS latency covers why streaming TTFA is the production metric), and the benchmark then goes one step further and processes the returned audio to find when speech actually starts.

The reason is a specific failure mode: some models send audio back quickly, but the stream opens with a second of silence. Conventional benchmarks measure the time until any response arrives, so those models score well on paper and feel slow in production.

The metric is perceived TTFA, the time an enqueue-and-play client waits before it hears sound:

TTFA = (first audio chunk arrival - synthesis start)
       + leading silence inside the stream before the first audible sample

How the first audible sample is detected

Most latency benchmarks measure the time between sending a request and starting to receive streamed audio back. The clock stops regardless of what that audio contains. Perceived TTFA also assembles the streamed audio into a single clip and processes that clip to find the moment a response actually begins.

Conventionally this is done with a VAD model, trained to identify when speech starts or ends. But many TTS models open their response with an in-breath before beginning to speak. Listeners perceive that in-breath as the start of a response; VAD models often don’t.

To match the perception to the reality, the benchmark instead uses an RMS-threshold onset detector over a sliding window. RMS essentially asks “how loud, on average, was this short window of audio?” The runner converts the returned 16-bit PCM to a fixed [-1, 1] scale, then slides a 10 ms window across the assembled audio, advancing 1 ms at a time, and computes RMS loudness for each window. Each window’s DC offset is removed first, so a constant silent signal does not read as sound. The perceived onset of speech is the first window whose RMS exceeds 0.01 on that scale; everything before it is leading silence. This leading-silence term is then added to the conventional arrival-time measurement to give perceived TTFA. The offset is best-effort: if it cannot be computed for a response, that row’s TTFA falls back to arrival time alone. The full definition is published in Coval’s benchmark methodology.

The change shipped with an explicit warning about comparability. Adding the leading-silence term shifts every provider’s reported TTFA upward by its own offset: near zero for models that emit audible audio immediately, several hundred milliseconds for models that front-load silence. Rows recorded before the change are not comparable with rows recorded after it. The metric kept its name and schema, and older rows carry no methodology marker, so current comparisons simply exclude everything recorded before the change.

What the new TTFA metric exposed in Gradium’s model

Gradium’s arrival time was competitive; its perceived latency wasn’t. The model was emitting a median of 225 ms of inaudible samples before the first phoneme, and as much as 655 ms in the tail. Once the methodology changed, Gradium prioritized closing that gap and trained a new model to produce audible audio from the first sample.

The model that came out of that work is now Gradium’s default. Its leading silence measures 0 ms: the first 10 ms frame it returns is already above the audibility threshold, so nothing is added to its arrival time. On the board, its median TTFA is now 214 ms, more than 200 ms faster than the 429.6 ms the model it replaced posted when the methodology changed.

Here is a selection of rows from the board, median values in milliseconds, taken from the benchmark’s run data in early September 2026. The public leaderboard lists more models than are shown here and updates continuously, so check benchmarks.coval.ai/tts for current numbers. Model names follow the public leaderboard, and each one links to that model’s page with its current numbers:

Model	Provider	TTFA	Round trip	Leading silence	Silence share of TTFA
TTS Flash 2	Inworld AI	72.2	72.2	0	0%
TTS 2	Inworld AI	175.8	120.8	55	31%
Flash v2.5	ElevenLabs	182.4	141.9	40.5	22%
Default	Gradium	213.9	213.9	0	0%
Sonic 3.5	Cartesia	270.3	137.3	133	49%
S2.1 Pro	Fish Audio	286.1	279.1	7	2%
S1	Fish Audio	353.1	241.1	112	32%
Eleven v3 Conversational	ElevenLabs	354.2	313.2	41	12%
Sonic 3.6	Cartesia	437.2	296.2	141	32%
Chirp 3 HD	Google	438.0	152.0	342	78%
S2.1 Pro Free	Fish Audio	920.5	912.5	8	1%

Of the eleven models shown, only two, Inworld TTS Flash 2 and Gradium’s new default, show 0% in that last column: audible audio in their first frame. Everyone else pays a silence tax on top of their network time, and the last column is where that tax shows up. Chirp 3 HD’s wait is 78% silence, and Cartesia Sonic 3.5, despite having the third-fastest round trip in this set at 137 ms, still lands fifth on TTFA because silence is nearly half its total wait.

How evaluation design changes what labs optimize

Labs optimize what is scored. When the metric is time to first byte, the cheapest way to move it is to send something early, and the thing sent does not have to be audible. The metric improves and the product does not.

This is the same trap that makes vendor-published TTS benchmarks unreliable on their own: a benchmark someone can game by construction stops measuring the thing you actually care about.

A benchmark that isolates the model from the stack tells you about the model. A benchmark that measures the stack the way a user hears it tells you about the product. The second is harder to build, because it requires processing the output rather than timing the response, and it returns worse numbers for everyone on the board. The change cost the leaderboard comparability with a year of historical rows. It is also the change that produced a faster model.

Reproducing the TTFA benchmark

The benchmark is open source under the Apache 2.0 license. The runner, the pinned datasets, the provider matrix, and the full methodology contract are in the repository, so anyone can run the same measurement against their own provider API keys. See Coval’s TTS and STT benchmark results via API for the programmatic path.

A single local run reproduces a comparable number, not an exact leaderboard cell. Published leaderboard values are aggregated across multiple scheduled runs over a time window, and each run draws an independent random sample from the dataset manifest, so it will not reconstruct one specific historical cell. Match the runner version, dataset manifest, and provider model identifier documented in the methodology, and the result will track the published trend even if the exact millisecond differs.

• Code: github.com/coval-ai/benchmarks
• Methodology: github.com/coval-ai/benchmarks/blob/main/docs/methodology.md
• Live results: benchmarks.coval.ai/tts (see also Coval’s benchmark roundup covering Gradium, ElevenLabs, Cartesia, Rime, and OpenAI)

Gradium’s current default model is available through its API and in Gradium Studio, with Pipecat and LiveKit integrations supported directly. Gradium is collecting hard latency and pronunciation cases to improve its next models; send them through Gradium’s Discord. If you’re evaluating TTS providers for a production voice agent, Coval runs this same perceived-latency methodology against your own scenarios. Go to the pricing page to get started.

FAQ