When Self-Hosted Voice AI Models Make Sense: Baseten x Coval

Welcome everyone to this episode of The Voice of Voice AI. I'm super excited because today I'm here with Tianshu Cheng, who is the lead of voice AI at Baseten. I've been super impressed with the work of Baseten and all they're doing in voice AI. I think Tianshu is operating at the bleeding edge of voice AI because the most advanced teams are deploying self-hosted models or open-source models on neoclouds. You get to see very advanced voice systems, and you're the one advising them. I'm excited to hear everything you have learned in the field. We're also announcing our benchmarks for open-source models hosted on different neoclouds, and how different configurations and deployments can impact your performance, latency, and all of these metrics that you care about most as voice AI teams. Thanks so much for joining.

Thank you, Brooke, for the introduction and for having me here. I'm as excited as you to partner on this launch together. Hi everyone, I'm Tianshu from Baseten. I'm leading the voice AI product at Baseten. We work with a lot of customers building voice agent applications and applications across transcription, text-to-speech, and all types of voice AI. We're here to help them figure out how to deploy open-source models, including some closed models, on Baseten with the best performance and latency, and cut costs for them. I'm glad to be here to compare notes and share what we saw from the field and what we think about the technology in the next stage.

Amazing. Today we wanted to dive deep on when voice teams should be considering open-source models. When should they start thinking about using models self-hosted on neoclouds to improve performance and reduce costs? What are some leading indicators, and how should they be planning their voice AI developments and roadmaps? And then, how to make that transition and those engineering decisions. We're going to be in the weeds and hopefully get to the meat of how you should be making these decisions as an engineering leader.

On my side, I can share a lot about the thinking framework behind making the right inference provider decision: why you should consider different inference providers to give the best performance and user experience for your voice agent application, when that should happen, and how to make that decision for your business needs.

Totally. Starting at the top, what are some of the biggest reasons that companies and teams come to Baseten for voice models? How would you recommend thinking about this decision of using a model hosted on Baseten versus using a model through a managed API or some other off-the-shelf service?

I think a lot of customers come to Baseten because they see some critical business requirements that cannot be satisfied by the managed API. For example, the cost is too high. As you scale up your WebSocket connections from 10 to tens of thousands, you will be surprised by the expensive bill from serverless. That's when they start to look at a self-hosted solution to make it more cost-efficient as they scale up to the next level.

We also see a lot of customers trying to use a dedicated inference provider to serve their fine-tuned model, the model they own themselves. It could be a model they train from scratch or a model they fine-tune with their own data. For some modalities, like TTS today, we still see a gap between closed and open-source models. One way to bridge that gap is to fine-tune a TTS model with your own data. You might hire a voice actor and use their data to clone a special voice, which can sound more unique and natural compared to the stock options from open-source TTS. When you have any model customization needs, you will need a dedicated inference partner to host your own model.

A lot of times, it's just a higher standard for latency. For voice agents, you should not just look at the P50 latency, which is probably pretty good everywhere. When you look at the long-tail latency, like P90 and P99, it's actually a game changer. You will see variance across different inference providers. When that becomes a big factor for your business, you will start looking at dedicated providers.

That's something I'm consistently surprised by: people often focus on the median versus looking at this long-tail performance. But if you have 10 to 20 turns in the conversation, it's really important that the long-tail performance is high because of the compounding effects.

Exactly. This is also an interesting insight recently shared by one of our FDEs. When he's having this conversation with a customer, he will correct their understanding about latency by saying: when you look at P90 long-tail latency, it doesn't mean one out of 10 users will be hit by that long silence in a conversation. It means one out of 10 conversation turns will be slowed down by the model latency issue. That's going to hurt every customer. P90 and P99 latency are way more important for voice agent applications. That's also why you should think about the impact of the infrastructure provider on P95 and P99 latency, which is core to your user experience.

One hundred percent. As engineers, you're so trained to look at P75 and P90 as covering the majority of users. That number can be really deceiving.

I think that's also how I was impressed by your product, because on Coval you have that candlestick graph to show latency at different percentiles, which is pretty important. I never saw that before from other providers. It's differentiated for you guys, and it tells me how you're on the same page with customers when they're thinking about the production readiness of the product.

Totally. That's exactly what we want the benchmarks to be. We built them for our customers because so many were asking, “What model should I be using? How should I be thinking about making these trade-offs?” Something special about our benchmarks is that we don't have any financial incentive in any way for them. It's open source and maintained in service of our customers. The reason that's so important is that we're not incentivized in any way on the benchmarks. We're aiming to make them representative of the questions our customers are asking us, and not necessarily to make any one model look good.

Our goal is to capture what we're actually seeing with customers: multiple turns or instruction following for real use cases. Whether you can follow a procedure for a coding assignment doesn't matter for a voice agent. There are domain-specific use cases, medical use cases, hospitality use cases—there's such a range. We're trying to embed a lot of these real-world use cases into the benchmarks versus making them overly academic, so that they're representative of what people are actually trying to do. You touched a bit on hosted models versus hosting on Baseten with more of your self-hosted inference. How do you compare to self-hosting inference on your own infrastructure, say through AWS or some of these hyperscaling clouds? Where do you see the trade-offs for people there?

To echo what I mentioned earlier, I feel you should start thinking about all the different inference options from day one when you're building out your voice agent application. Latency and performance are such a critical core of the user experience from day one, when you start shipping this out to your first user. You should try as many options as you have out there as an inference provider, whether it's your own VPC, a cloud provider, or serverless. In terms of the difference between self-hosting your model on your own VPC and on a cloud, as a cloud we consolidate all the requests and usage from different users. That gives us leverage to pool a lot of GPU resources together in a more efficient and available way.

Today we're still, unfortunately, in this worldwide GPU crunch. For anyone who's buying GPUs to build up their own VPC, you probably already feel that it's really hard to get actual GPU capacity from cloud providers like AWS or GCP, not even thinking about getting the latest, most advanced GPUs from them. With Baseten as a cloud provider, we can give you that guarantee: for whatever new GPU is going on the market, we can give you access on day one. With infrastructure-level features like autoscaling and scaling across cloud providers, we can give you a guarantee to always have GPUs available 99.99% of the time. That high level of uptime and GPU availability guarantee is really hard to build yourself. I'm not saying you should not try, but once you try, you will know how hard it is.

Totally. You come from Twitter and big tech, where it's interesting to see the industry shift. Previously, the idea of not having compute was almost unfathomable, and now the idea of not having enough compute is so top of mind for companies. As you see customers starting to think about hosting their own inference, what are some of the biggest challenges in deploying that? How can customers start to prepare or make engineering decisions to set themselves up for success? Where do people often get hung up, or what is harder than they expect?

The challenge is definitely split into two major parts. One is model selection. After you decide to use a model, the other is how to meet your SLA requirements on latency, performance, and cost. For the first part, model selection, I feel this is something we are still learning how to guide our customers through. Even today, I'm still surprised by how lacking the eval ecosystem is, whether it's internal or external evals, especially for voice agent applications. Part of the reason is that, by nature, this is a very challenging problem. For voice agents, there are so many dimensions you want to evaluate. On one side, you want to make sure the agent can get the task done. On the other, you want the agent to sound human. The voice quality matters; the tone matters.

That's why, whenever we're in the first or second call with a customer, we always feel there's a big missing part on the evals. In this case, we either refer to a public dashboard or model leaderboard, which might not be representative all the time, or we try to collect as many sample tests as possible from the customer. We can use those as an indicator or approximation of overall model quality. On that part, I actually have a lot of question marks. I want to learn more from you about how to give our customers a better, more reliable tool to help them select which model to use.

Totally. This is one of the biggest mistakes we see teams making: they don't implement evals from the beginning. It's much easier to add incremental tests than it is to do one large upheaval of a system. I think every engineer knows this, but setting up mocks or setting up your system so that it is testable—even having a user that you can create or test databases—is traditional software engineering. But then also incrementally adding test suites. At one point, there's one engineer working on your voice system. Most teams start with a couple of engineers working on a POC, and then it expands and grows. Having one person who knows everything the voice system does makes it a lot easier to set up evals than when there are so many different capabilities and no one person knows all of them anymore.

Now it becomes really hard to change core parts of the system, like the model or the voice, because you don't have a baseline. It's something that feels like it takes a lot of effort to do, but even having any sort of system in place that you can incrementally build off over time is a huge asset. Then, when you get slapped with a huge bill and you're overnight trying to figure out how to switch, the lift to change models or pieces of your infrastructure isn't quite so overwhelming.

I think about this in three ways when people are choosing models. First, the coarsest-grain knob is: what model family should I even be looking at? That's where public benchmarks are really helpful. That's why we created benchmarks.coval.ai. There's Artificial Analysis. There are lots of benchmarks out there, although not as many as I would like. I think there should be a lot more voice benchmarks. They let you understand broadly what models are out there and which might be a good fit for your use case. Again, that's why we're focused on providing real use cases for voice AI that are based on true customer conversations we've been having, rather than academic benchmarks.

The next step, once you've selected three to five models, is to start experimenting with those on your own local benchmarks. Local benchmarks are underrated. Most people don't do these, but it's running those different models with your own datasets. We have this capability in our platform. Say you're trying out self-hosted inference with Baseten, OpenAI, Gemini—whatever models you're playing around with. You can run those on your datasets to evaluate latency, word error rates, instruction following, and so on.

The third step is plugging it into your system, which is obviously the biggest lift because it requires some engineering integration. You have to deploy it. Sometimes that's easier than others, depending on how arduous your deployment system is, and then actually run simulations of those conversations. That's where you're going to see the true effectiveness. But each step along the way, one is cheaper than the other. You're incrementally narrowing down which models to look at. This is also why the Pareto frontier is so important: balancing cost, latency, and all these different aspects of the model. I'm excited about what we're launching today because Baseten is really leading the Pareto frontier. It's been exciting to see your success across lots of different dimensions.

I'd love to hear how you've seen people deploy. Does this three-step process—first looking at public benchmarks, then running your own local benchmarks, and then doing these end-to-end evals—resonate with what you've seen your customers doing?

I think you just painted the dream process of model selection and production rollout. Most of the time, we get stuck at step two because it takes effort for customers to collect enough data to work as a golden test dataset. They prefer not to do that until later, when there's a strong need to form that data collection and eval cycle. To echo your earlier point, it's really important to start building that data flywheel as soon as possible, because you will only start getting more insights into your user experience and failure modes when you have a baseline out there. Even if it's just 10 audio samples or 10 conversation runs, from there you will identify more and more failure modes that aren't caught by the existing baseline. You will add more data into that, gradually form your own data flywheel, and use that as your own secret sauce to improve your product much faster.

Definitely. This is what we talk about as really core to our product: creating this flywheel so that your agents are constantly improving. It's not about deploying an agent one time. Today, a lot of people think about it that way because it is such a monumental effort to switch from either human conversation or an IVR system to a voice agent. But that's just step one. Creating a flywheel means you're constantly improving and taking learnings from the field. If a ton of people are calling over Christmas asking about refunds, and that is a significant model drift from everything you've tested before, you need this lens into observability of all these conversations coming in. Then you take learnings from that and feed them back into your simulated tests so that you can constantly improve your agents.

That will also make life much easier for our FDEs when we work with you on what should be the best model to optimize for. With that visibility, you can easily tell which model fits your business needs the most. Of course, you can look at the frontier model from the public leaderboard. But a lot of lessons have told us that the public leaderboard only speaks to some degree; it cannot really represent your specific workload.

What are some of the different measures? There's obviously cost, latency, and accuracy. These are the holy triad of things people are constantly trying to balance. But in terms of how that translates into metrics, what are some things people should be looking at? How should they be thinking about what to evaluate in these models?

Are you talking more about the performance-side metrics?

Performance. I think this is a great differentiation: you have your performance-side metrics and then more qualitative metrics. I'd love to hear a little on both. What are people looking at from the performance side, and then the quality and semantic side?

On the performance side, it depends on whether you have a process to evaluate the voice agent end to end, or want to do more granular evals on each component. Today, a common pattern for voice agents is still a cascaded system that pipelines STT, LLM, and TTS together. From our perspective, when we're working with a customer on a performance target, we start by targeting the requirements for each module. Once we get really good performance out of each STT, TTS, and LLM model, we work with them on integration tests to see how the end-to-end voice agent performs in real-world use cases.

These two processes are interlinked, because the ultimate goal is to achieve really good end-to-end latency. Sometimes you can only optimize an LLM's latency to a certain number, and you cannot go further because there's a theoretical limit you can hit. At this point, we'll zoom out and look at the big picture to see what other latency overhead we can shave off the end-to-end path. Maybe we can achieve better latency by collocating all three models together. You can utilize the local network on the same GPU cluster for the whole system. That's another lever outside of the model to optimize overall latency. There are also techniques like fine-tuning the LLM to a smaller size, just for this voice agent use case, which can also become a big unlock for end-to-end latency.

For those who maybe didn't catch that, what's so cool about collocating these models is that, in a cascaded system, you have your TTS, STT, and LLM. The transcription model, or STT, takes what you're saying and transcribes it for the LLM to reason about, and then for the voice model to say out loud. What Tianshu is saying is that you're able to collocate these three models together. You can reduce that network latency between each of these models.

With voice AI, you're fighting for latency at every step of the way. The latency requirement on the user side is so high because the moment something doesn't respond for five seconds, you're going to assume all hope is lost and leave the conversation. But there are also so many different models involved in this process and opportunities to introduce latency. Telephony introduces latency. Network time introduces latency. Tool calls introduce latency. Every model you introduce—STT, TTS, LLM, but also VAD or any background agents or guardrails you're adding in—introduces latency. The more complicated your voice agent gets, the more you're fighting up against the limits of your latency budget, or how much latency you want to allocate to each turn. That's exciting: the idea of not just pushing models to be faster, but also the infrastructure around them.

That's also why I got fascinated by this paradigm innovation from Thinking Machines Lab. When they released this new interaction model several months ago, I was surprised. It opened the public's eyes to a new paradigm of orchestrating a voice agent system. Instead of using one single LLM in charge of interaction and intelligence at the same time, you decouple them into two different models. You use a smaller model to be more responsive and do the frontline interaction, while dedicating the complex tasks to an intelligent model working in the background. A lot more paradigms will grow out of this space as people think about how to solve the end-to-end latency problem in a more innovative way.

Definitely. It's not just latency; it's also accuracy. In voice agents, each turn is usually not very complicated. Turns over time are where the complexity comes from. This part is interesting: with what Thinking Machines did, each turn is not very difficult to respond to. You can have a smaller model deciding what you need for this turn versus overloading all the complexities of what could be in that conversation. The difficult part of conversation is not saying, “What should I say back to ‘hi’?” It's saying, “They could ask me anything, and I need to be ready to answer all of those questions.” We're so clearly at the beginning of the architecture innovation of voice models.

I see a lot of parallels with self-driving. In self-driving, you have perception, planning, and controls: what is happening in the world around me, what should I do next, and how do I actually take that action? I see parallels with voice, where you have transcription—what is happening in the world around me—LLMs that reason about what to do next, and then voice, or tools, which are how to take that action in the world. If we look at the history of self-driving, a lot of these models are evolving in the same way. Originally, you had discrete models for each step of the pipeline that would pass certain amounts of information to the next. You would have a behavior prediction model that was isolated, then passed information to the planner, which would pass it to controls.

Over time, throughout self-driving, there's been this consolidation and expansion of models. You have Tesla famously saying they're only going to have this one model to rule them all. In more micro ways, there's been consolidation and expansion of different models throughout. I think we're going to see the same thing happen with voice, where people combine transcription and LLMs, or combine the entire voice-to-voice model but then have background models. What are some other architectures you've been seeing that are particularly interesting?

Another variant is to pair an audio reasoning model with a TTS model, which is something in the middle. This is also the model released by Thinking Machines, the Inkling model that can take audio in and text out, but with a high level of intelligence to do complex reasoning and tool calling. Then you connect this model to another TTS. Of course, speech-to-speech models are always an exciting area that people are paying a lot of attention to. We always get this question from customers: when will there be a great open-source speech model so we can host it on Baseten? I would say that future is pretty close. We're working closely with some open-source model providers who are going to publish a great speech model that's industry-quality ready. Stay tuned; we'll give day-one support for those models on Baseten.

That's super exciting. The beauty of open source, with speech-to-speech models or all of these self-hosted models, is that you can start to tailor them to your use case. The beauty of voice is that most use cases are incredibly specific, versus a model that needs to handle all coding tasks across every industry. With voice, you're usually solving for banking customer support or healthcare requests. You're doing this at massive scale: tens of millions of conversations. How can I make sure that, in tens of millions of conversations, I'm booking the appointment correctly and providing accurate balance information? It's still incredibly complex, but there's a much narrower distribution of tasks that you have to be really good at, with much higher accuracy. With coding, you have a wider distribution, but the accuracy can be a bit lower because you have a human in the loop or self-correction.

That also aligns with another trend I've started to see more this year: customer interest in customizing models for their voice agent pipelines. Either fine-tune a Whisper model to better identify domain-specific terminology for medical or financial use cases, fine-tune a TTS model to clone a voice, or, as I mentioned before, fine-tune an LLM to a smaller size only for voice agent use cases. You don't need the largest brain in the world to do the voice agent task. You need an appropriately sized brain for it. That's why I recommend voice agent builders factor that into the decision process when you pick a model and an inference provider. Does this model have enough flexibility for you to add more knowledge and domain-specific information, with a good fine-tuning or post-training ecosystem around it?

I think that's part of the reason Whisper is still a popular model for ASR tasks, even though it's a two-year-old model: the ecosystem around Whisper is so mature. You can easily fine-tune Whisper to whatever domain-specific task, and you can serve those models easily. At Baseten, we keep that in mind. We want to give people good building blocks to customize models and make it a seamless process to transition a model from the training stage to the inference stage. Ideally, with one button click, you can deploy any training checkpoint to the Baseten platform.

One hundred percent. I think people often get the evaluation of these models wrong in benchmarks as well, especially for LLMs. The majority of LLM intelligence and reasoning benchmarks are based on all sorts of things, from LegalBench to medical use cases to coding and the reasoning capabilities there. But voice is a very specific case that is often outside the distribution, both in the labs and in these benchmarks. When you're looking at which model to use for voice AI, it's important to remember what it's being optimized for. Just because 5.1 is a later model, it doesn't necessarily mean it's going to be significantly better than earlier models. That's something we've seen, and one of the reasons so many people in voice AI are still on older models such as 4.1 mini or 5.1 mini versus the latest model. It's the latency, cost, and reasoning capabilities needed, versus that trade-off between cost and speed.

Exactly. That adds importance to the local benchmark. Whenever you have a domain-specific model, you probably also want to pair this model development process with a domain-specific dataset. I'm curious: how do you think evals should look different for different domains? Or can you use the same methodology for any specific task, whether it's legal, financial, or medical, specifically in voice?

The holy grail of running your own benchmarks is using your own data. Going back to the question of what to do if you don't have this dataset: can you take conversations you've seen in production? This is obviously difficult if you don't have many cases in production. But by the time you're thinking about self-hosted inference, you usually have these conversations, and that's a great way to subsample. Human conversations are also great. At Waymo, or in self-driving, you have constant comparisons to manual driving: are you faster than manual or slower than manual? This is a great way to do the same thing in voice: are you better or worse than manual conversations?

Creating this dataset also means setting up a system where you're refreshing datasets, versus having a static dataset. Sampling all the failures from the last seven days, or 1% of conversations from the last 30 days, can be a great way to constantly evaluate different models. You can make it very low effort to say, “A new open-source model came out. Now I want to deploy that on Baseten and swap out my inference.” Now I can see it's doing well in simulation. Once it does well in simulation, I can graduate it to production and see, over a slow rollout, with A/B testing between the two models.

I see. For app builders out there who, after listening to this podcast, start thinking about building their first dataset, what are some tips? What could be the biggest challenge in building it? One thing on top of my mind is compliance or data privacy issues. Are there any workarounds you can recommend?

To answer your first question, sampling conversations is the easiest way to do this, and then thinking about the basic principles of how to create unbiased datasets. From there, you can start to think about capturing more failures than you might otherwise see, because otherwise they're so low in the distribution—hopefully. You might want to over-index on those so you see more opportunities for weird background noises or different accents, to have a higher signal. Thinking about the distribution of your dataset is important.

If you do have privacy concerns, I think this is probably the bigger obstacle for a lot of people. One trick is that, as the conversation is happening, you can create conversations that mimic it and store those. You never actually see the customer-facing data, but you can create a redacted, similar conversation. It's not actually customer data, but it's inspired by that same conversation. That's one way of getting around it: I can see what the topic was or what the errors were live as it's coming off the call, but not actually store the call. Or create a very similar conversation to have a representative dataset.

Those are pretty helpful tips.

I would love to hear where you see voice teams going next. What are the biggest things on engineering leaders' roadmaps for voice AI going forward, given that you work with so many of these teams?

I can be transparent about our Q3 roadmap here. What we want to double down on is model customization and multi-model solutions. Model customization is important because, to further close the gap between open-source and closed models, being able to post-train your model is such a strong weapon, especially when you have a lot of internal datasets. That's why we want to provide as seamless an experience as possible for customers who have the intention and ability to fine-tune their models and serve them on Baseten. We've already published a blog post about what you can do by fine-tuning a TTS model like Qwen3-TTS to achieve much better voice quality than the stock options.

Another part is the multi-model solution. As an inference provider, we always try to think ahead of customers about how we can improve overall system performance using all the infrastructure-level levers and capabilities we have, even beyond their thinking. For example, collocating the three modules together. A lot of customers, when they hear about this idea for the first time, are impressed. They never thought this could be an option available to them because they started with serverless endpoints. You never hear a serverless endpoint provider tell you, “You can deploy your model in this specific region so you can collocate all your models together.” But that's what we can provide as a dedicated inference provider. We're going to double down in that direction to make multi-model systems more performant, not only from model runtime optimization, but from a more systematic, big-picture methodology to optimize performance end to end.

Nice. That's super exciting. I'm personally excited about that roadmap because this year a lot of people have been focusing on scaling capabilities: adding more complex functionality, complex instruction following, and expanding from the original intents they were focused on with a smaller scope. But starting to do cost optimization, model customizability, and performance are definitely more and more top of mind as people harden these production systems.

Especially if you want to test out different paradigms of orchestrating voice agents: not just the three-stage cascaded system, but also two-stage cascaded systems or this interaction and intelligence model pair. All this requires very tight network latency between the components. This could unlock a lot of possibilities for how to orchestrate the best voice agent for your needs.

That's super exciting. I'm excited for the benchmarks we're launching today because this is the first step in a lot of teams being able to explore the real trade-offs and benefits they can get from self-hosted inference, start experimenting, and seriously consider how they might deploy it themselves. To close things out, I want to do a rapid-fire round, a lightning round. Answer all these questions in one word or a few words. No hedging. Are you ready?

Mm-hmm.

Voice AI in 2026: model race or infrastructure race?

Model race.

Really?

Because I think we are so lucky to be in this temporary period of the model blowup this year. I'm confident we can solve the infrastructure problems, but I feel uncertain about what could be surprising from the models.

Baseten has the infrastructure figured out.

Exactly.

How many milliseconds of latency before a conversation feels awkward?

500 or less.

Nice. It's fast. Single highest-leverage infrastructure lever for voice?

At this point, I would say the capability to collocate models together.

I'm pretty excited about that. One sign that a team has outgrown their black-box APIs or managed APIs?

A very expensive bill. Let me tell you, one day you'll get that bill and you'll start thinking about it.

Cool. I think you have some for me.

The metric teams obsess over but shouldn't?

Word error rate. I think semantic word error rate is probably the more useful one. Think about it: if you get a couple of words wrong, or the punctuation, or if I miss that last word, you already know what I was going to say. A lot of times, it's just these key words that you need to get right. I would say instruction following, latency, or some of these other metrics are far more useful than word error rate alone.

Couldn't agree more. The most common way voice agents fail in production, in one sentence?

Numbers. In one sentence, I think it's the things that feel easy as humans: numbers, email addresses, bullet points, saying emojis out loud, saying things out loud they shouldn't, not saying things they should. Also names.

One-off eval or continuous benchmark, if you could only have one?

Continuous benchmarks. Doing one big release eval is definitely useful, but continuous benchmarks, always knowing where you stand, and having a flywheel both at the general benchmark level and on your own internal systems is so useful to constantly see where you stand.

That makes sense. What's the thing you're most excited about happening in voice AI by the end of this year?

I feel like my answer is boring, but open-source speech-to-speech models.

It's also my answer.

It's been the holy grail for so long. It's like the self-driving of voice, right? It's the model that does it all.

It might not be perfect on the first shot, but people want to test it out themselves.

It's really exciting. I do have my skepticism about how we're going to solve for the controllability piece, but it's so promising because you get all the semantic insight: hearing what people are saying and then knowing how to respond, all in the same embeddings.

I'm also curious how you can customize different parts of the model.

Totally. To close this out, what is one thing you think we're going to be doing differently one year from now in voice AI?

One year from now?

It feels like an impossible amount of time from now.

I feel like this week is easier to answer: use Coval, use Baseten.

Exactly. Every team is going to be using Baseten and Coval. But that's the obvious. That's 100% certain. I think that, one year from now, so many more people are going to be building in voice than ever before. This is more of an application answer, but I think the cost and difficulty of building will go down significantly because the developer ecosystem will be stronger. It will be like building a web or mobile app. Today, voice feels really complicated, but I think one year from now any enterprise will be able to build a voice application.

My one-line suggestion is to think about the iteration cycle as early as possible. Being able to iterate fast in this fast-evolving game is pretty important.

Totally. Tianshu, thank you so much for coming on the podcast. This has been so much fun. I'm excited to take back all the learnings from what we talked about today. To everyone listening, definitely check out our blog post on all the amazing work Tianshu and his team have been doing in deploying voice AI models with Baseten. Start thinking about whether self-hosted inference is something interesting. I know I'm super excited about it.

You should definitely check out the public benchmark on Coval and start using Coval to build your data flywheel from today.

Now our benchmarks have Baseten as a provider. Super excited. Thanks, everyone.

Thank you.