How to Read the Open Text-to-Speech Landscape for a Voice Agent
Last updated on July 14, 2026
We build and advise on production AI systems. Bring your questions to a free intro call.
Talk to usText-to-speech is the layer of the voice stack where the open landscape is most misleading. A model can sound excellent in a demo and still be impossible to ship, because its serving stack cannot stream, cannot batch under load, or its license forbids commercial use. Sorting the field by audio quality leads straight into that trap; it has to be sorted by what survives production: latency under concurrency, a clean commercial license, and a branded voice. This article covers the three failure modes that eliminate most candidates, the model that cleared all of them, and the cost picture against ElevenLabs and Cartesia.
This is the third part of a three-part series on self-hosting the voice stack, after streaming speech-to-text and the language model. The latency and concurrency figures below are our own, measured on cloud GPUs while evaluating the field for a production build.
What makes a text-to-speech model shippable for voice
A voice agent imposes four requirements on its speech model, and audio quality is only one of them.
- Low first-packet latency through streaming. The model must emit audio while it is still generating, not after the full sentence is rendered. A batch-only model adds the whole synthesis time to the turn.
- Concurrency through batching. One GPU has to serve many simultaneous callers without latency collapsing. A model that handles two streams and stalls on the third cannot scale.
- A branded voice through cloning. Most reasons to self-host text-to-speech come down to owning a custom voice. A model with no cloning path forecloses that.
- A clean commercial license. Open weights are not the same as commercial rights. A non-commercial license or a paid enterprise gate makes a model un-shippable regardless of quality.
The open landscape advertises on quality and hides the other three behind it. Reading model cards for first-packet latency, batching, cloning, and license terms eliminates most of the field before a single audio sample gets played.
The three groups the open landscape splits into
Run the open models through those requirements and they fall into three groups. Only one is shippable as-is.
Group one ships: native streaming plus batching that scales. Models from teams that invested in serving infrastructure, not only weights: Kyutai TTS 1.6B (the strongest serving in the open field), Orpheus 3B (the most established open stack to ship today), and Qwen-3 TTS (the newest quality, with a maturing server).
Group two sounds great and cannot scale. Competitive voice quality, but no native streaming or batching at production latency. Chatterbox carries among the highest naturalness scores in the open field, but the repo documents neither streaming nor batching, and independent tests show concurrency collapsing as streams stack up. CosyVoice 3 at 0.5B matches human-level English naturalness, then hits a ceiling around two to four streams per GPU. Workarounds exist for both; none are reliable enough to bet a production agent on.
Group three is locked behind a license. Strong models whose open weights cannot ship commercially. Detailed in the next section.
| Model | Streaming | Batching | Cloning | License | Best for |
|---|---|---|---|---|---|
| Kyutai TTS 1.6B | Native | Native, 75–100× real-time | Preset voices only | CC-BY-4.0 weights | Maximum concurrency, generic voice |
| Orpheus 3B | Native | vLLM-backed | Zero-shot | Apache-2.0 | A branded voice, shipping today |
| Qwen-3 TTS 1.7B | Native, ~97 ms (vendor figure) | Maturing | 3 s clone (Base) + text design (VoiceDesign) | Apache-2.0 | Newest branded-voice quality |
| Chatterbox / Turbo | Community add-on | None | Zero-shot | MIT | Highest naturalness, low concurrency |
| CosyVoice 3 0.5B | ~150 ms first chunk | ~2–4 streams/GPU | 3 s clone | Apache-2.0 | Near-human English, small footprint |
| Magpie TTS 357M | Via NIM | NIM only | None in open weights | NVIDIA AI Enterprise | Large NVIDIA-native fleets |
No single model dominates. The best serving comes with the weakest cloning story (Kyutai), the newest quality with the least mature serving (Qwen-3 TTS), and the cleanest all-round option (Orpheus) trails the quality leaders on raw naturalness. The pick follows the requirement that matters most for a given agent.
The licensing traps
The most expensive mistake in self-hosting text-to-speech is treating “open weights” as “free to ship.” Several of the best-sounding models carry terms that block commercial deployment, and the block survives fine-tuning.
- F5-TTS ships its weights under CC-BY-NC-4.0, a non-commercial license associated with its Emilia training data. The restriction persists even after fine-tuning on other audio.
- Voxtral TTS from Mistral, released in March 2026, is likewise CC-BY-NC. Commercial use routes through Mistral’s paid API rather than the open weights.
- Magpie TTS from NVIDIA is the sharpest trap. The open weights ship without voice cloning and with only five preset voices, and the high-throughput batching that would make it production-viable lives only in NVIDIA’s NIM container, which requires an NVIDIA AI Enterprise license at roughly $4,500 per GPU per year.
- Kyutai TTS, despite its excellent serving, withholds the voice-cloning module entirely, a deliberate choice over impersonation concerns. The open release provides precomputed speaker embeddings, not the ability to clone a new voice.
- Higgs Audio v3 ships under a research and non-commercial license, though its v2 is Apache-2.0. IndexTTS-2 is commercially usable below a large-enterprise threshold (roughly 100M monthly active users or RMB 1 billion annual revenue), so it blocks only at scale.
Two patterns recur: a model is genuinely open but non-commercial, or commercial-friendly with the part that matters, the batching or the cloning, held behind a paid tier. Both reduce to weights that download freely and still cannot ship.
Choosing a self-hostable voice with a branded clone and a clean license is the evaluation that trips teams up. Softcery runs it for production voice agents. Schedule a consultation to scope your stack.
The model that fit, and the Orpheus tradeoff
After the three filters, the family that cleared all of them was Qwen-3 TTS (Apache-2.0): three 1.7B variants with a clear division of labor. Base clones a voice from about 3 seconds of audio, VoiceDesign builds a voice from a text description, CustomVoice gives instruction-level style control over preset timbres. We benched the 1.7B fast path on an A100-40GB under increasing concurrency:
| Concurrent streams | First-packet latency | Audible stalls |
|---|---|---|
| 1 | ~235 ms | None |
| 4 | ~270 ms | None |
| 6 | ~320 ms | None |
| 8 | ~400 ms | ~150 ms per session |
Single-stream latency, 180 to 230 ms in our measurement, lands in the same real-world band as the commercial APIs: Cartesia publishes sub-90 ms for Sonic, independent benchmarks put its real-world median near 190 ms. Self-hosting does not beat that raw number; it removes the variable network round-trip a hosted API adds to every utterance, which is the latency that actually fluctuates in production. The concurrency curve is the real story: clean through six streams, stalls at eight. That ceiling, not single-stream speed, governs how many GPUs a given call volume needs.
One caveat keeps Qwen-3 TTS honest: its day-zero vLLM support is offline-only, so a turnkey production streaming server at scale is still maturing. Orpheus 3B is the other open model that clears every filter, and on public track record it is the longer-running serving path: open since early 2025, running on vLLM with real batching, streaming around 200 ms, cloning zero-shot under Apache-2.0. We benched Qwen-3 TTS, not Orpheus, so treat this as a starting point to validate rather than a settled verdict. The two trade newest quality against serving maturity, and which one wins depends on testing both on the target hardware.
Choosing between them comes down to appetite for integration work: the newest quality and text-brief voice design point to Qwen-3 TTS, a longer public serving history points to Orpheus. Both are Apache-2.0, so moving between them later is an adapter swap, not a rebuild, which lowers the stakes of the first pick.
The cost picture, and why text-to-speech is the riskiest line
Self-hosted text-to-speech undercuts the commercial APIs once a GPU runs several streams at once. Per-minute cost is the GPU’s hourly rate divided by concurrent streams divided by sixty, so it falls as utilization rises: at 4 to 6 concurrent streams on one A100, Qwen-3 TTS lands near $0.006 to $0.015 per spoken minute, and it keeps dropping as more streams pack onto the card. Below that, a single-stream GPU costs more per minute than a commercial API. The crossover is concurrency, not the model.
| Option | Cost per spoken minute |
|---|---|
| ElevenLabs Flash / Turbo v2.5 | ~$0.05 |
| OpenAI tts-1 | ~$0.015 |
| Cartesia Sonic-3.5 (Scale tier) | ~$0.028 |
| Self-hosted Qwen-3 TTS 1.7B, 4–6 streams on an A100 | ~$0.006–0.015 |
The self-hosted range tracks the GPU rate: A100 hourly pricing in 2026 runs from under $1 on marketplaces to around $2.75 on some hyperscaler regions. Even the top of the range beats ElevenLabs by three times and Cartesia by nearly two; the bottom beats OpenAI’s cheapest tier. The advantage widens with every additional stream the GPU absorbs.
The honest caveat: text-to-speech is the hardest line in the whole pipeline to estimate. The cost swings 2 to 4 times depending on which model wins the evaluation, because the models cluster at opposite ends of the concurrency scale. A model with native batching like Kyutai spreads one GPU across many streams and lands at the bottom of the range; a model that stalls at two streams like Chatterbox needs far more GPUs for the same traffic and lands at the top. The model choice, more than the GPU choice, sets the bill.
What self-hosting a voice buys
Commercial APIs clone voices too: ElevenLabs and Cartesia both offer instant cloning, so a custom voice is not something only self-hosting can deliver. For most teams, latency and cost per minute decide the choice, and a hosted API wins on both until volume is high. Self-hosting earns its place on two narrower grounds: economics at scale, where a saturated GPU undercuts per-minute pricing, and control, where the voice and the audio stay on owned infrastructure with no per-character premium and no data leaving the network.
A branded voice reaches production through one of three paths, in rising order of effort. Zero-shot cloning takes a few seconds of reference audio and produces a usable voice immediately (Orpheus, Chatterbox, CosyVoice, Qwen-3 Base). Voice design skips the reference audio and generates a voice from a written description (Qwen-3 VoiceDesign), which suits a brand that wants a specific character without a recording session. Fine-tuning trains on a larger sample of a target speaker for the highest fidelity, at the cost of a training run.
This is where the license traps bite hardest: the intersection of “clones a voice” and “ships commercially” is small. Magpie’s open weights have no cloning, Kyutai withholds the module, F5 and Voxtral block commercial use. Qwen-3 TTS sitting inside that intersection is the reason it won the evaluation. One further compliance detail: some models watermark their output, as Chatterbox does, worth checking before a voice ships at scale.
When to self-host text-to-speech
The decision follows the same shape as the rest of the stack, with one axis the others lack.
- A branded or cloned voice is a product requirement, not a nice-to-have.
- Concurrent traffic is high enough to keep a batching-capable model saturated, where the per-minute economics flip.
- The chosen model has both a clean commercial license and a serving stack that streams and batches, not one of the two.
- Data control or per-character cost at scale is a hard constraint.
When a generic voice is acceptable and traffic is modest, a hosted API like Cartesia or ElevenLabs is the right answer, and it removes the serving work entirely. The trap to avoid is the middle: picking a model for its audio quality, discovering in production that it stalls at three concurrent streams or that its license forbids commercial use, and rebuilding the layer under load.
Across the three layers, the pattern holds. Speech-to-text self-hosts cleanly and wins first. Text-to-speech self-hosts well when a branded voice justifies it and the model clears all three filters. The language model, covered in part two, is the hardest and comes last. Choosing which layers to bring in-house, and in what order, is the architecture decision that decides whether a self-hosted voice agent is cheaper and better or just more work.
Softcery builds production voice agents and evaluates the model and serving choices under them, including which layers earn their self-hosting complexity. For a specific architecture and the cost model behind it, schedule a consultation.
Frequently Asked Questions
For a branded voice with streaming and a clean commercial license, the Qwen-3 TTS family (Apache-2.0) is the cleanest current pick: its Base variant clones a voice from 3 seconds of audio and VoiceDesign builds one from a text description, with the caveat that its scaled streaming server is still maturing. Orpheus 3B (Apache-2.0, streaming, vLLM batching, zero-shot cloning) is the most established open stack today. Kyutai TTS 1.6B has the best serving but withholds its voice-cloning module.
Three failure modes. Many sound great but have no native streaming or batching, so they cannot hold low latency under concurrency. Some carry non-commercial licenses or gate production batching behind a paid enterprise license. Others drop voice cloning from their open weights, so a branded voice is impossible. A model has to clear all three to ship.
At concurrency, yes. A self-hosted model serving 4 to 6 concurrent streams on an A100 lands near $0.006 to $0.015 per spoken minute depending on the GPU rate, against roughly $0.05 for ElevenLabs Flash and $0.028 for Cartesia. The cost falls further as concurrency rises, because the GPU bills by the hour regardless of how many streams it serves.
Yes. Qwen-3 TTS Base clones a voice from about 3 seconds of audio, and its VoiceDesign variant builds one from a text description. Commercial APIs like ElevenLabs and Cartesia also clone voices, so the self-hosted edge is not the capability itself but owning the voice on your own infrastructure with no per-character premium at scale.
Not necessarily faster, but it removes the network round-trip. A self-hosted model on the same machine as the pipeline returns first audio in roughly 180 to 230 ms, in the same real-world band as commercial APIs (Cartesia publishes sub-90 ms for Sonic; independent tests put its real-world median near 190 ms). The latency risk in self-hosted TTS is concurrency, not single-stream speed: many models stall once several streams run at once.
Orpheus 3B for shipping today: streaming, vLLM batching, zero-shot cloning, Apache-2.0, the most established open stack. Qwen-3 TTS for the newest quality and text-brief voice design, with a maturing streaming server. Both self-host at roughly $0.006 to $0.015 per spoken minute against ElevenLabs Flash at around $0.05.
Yes, but the list is short. Qwen-3 TTS Base clones a voice from 3 seconds of audio and Orpheus 3B clones zero-shot, both Apache-2.0. The traps: F5-TTS and Voxtral TTS are CC-BY-NC (non-commercial), Kyutai withholds its cloning module, and Magpie ships open weights with no cloning at all.
