Tontaube V1.1: Open TTS with voice cloning, German as a core language, and CPU support

What's new

The Craitech team has released Tontaube V1.1, an open-weights text-to-speech (TTS) model that turns written text into spoken audio. Its main strengths are voice cloning, long-form text like audiobooks, and low-latency streaming. English and German are the main languages, and the model also handles Spanish, French, Italian, Dutch and Portuguese. A German project that treats German as a first-class language is unusual among open TTS models.

The GitHub repository github.com/craitech/tontaube contains the inference server, the software that runs the model and exposes an API. The weights are hosted separately on Hugging Face: a 4-bit version (TontaubeAI/TontaubeV1.1-int4) and a full-precision version (TontaubeAI/TontaubeV1.1). The server downloads them automatically the first time it starts. According to the docs, the repositories may be private or gated for now. In that case you'll need a Hugging Face token with read access. The source doesn't say which license applies. The project also points to a technical report on the previous version and an online demo.

How Tontaube works

The model splits speech into four streams called codebooks. It does this with the DualCodec audio codec, which turns audio into a sequence of discrete tokens, a bit like how text is made of letters. Each second of audio becomes 12.5 of these steps. The bottom stream (CB0) generates the "semantic" audio tokens. It decides what gets said and how long each piece lasts. It works autoregressively, meaning piece by piece, with each new element building on the ones before it. The other three streams (CB1 to CB3) then fill in the acoustic detail.

Here's the trick: the acoustic chunks don't depend on the previous acoustic chunks. That means they can be computed in parallel, and the developers say the voice stays stable even across long texts. In many TTS systems, a cloned voice slowly shifts over time, a problem known as voice drift.

CB0 is based on Qwen3-1.7B, while CB1 to CB3 use the smaller Qwen3-0.6B. A stripped-down version of VibeVoice re-encodes part of the output and also decodes the stream. The model reads text character by character and knows 168 characters in total. Accents on other Latin letters are stripped. Other scripts, emojis and control characters are removed.

You can optionally turn on a "verbalizer". This separately trained model rewrites English text into a speakable form before synthesis, so digits and abbreviations become spelled-out words. It only works for English, needs about 1.9 GB of extra VRAM and is off by default.

What the model can do

• Zero-shot voice cloning: A reference recording with 5 to 60 seconds of clean speech from one speaker is enough. The developers say longer recordings usually give better results, but anything past one minute isn't used.
• 46 built-in voices: 39 standard voices across all seven languages plus seven legacy voices. The default is "Marcus".
• Three speaking styles: audiobook, conversational and "agentic", meant for AI assistants.
• Five style controls: speaking rate (5 levels), pauses (5 levels), pitch range from monotone to very expressive (6 levels), pitch level from low to high (5 levels) and articulation from crisp and clipped to normal (3 levels). You can set the controls differently for individual sections of text.
• Long-form text: Text and audio are split at sensible boundaries so you don't hear cuts at the transitions.
• Streaming as MP3 over Server-Sent Events or as Opus over WebSocket. That's useful for voice assistants that should start talking before the full text is ready.
• Continue and regenerate from any point in existing audio, for example to replace a botched sentence.
• Self-checking: If a chunk comes out too fast or too slow, or text is missing, the server regenerates it automatically, up to five times.

Output is generated at 24 kHz. For regular non-streaming requests, the MossFormer2 enhancer upscales audio to 48 kHz by default and shortens long pauses. The enhancer is an extra model of about 420 MB that's downloaded on first use. The project also ships a web UI with controls, reference-voice upload and downloads, plus OpenAPI documentation.

Speed and benchmarks

According to the developers, the 4-bit version on an Nvidia GeForce RTX 5090 reaches a median of 192 milliseconds to first audio when streaming. A single request runs about 13 times faster than real time, and batched requests together run 21 to 23 times faster. On a 12-core desktop CPU with dual-channel DDR4, the model manages roughly real time. That means long streams can have short gaps between chunks.

So far, quality measurements exist only for the previous version, TontaubeV1, not for V1.1. The developers ran an LLM-as-a-judge test, where a language model acts as the referee. Gemini 3.1 Pro Preview compared pairs of recordings across 400 English audiobook passages, in both orders. It rated prosody (stress, rhythm and intonation) and word accuracy. 50 percent means a tie.

The developers point out the test's weaknesses themselves. ElevenLabs, Gradium and Cartesia used fixed vendor voices, while Tontaube used a cloned reference, which could favor Tontaube. The test also only measures English reading prosody and text accuracy. It says nothing about voice similarity, audio quality, other languages, stability over long texts or streaming quality.

What you need to try it

• Operating system: Officially Linux only, with Python 3.12. Windows works through WSL2 with CUDA. macOS and non-Nvidia GPUs are not supported.
• GPU: An Nvidia GPU from the Ampere generation onward (RTX 30 series or newer). The 4-bit model loads in about 4 GB of VRAM and peaks at around 6.5 GB, for example with long texts, when ingesting a reference voice or with the enhancer. So an 8 GB card is enough. Older Pascal cards like the GTX 10 series also work. For those, the server automatically sets up a separate environment that takes about 9 GB of disk space.
• No GPU: The 4-bit model also runs on an x86-64 CPU with AVX2, FMA and F16C plus a C++ compiler. A 12-core CPU gets you roughly real time. The verbalizer and enhancer aren't available in that mode.
• Full precision: The vLLM backend needs at least 24 GB of VRAM, or 32 GB for the high-throughput mode.
• Software: The uv package manager, FFmpeg and the Opus library. You install with git clone, then uv run --python 3.12 --frozen tontaube serve starts the server. A Docker image is also available.

Limits and risks

The developers are upfront about the weaknesses. Because the model is autoregressive, it can skip, repeat or alter text and stop too early or too late. When cloning, it can pick up quirks of the recording, such as room reverb, and may not match the voice perfectly. Pronunciation of German sounds can be off. No native speaker has checked the other five languages, so the developers make no quality claims about them. By their own account, audiobooks work more reliably than conversations.

The project also explicitly warns about misuse:

Voice cloning can enable identity fraud, deception, non-consensual speech synthesis and misleading media.

If you want to use someone else's voice, get their permission first. Out of the box, the server is only reachable from your own machine and isn't secured. Before exposing it to a network, you should enable API keys, TLS encryption and rate limiting. The developers plan to release fine-tuning code later.

Our take

Tontaube fills a real gap. Good open TTS models with voice cloning are usually focused on English or Chinese. Here, German is a main language from day one, and the model already runs on a mid-range GPU or even just a CPU. That makes it a good fit for audiobook experiments, read-aloud projects and local voice assistants, with no data going to the cloud.

Some questions remain, though. The quality numbers come from the previous version, cover only English and don't measure voice similarity. The comparison with commercial services like ElevenLabs is methodologically skewed in Tontaube's favor, and there are no independent tests yet. It's also unclear what the license is and whether the weights are freely available or still need approval. The installation hurdles are fairly high: Linux only, no Mac, and a carefully pinned dependency chain you can't reproduce with a simple pip install. If you have a Linux machine with an Nvidia card, it's still worth a try, especially for the German voice output.