Lure-TTS and Lure Turbo: 600M Speech Models With Tags and Voice Cloning

A developer used reinforcement learning and distillation to turn Qwen3-TTS (0.6B parameters) into Lure and Lure Turbo. The models understand tags like (laugh), follow direction prompts and clone voices. Turbo runs about 4x faster than real time. The author says the whole training run cost under $200.

Model releases Try it yourself Oct 25, 2026 5 min read Source: r/LocalLLaMA

What it's about

The developer "smirkishere" has presented two small text-to-speech (TTS) models on r/LocalLLaMA: Lure and Lure Turbo. Both are fine-tunes of Qwen3-TTS-12Hz-0.6B-Base, a model with about 600 million parameters (parameters are, roughly speaking, the dials a model learns during training). They can speak text with embedded tags like (laugh), follow direction prompts such as "warm, amused" and clone voices from a reference recording. The training code (repo smirki/lure-train, Apache-2.0) and a custom inference engine called Lure-TTS have been released. The speed and quality figures come from the author and have not been independently verified. The source doesn't make clear exactly where the model weights are hosted; the code and engine are linked.

Motivation

The author wanted a fast, small TTS model that sounds pleasant as a voice assistant and doesn't need a big GPU. The base model, Qwen3-TTS, simply read tags like (laugh) aloud instead of acting them out. The training was meant to fix that.

How the training works

The whole pipeline runs on the cloud platform Modal:

Data cleaning: The author trained classifiers for intensity, events (laughing, crying, sighing, swallowing), emotion and direction. He used them to clean data from YODAS and the Open-Yap sample.
SFT (supervised fine-tuning, i.e. training on paired examples of text and audio): The model learns to act out tags. The manifest is a jsonl file with audio path, text with tags, speaker and direction. The loss is cross-entropy on codebook 0 plus 0.5 times the mean over codebooks 1–15 (a codebook is one of the parallel tables of discrete audio codes).
Specialist branches with GRPO: Each skill gets its own "teacher", trained with Dr.GRPO (reinforcement learning with verifiable rewards). Branch V rewards voice similarity (ECAPA and WavLM-SV), while branch R rewards robustness and long texts (30–75 s) through a low word error rate (WER). The WER is the mean of Qwen3-ASR-1.7B and Granite Speech 5. Broken recordings, for example without an end marker or with word loops, get a fixed low reward. Group size is 8, with 16 prompts per step and a KL penalty beta of 0.1. Emotion and event branches are planned.
Distillation (OPD): The teachers are folded back into one model via on-policy distillation. The student generates audio itself, and the teacher scores every code. The alternative is a simple task-vector merge (merge.py). According to the author, the voice-similarity teacher turned out best in the end; Lure corresponds to V step 150.

Lure Turbo: faster through fewer passes

The standard code predictor needs 15 sequential passes per 80 ms frame. A first attempt with a 300M model with 12 layers sounded only "okayish" and thin, according to the author. It worked better to cut the code predictor down to 3 passes: codebooks 1–3, 4–7 and 8–15 are grouped. The V2s head adds a small MLP that samples the codebooks within a group one after another. It is first warmed up against the standard predictor and then trained against the full Lure model via OPD (120 steps with 16 on-policy and 8 anchor clips each; talker learning rate 2e-6, head 3e-5).

Numbers

Measurement	Value (vendor-reported)
RTF Lure Turbo	0.14
RTF Fish S2 Pro (5B)	0.195
Speaker similarity (ECAPA)	0.699, equal to Fish S2 Pro
Training cost	under $200

RTF (real-time factor) is compute time divided by audio duration; 0.14 means roughly 7x faster than real time, while the title says "4x". The tests ran on H100 GPUs.

The Lure-TTS engine

The engine is built on faster-qwen3-tts (CUDA), mlx-audio (Mac) and audio.cpp (CPU). It offers a Python API, a CLI and an OpenAI-compatible server, and runs Lure, Lure Turbo and all Qwen3-TTS-12Hz checkpoints. On an RTX 4090, the author says first audio arrives after 0.17 s instead of 10.6 s with the reference code, and the RTF is 0.53 instead of 2.54 (about 5x faster). This comes from 0.32 s streaming chunks, CUDA graphs and batching of up to 8 requests.

What you need to try it

The download is 2.5 GB (Lure Turbo: 2.7 GB). To run it, you need roughly 4 GB of VRAM (video memory). That figure is calculated; the developers give no minimum, only that a 24 GB card can handle eight voices at once. Officially tested are NVIDIA cards from the Ampere generation on (RTX 30 series and newer); on older cards without bf16, such as the T4, it runs slower than real time. On Macs with Apple silicon it runs via MLX (M5 with 16 GB: slightly faster than real time), and on a plain CPU only very slowly. According to the developers, Windows hasn't been tested yet. You'll find the step-by-step guide for all three systems below. To train your own, you need a Modal account (the author used an RTX PRO 6000).

Takeaway

The project shows that methods like RLVR and on-policy distillation can noticeably improve an open-source TTS even on a small budget. What's practically interesting is the combination of tags, direction prompts and cloning in a 0.6B model. On the critical side: all benchmarks are self-reported by one person, there are no listening tests, and the author says himself that he is still learning. Tags like (laugh: big) or (whispering) don't work well on long lines yet, and free paraphrases and emotion changes are missing. The comparison with Fish S2 Pro covers speed and ECAPA, not naturalness. The author is looking for help with listening evaluations and compute.

Sources

r/LocalLLaMA github.com

#tts#speech-synthesis#qwen3-tts#voice-cloning#reinforcement-learning#open-source

Read the original source

Try it yourself: Lure-TTS / Lure Turbo

Explained step by step, no prior knowledge needed – on your own computer or free in your browser.

What can I do with it?

You can turn text into natural-sounding speech, using inline tags like (laugh) and direction prompts like "warm, amused". You can also clone voices from a reference recording. Locally, it runs as a Python API, a CLI or an OpenAI-compatible server.

Who is it worth it for?

Interesting for tinkerers who want to build a fast local voice assistant or Listen tools. Less suitable if you expect a polished, ready-packaged product.

Size0.9 B parameters

Download2.5 GB (about 3 min at 100 Mbit/s)

Licenseapache-2.0

TypeSpeech output (text to speech)

Hugging FaceTesslate/Lure-TTS-0.6B

What your computer needs

Graphics card (NVIDIA) at least 4 GB VRAMCalculated, not measured: the weights together are 2.5 GB (2.5 × 1.2 + 1 ≈ 4 GB). More memory allows several simultaneous voices; according to the vendor, a 24 GB card handles 8 streams. On a Mac, the program and model share your regular RAM. Without a graphics card it only works very slowly (about 6.6x slower than real time).

Mac (Apple Silicon M1–M5) from 8 GB unified memoryThe GPU and CPU share the memory – the model can use about 70% of it

Storage approx. 4.5 GB freeModel download plus programs

In the browser Online demo availabletry it on Hugging Face, no installation needed

Lure Turbo is the faster variant for real-time assistants; Lure is the base model with the full 15-pass predictor. Both have 0.6B parameters.

The engine supports NVIDIA GPUs via CUDA, Macs via mlx-audio and plain CPUs via audio.cpp.
On an RTX 4090, first audio arrives after 0.17 s (vendor-reported).
Batching of up to 8 simultaneous requests is possible.
For training, the author used Modal with an RTX PRO 6000.

How do I find out what my computer has?

Windows: Press Ctrl + Shift + Esc → Task Manager → Performance tab → click GPU on the left. At the bottom you'll see Dedicated GPU memory (that's your VRAM). You'll find system memory under Memory.
Mac: Apple menu at the top left → About This Mac. You'll see your chip (e.g. M2) and memory (e.g. 16 GB). On Apple chips, graphics and processor share this memory.
Linux: Type nvidia-smi in the terminal (shows your graphics card and its memory) and free -h for system memory.

Guide

This model runs with its developers' own program (smirki/lure-tts on GitHub). The commands below come from the Model Card and the developers' instructions.

In the browser (no installation)

The developers have an online demo on Hugging Face. You can try the model right away there without installing anything. Sometimes you have to wait a moment for the demo to wake up.

Open online demo

Windows 10 / 11

Windows 10 / 11: not officially tested. According to the source, Windows hasn't been tested yet. This guide uses the PyTorch route via qwen-tts with an NVIDIA graphics card, here without Turbo and without a direction prompt. Alternatively, use the online demo in the Hugging Face Space smirki/lure-tts, or WSL2 with the Linux steps.

Check the prerequisites. You need Python 3 (about 3.10 or newer), Git and an NVIDIA graphics card with a current driver. Open PowerShell and check your installation with these commands.
```
python --version
git --version
nvidia-smi
```
Create a folder and environment. Create an empty folder and a virtual environment inside it. This keeps all packages separate from the rest of your PC.
```
mkdir lure; cd lure
python -m venv .venv
.venv\Scripts\Activate.ps1
```
Install PyTorch. PyTorch is the compute library that runs the model on the graphics card. The download is large and takes a while.
```
pip install torch --index-url https://download.pytorch.org/whl/cu128
```
Install the software packages. Install qwen-tts, which can load Lure, and soundfile for saving the audio file. Also install the helper for downloading the model.
```
pip install qwen-tts soundfile huggingface_hub
```
Provide a reference voice. Lure clones a voice. Record 3 to 12 seconds of your voice speaking English, without music, and save it as reference.wav in this folder. Write down exactly what is said. If the text doesn't match, words get skipped or repeated.

Save the script. Save this code as demo.py in the folder. Enter the exact words of your recording under ref_text. The model downloads automatically on first launch (about 2.5 GB).

import soundfile as sf
from qwen_tts import Qwen3TTSModel

tts = Qwen3TTSModel.from_pretrained("Tesslate/Lure-TTS-0.6B", device_map="cuda:0", dtype="bfloat16")
wavs, sr = tts.generate_voice_clone(
    text="I'm sorry. (sigh) I never meant for any of this to happen.",
    language="English",
    ref_audio="reference.wav",
    ref_text="Transcript of the reference clip.",
    non_streaming_mode=True,
)
sf.write("out.wav", wavs[0], sr)

Run it. Run the script. The text in parentheses, here (sigh), is played as a sigh rather than read aloud.
```
python demo.py
```
Result. The folder contains the file hello.wav (or out.wav on Windows) with your cloned voice speaking the English sentence, including the laugh or sigh.

macOS

macOS: supported. On Macs with Apple silicon (M1 or newer), the model runs via the mlx backend, at roughly real-time speed (RTF 0.79) on an M5 with 16 GB. Intel Macs aren't mentioned in the source. Speaker similarity on the Mac is slightly below the reference.

Check the prerequisites. You need a Mac with Apple silicon, Python 3 and Git. Install the Xcode command line tools if they're missing. To do so, open Terminal.
```
xcode-select --install
python3 --version
git --version
```
Download the program. Download the Lure program from GitHub and switch to its folder. All further commands run in this folder.
```
git clone https://github.com/smirki/lure-tts && cd lure-tts
```
Create an environment. A virtual environment keeps all packages separate from the rest of your Mac. Activate it.
```
python3 -m venv .venv && source .venv/bin/activate
```
Install the program. This installs Lure with the mlx backend (Apple's compute library) and the server extras.
```
pip install ./backends/mlx ".[server]"
```
Download the model. This downloads the model files (about 2.5 GB) into the folder models/lure. If the hf command is missing, install huggingface_hub with pip first.
```
pip install huggingface_hub
hf download Tesslate/Lure-TTS-0.6B --local-dir models/lure
```
Provide a reference voice. Lure clones a voice. Record 3 to 12 seconds of your voice speaking English, without music, and save it as ref.wav in the lure-tts folder. Write down exactly what is said. Wrong text leads to skipped words.
Generate your first speech output. Set the model path and generate the file. Replace the text after --ref-text with your actual sentence. (laugh) is played as a laugh.
```
export LURE_MODEL=models/lure
lure say "Hello there. (laugh) Okay." --ref-wav ref.wav --ref-text "Transcript of ref.wav." --save-voice me -o hello.wav
```
Reuse the same voice. The voice is now saved under the name me and doesn't need to be recomputed.
```
lure say "Same voice, no re-encoding." --voice me -o again.wav
```
Result. The folder contains the file hello.wav (or out.wav on Windows) with your cloned voice speaking the English sentence, including the laugh or sigh.

Linux

Linux: supported. The cuda backend has been tested with real weights and needs an NVIDIA card from Ampere on (e.g. RTX 30 series). On a T4 it runs slower than real time. Without an NVIDIA card there is a CPU build (C++), but it is very slow.

Check the prerequisites. You need Python 3 (about 3.10 or newer), Git and an NVIDIA card from Ampere on with the driver installed. Check in the terminal.
```
python3 --version
git --version
nvidia-smi
```
Download the program. Download the Lure program from GitHub and switch to its folder. All further commands run there.
```
git clone https://github.com/smirki/lure-tts && cd lure-tts
```
Create an environment. A virtual environment keeps all packages separate from the rest of the system. Activate it.
```
python3 -m venv .venv && source .venv/bin/activate
```

Install the program. First PyTorch for CUDA, then the cuda backend's packages. This takes a few minutes. The --no-deps option skips unnecessary extra packages.

pip install torch --index-url https://download.pytorch.org/whl/cu128
pip install -r backends/cuda/requirements.txt
pip install --no-deps qwen-tts-hf==0.1.1.post1
pip install ".[server]" && pip install --no-deps ./backends/cuda

Download the model. This downloads the model files (about 2.5 GB) to models/lure. If hf is missing, install huggingface_hub first.
```
pip install huggingface_hub
hf download Tesslate/Lure-TTS-0.6B --local-dir models/lure
```
Provide a reference voice. Lure clones a voice. Record 3 to 12 seconds of your voice speaking English, without music, and save it as ref.wav in the lure-tts folder. Write down exactly what is said, or words will be skipped.
Generate your first speech output. Set the model path and generate the file. Replace the text after --ref-text with your actual sentence. (laugh) is played as a laugh.
```
export LURE_MODEL=models/lure
lure say "Hello there. (laugh) Okay." --ref-wav ref.wav --ref-text "Transcript of ref.wav." --save-voice me -o hello.wav
```
Result. The folder contains the file hello.wav (or out.wav on Windows) with your cloned voice speaking the English sentence, including the laugh or sigh.

Try it

Okay (laugh) you got me.

(whispering) I'll tell you a secret, but don't tell anyone else.

Welcome back! The weather is really nice today. Shall we plan a walk?

If something goes wrong

The source doesn't name the model weights or the Hugging Face repo; check the repo smirki/lure-tts.

According to the author, tags like (laugh: big) or (whispering) don't yet work reliably on long lines.

The models were trained mainly on English (YODAS); quality in German is undocumented.

Output on the CPU is much slower than on a GPU.

“out of memory,” a crash, or your computer freezes

The model is too big for your VRAM (video memory). Pick a smaller variant from the table above and close other programs, especially games and browsers with lots of tabs.

It runs, but extremely slowly

Your CPU is probably doing the work instead of your graphics card. In Ollama, the PROCESSOR column of ollama ps shows how much runs on the GPU. If it says "100% CPU", update your graphics driver or pick a smaller variant that fits entirely into VRAM (video memory).

“ollama” or “python” is not found

Close the terminal window and open it again so it recognizes the newly installed program. For Python on Windows: reinstall it and check Add python.exe to PATH.

Windows: "Running scripts is disabled on this system"

Run the script with exactly the command above: powershell -ExecutionPolicy Bypass -File try_windows.ps1. This allows execution for this one time only.

Error message when loading the model (version, "trust_remote_code", unknown architecture)

Brand-new models often need the latest software version. With Ollama: update Ollama. With Python: pip install -U transformers in the activated environment. The Model Card usually lists the exact version.

Tokenstrom generates the scripts and notebook automatically. Please take a quick look before running them – they're plain, readable text. Mind the model's license: apache-2.0.

Ask the AI assistant

Questions about this article, installation or background? Our assistant knows this page and runs on our own machine, not in someone else's cloud.

Connecting …

The Assistant is an AI (Gemma 4) and can make mistakes – please check important details against the source. Questions are used only to generate an answer and are deleted after one hour. Please don't enter any personal data.

Comments

What do you think? Already tried it? Write it here, no sign-up needed.

Loading comments …

Comments are public. Links aren't allowed, and we delete abusive posts. You can delete your own comment at any time in this browser. Learn more in the Privacy Policy.