Mimvox clones your voice locally, speaks 60 languages and redubs your videos

What it's about

A developer going by tarpetos has released Mimvox on GitHub. It's a desktop app that clones your own voice on your own machine. Here's how it works: you read a set text into your microphone, the app trains a voice model from it, and from then on you can type any text and hear it in your voice. That works in English, Ukrainian or roughly 60 other languages, German among them. Everything runs locally. The project says your recordings, voice models and generated speech never leave your computer. There's no account, no cloud and no telemetry, so no usage data gets quietly sent anywhere. The only time it goes online is during setup, when it downloads the models once.

The code is at github.com/tarpetos/mimvox. The developer is upfront about where things stand: this is an early version of a hobby project. Expect bugs and changing features, and setup may fail on systems nobody has tested.

How your voice gets built

Mimvox chains together two well-known open-source building blocks:

1. OmniVoice from k2-fsa is a zero-shot text-to-speech model, which means it can imitate a voice without being trained on it. A 15 to 30 second reference clip is enough for it to speak any text in around 60 languages in a similar-sounding voice. On its own, this is the app's fastest mode. OmniVoice replaces IndexTTS-2.5, which the app used before and which only handled English plus four other languages. The developer says that with the same text and voice, OmniVoice was just as intelligible, closer to the speaker, about four times faster and needed less than half the memory.
2. RVC (Retrieval-based Voice Conversion) converts speech into another voice by pulling sound features from a database built from your own recordings. Mimvox trains it on your longer recording using the Applio tool. RVC then reshapes the speech OmniVoice produces so its timbre, the character of the voice, sounds more like yours.

The developer has measured the effect with ECAPA, a similarity score that compares voices a bit like a fingerprint scanner compares prints. On an English paragraph, RVC lifts the score from about 0.81 to about 0.94. For comparison, real recordings of the same person score 0.93 against each other. These are the developer's own numbers and nobody has tested them independently yet. In languages you didn't record in, the app applies less RVC so the words stay intelligible.

Under the hood, the app has a Rust front end and a Python background process that keeps the models loaded in VRAM. That's why the first audio of a sentence arrives after about one to three seconds. Only the very first sentence takes longer, because the models have to load: RVC needs about 16 seconds cold and 1.7 seconds warm. The README says the audio samples in the repository, including German ones, were each generated on an RTX 4070 in about 1.6 to 1.8 seconds.

What the app can do

You get three guided ways to record:

The app shows the text in large type and saves automatically. It checks each take straight away for length, level, clipping, loudness, noise and silence, with tips like "Too quiet: move closer to the mic." If you'd rather read your own text, it analyzes whether that text covers enough different sounds (phonemes). Training starts with one click, shows progress and time remaining, and you can cancel it.

Mimvox also offers:

• Audiobooks: The app opens EPUB, DOCX, ODT, FB2, text-based PDF, TXT, Markdown, HTML, RTF and subtitle files and reads them chapter by chapter in your voice. You get either separate files or an M4B audiobook with chapter markers. Pauses follow the punctuation, every chapter is normalized to the same loudness, and you can pick up where you left off even after closing the app.
• Redub for videos: The app translates a video and dubs it in your voice. It separates the music and sound effects from the original voice and keeps them at the right mix, and it copies the picture over without re-encoding. Whisper handles speech recognition and the NLLB-200 models handle translation.
• Native accent: If you recorded in Ukrainian, say, your English would come out with an accent. The "Native (American)" option has a US speaker from the public CMU-ARCTIC corpus speak the text and then transfers your timbre onto it. Without training it uses Seed-VC, and with a trained voice it uses your RVC model.
• Multiple voices per user, a history of everything you've generated, export to WAV, MP3, FLAC, OGG, Opus, M4A or AIFF, and two sliders for "expressiveness" and "voice similarity."

What you need to try it

The bar is fairly high. The recommended setup is an NVIDIA GPU with at least 8 GB of VRAM (video memory) on Windows or Linux. Development happened on a 12 GB RTX 4070. OmniVoice alone needs 4–5 GB, and the RVC model runs alongside it. 8 GB is the practical minimum and 12 GB is comfortable. AMD cards (RDNA2 or newer, at least 8 GB) only work on Linux and have had less testing. On Windows with AMD, everything runs on the CPU. Macs with Apple Silicon are only supported on a best-effort basis. They haven't been tested and will probably be slower.

You'll need a lot of disk space: about 30 GB for the runtime, or about 33 GB with the better translation model. Setup won't install with less than 34 GB free (40 GB with the large translation model). You'll also want a decent microphone, ideally a headset or desktop mic 15–20 cm away. To install, you clone the Git repo and run setup.sh (Linux/macOS) or setup.cmd (Windows). Then you launch the app with cargo run --release in the app folder. If setup gives you trouble, the README has a troubleshooting list covering things like interrupted Hugging Face downloads and certificate errors on Windows.

Consent and licensing

The developer puts the legal notes right at the top:

Only clone your own voice or a voice whose owner has given you explicit permission.

Recordings of other people from phone calls, videos or podcasts are off-limits. So are fraud, deception, fake voice messages and getting around voice authentication. If you publish generated speech, you should label it as synthetic. There's another catch: because of their training data, the OmniVoice weights and the NLLB-200 translation models are licensed under CC BY-NC 4.0, and the app code is under the PolyForm Noncommercial License. Commercial use is therefore ruled out unless you get a separate license from the author.

Our take

Mimvox doesn't invent anything new. What it does is bundle several strong open-source components (OmniVoice, RVC, Whisper, NLLB-200, Seed-VC and a music separator) into a usable interface. It adds thoughtful touches like the recording quality check, the audiobook mode and video dubbing. Cloud services like ElevenLabs offer similar features. The difference here is that your voice, which is biometric data, never leaves your machine. That's appealing if you want to listen to your own writing as an audiobook or translate your vacation videos.

There are several caveats, though. It's a one-person project at an early stage, and the similarity scores come from the developer. At over 30 GB, it takes up a lot of disk space, and without a capable NVIDIA card you'll struggle. Then there's the basic misuse problem. Technically, the app can clone any voice it has a recording of. The consent rules are a request, not a technical safeguard.