Sophon-Realtime v1.1: Open voice model clones any voice from 3 seconds of audio
The team behind Sophon-Realtime has updated its open voice models. Give version 1.1 a three- to ten-second audio clip and it takes on that voice and talks with it in real time. You don't need to train anything or restart anything, and the developer says a single GPU is enough.
What's new
On November 11, 2026, Hugging Face user BitKnight announced version 1.1 of Sophon-Realtime. The project comes from the adventists-ai organization and includes two open voice models with 32 billion parameters each: Sophon-Realtime-32B-Tag and Sophon-Realtime-32B-Flash. The weights are on Hugging Face, and the code and instructions are on GitHub at github.com/adventists-ai/sophon-realtime.
The headline feature is zero-shot voice cloning. The model picks up a voice without being trained on it first. A short audio snippet is all it needs. After that, it talks in that voice during the conversation, in real time.
What "full duplex" means
Sophon-Realtime is a full-duplex voice model. It can listen and talk at the same time, the way people do on a phone call. Classic voice assistants work more like a walkie-talkie. You speak, your sentence is transcribed to text, a language model writes a reply, and a speech synthesizer reads it out. That takes time, and while the assistant is talking it usually isn't listening to you.
Full-duplex models drop that rigid back-and-forth. That's why, according to the developer, the Sophon models also support barge-in: you can cut the model off mid-sentence and it responds to you instead of talking right over you. The models can laugh, too. The post says both still work with a cloned voice.
How voice cloning works
According to the post, the process is very simple:
- Put an audio clip of 3 to 10 seconds into the
model/voices/folder. - In the config, set
"voice": "<name>"to the name of that voice. - Keep talking. From then on, the model answers in the new voice.
You don't need fine-tuning (retraining the model) and you don't need to restart. The developer says a single GPU is enough. In a demo video, BitKnight first plays 11 seconds of his own voice, then 9 seconds of a colleague's voice, and the two are swapped while the model is running.
Install the package with:
pip install -U "sophon-realtime[trt]"
The [trt] extra points to TensorRT, Nvidia's software for speeding up AI models on its own GPUs. The post doesn't say whether the model runs without an Nvidia GPU.
Live mode: the model fetches information
The video also shows a live mode. If you ask Sophon about the weather or the news, the model hands the request off to your own backend, meaning a server or program you run yourself that supplies the data. Meanwhile, the model keeps talking instead of going quiet while it waits for the result. That matters for voice assistants, because even short pauses make a conversation feel unnatural.
What you need to try it
The post only says "one GPU" and doesn't name a card or a memory size. As a rough estimate, a 32-billion-parameter model with 4-bit quantization (shrinking the model by storing its numbers at lower precision) needs about 20 GB of VRAM. It's unclear, though, whether quantized versions of Sophon exist. At full precision the requirement would be much higher, and real-time operation needs extra memory on top. Realistically, you're looking at an Nvidia card with 24 GB or more (such as an RTX 3090/4090) or a data center GPU. A Mac, CPU-only setups and the free Google Colab tier probably won't keep up with real-time speech. Check the Model Cards on Hugging Face before you download. They should cover memory requirements, the license and the difference between Tag and Flash. The source itself doesn't explain that difference.
Cloning voices responsibly
The developer explicitly adds:
Please only clone your own voice or voices you have permission to use.
That's more than a polite formality. If a few seconds of audio are enough to imitate a voice in real time, scammers can exploit it, for example with fake calls from "relatives" or "your boss." In many countries, including Germany, imitating someone else's voice without consent can violate their personality rights. As far as is known, nothing in the software technically prevents misuse.
The bigger picture
For a long time, full-duplex voice AI was something only big players offered, like OpenAI with Advanced Voice Mode or Google with Gemini Live. Open models such as Kyutai's Moshi showed it could be done in the open too. Sophon-Realtime goes a step further with 32 billion parameters and real-time voice cloning. That's interesting if you build voice assistants, audio drama or game projects, or accessibility tools, because you can run everything locally and never send audio to a cloud service.
A lot is still unknown, though. The post gives no benchmarks, no latency figures, no comparison of cloning quality with other systems, no license and no list of supported languages. You'll have to test for yourself how well languages other than English work. So far, every claim comes from the developer, and there are no independent tests yet. At the same time, the project lowers the bar for convincing voice impersonation even further. It's technically impressive, but it makes the debate over safeguards like watermarking more urgent.
Sources
- Hugging Face Posts huggingface.co
Try it yourself: Sophon-Realtime-32B (Tag / Flash) v1.1
Explained step by step, no prior knowledge needed – on your own computer or free in your browser.
What can I do with it?
Sophon-Realtime is a voice model you talk to directly with your voice. It listens and speaks at the same time, so you can interrupt it and it reacts right away. Since version 1.1, it can also imitate any voice you have a 3- to 10-second clip of, with no training and no restart. According to the developer, laughter and interruptions also work with the cloned voice. In live mode, it passes questions about weather or news to your own program and keeps chatting in the meantime. There are no independent benchmarks yet, and no comparisons with services like ChatGPT's voice mode. The source doesn't say which languages work well, so test languages other than English yourself.
Who is it worth it for?
It's worth a try if you're a tinkerer or developer with a powerful Nvidia GPU and want to build a local voice assistant, game characters with their own voices, or audio drama prototypes. If all you have is a laptop, a Mac or a card with little VRAM, a 32-billion-parameter model in real time won't make you happy. You're better off with smaller voice models or cloud services.
Ideas: What you can do with it
Set up Sophon with a microphone and speakers, and connect live mode to a small script that pulls in the weather or your calendar. That gives you an assistant that runs entirely on your machine and never sends recordings to the cloud.
For a tabletop RPG or your own video game, record short clips from fellow players who agree to it and save them as voices. Characters can then answer in those voices in real time and react when someone cuts in.
Practice job interviews, sales calls or difficult phone conversations by giving the model a role. Because you can interrupt it at any time, it feels more like a real conversation than with assistants that only take turns.
Assign different voices to different characters and switch between them on the fly. That lets you test dialogue quickly before real voice actors go into the studio.
People at risk of losing their voice to illness can record a sample early on. A local assistant can later speak in their own voice. Check the quality carefully before relying on it.
What your computer needs
There are two variants with 32 billion parameters each: Sophon-Realtime-32B-Tag and Sophon-Realtime-32B-Flash. The source doesn't say how they differ. The name "Flash" suggests a faster variant, but that isn't confirmed. Read the Model Cards on Hugging Face before you download, and if low latency matters to you, start with Flash when in doubt.
- According to the developer, the model runs on a single GPU, voice cloning included. Which card isn't specified.
- The install package uses TensorRT ([trt]), so you'll most likely need an Nvidia GPU with a recent driver.
- Rough estimate: with 32 billion parameters, you need about 20 GB of VRAM even with 4-bit quantization. Realistically, plan for 24 GB or more, such as an RTX 3090/4090.
- Real-time speech needs extra memory and a lot of compute. CPU-only setups and Macs are very likely not up to it.
- Set aside several dozen gigabytes of disk space for the model files. The exact size is listed on Hugging Face.
How do I find out what my computer has?
- Windows: Press Ctrl + Shift + Esc → Task Manager → Performance tab → click GPU on the left. At the bottom you'll see Dedicated GPU memory (that's your VRAM). You'll find system memory under Memory.
- Mac: Apple menu at the top left → About This Mac. You'll see your chip (e.g. M2) and memory (e.g. 16 GB). On Apple chips, graphics and processor share this memory.
- Linux: Type
nvidia-smiin the terminal (shows your graphics card and its memory) andfree -hfor system memory.
Guide
This model runs with its developers' own program (adventists-ai/sophon-realtime on GitHub). The commands below come from the Model Card and the developers' instructions.
Windows 10 / 11
Windows 10 / 11: not supported. According to the source, installation requires Linux with CUDA 13 and an 80 GB professional GPU. Windows isn't supported. If you have a card like that, you can try WSL2 (Linux on Windows), but the developer hasn't tested it. To try the model, use the free online demo in ModelScope Studio (AdventistsAI/Sophon-Realtime).
macOS
macOS: not supported. Sophon-Realtime needs an NVIDIA GPU with CUDA 13 and 80 GB of memory, which no Mac has. To try it, use the online demo in ModelScope Studio (AdventistsAI/Sophon-Realtime). You can also rent a Linux server with an H100 GPU.
Linux
Linux: supported. Linux is the only supported system. You need an NVIDIA GPU with 80 GB, such as an H100, H800 or A100 80G. It won't run on regular gaming GPUs. Instead, you can rent a server with an H100 or use the online demo in ModelScope Studio.
- Check the requirements. You need Linux and an NVIDIA GPU with 80 GB of memory. You also need a recent NVIDIA driver with CUDA 13 (CUDA is NVIDIA's computing platform), Python 3.12, Git and about 50 GB of free disk space. The first command shows your GPU, its memory and the CUDA version. The other two tell you whether Git and Python are installed.
nvidia-smi git --version python3 --version - Download the code. This command downloads the code from GitHub and switches to the folder for the Tag version. Afterwards, the path in your terminal should end in sophon-realtime/tag.
git clone https://github.com/adventists-ai/sophon-realtime && cd sophon-realtime/tag - Install everything and download the model. The install script creates its own Python 3.12 environment (a venv, a separate folder just for this program). Into it, it installs vLLM, vLLM-Omni, the plugin and TensorRT, an NVIDIA accelerator. Then it downloads the model files, about 42 to 45 GB. Depending on your connection, that can take an hour or more.
bash install.sh - Start the server. This starts the voice server on the first GPU (GPU=0) and on port 8000. The first start takes 15 to 20 minutes because compute components are being prepared. Later starts take 5 to 8 minutes. Keep this terminal window open.
GPU=0 PORT=8000 bash start.sh - Check the acceleration. Look for the line "Loaded flow-estimator TensorRT engine" in the server output. If it's there, the fast version is running. If it's missing, the voice is about three times slower under load and stutters.
- Record a question. Record a short question in English, for example on your phone or with the free program Audacity. Save it as a WAV file named my_question.wav in the sophon-realtime/tag folder.
- Get your first answer. Open a second terminal and switch to the sophon-realtime/tag folder. This command sends your recording to the server, in English and with the ryan_en voice. You get the answer back as text and as speech. If Python complains about a missing module, first activate the venv that install.sh created.
python client_example.py --wav my_question.wav --lang en --voice ryan_en - Result. The server runs on port 8000. The example program sends your recorded question, my_question.wav, to it, and Sophon answers with text and a spoken voice that can also laugh or sigh depending on what it's saying.
What to do after your first result
- Try other voices: chelsie (default), vivian_zh, ryan_en, brand_zh or brand_en. Pick one with the --voice option.
- Clone a voice: save 3 to 10 seconds of clean speech from a single person as model/voices/alice.wav, then run client_example.py with --voice alice. Only use your own voice or one you have permission to use.
- For real conversations where you can also interrupt Sophon, your own programs connect via ws://HOST:8000/v1/realtime. Alternatively, they can send audio to POST /v1/chat/completions.
- Live mode (cd sophon-realtime/live && BUILD=tag PORT=8000 LIVE_PORT=8780 bash start_live.sh) lets programs built for OpenAI GPT-Live connect.
Try it
(spoken) Hi, can you briefly introduce yourself and tell me what you can do?(spoken) Tell me a joke. (Then interrupt halfway through:) Wait, tell me a different one!(spoken, after adding your own voice) Please read this sentence back to me in my voice: Today is a wonderful day for a walk.(spoken, in live mode with your own backend) What's the weather going to be like in Hamburg tomorrow?(spoken) Let's practice a job interview: you're the head of HR, and I'm applying for a graphic designer position.Tips for better results
- Record the voice clip somewhere quiet, without echo or background music, and speak naturally. Use the full 10 seconds. The developer's demo clips were 9 to 11 seconds long.
- Test whether your language works well or whether the model sounds noticeably better in English. The source doesn't list supported languages.
- Use a headset or echo cancellation so the model doesn't mistake its own voice coming out of the speakers for an interruption.
- Try both variants, Tag and Flash, and compare how fast they respond and how good they sound.
If something goes wrong
Install with pip install -U "sophon-realtime[trt]". The quotes matter. Without them, your shell (zsh, for example) may misinterpret the square brackets.
The TensorRT extra needs a matching Nvidia driver and CUDA setup. If you get import errors, check your driver and CUDA version first.
The voice clip has to be in the model/voices/ folder and be 3 to 10 seconds long. The "voice" value has to match the voice's name exactly.
Only clone your own voice or voices you have explicit permission to use. The developer asks for this, and in many countries, Germany included, doing otherwise can violate personality rights.
The post doesn't mention a license or terms of use. Read them on the Hugging Face page before you use the model commercially.
“out of memory,” a crash, or your computer freezes
The model is too big for your VRAM (video memory). Pick a smaller variant from the table above and close other programs, especially games and browsers with lots of tabs.
It runs, but extremely slowly
Your CPU is probably doing the work instead of your graphics card. In Ollama, the PROCESSOR column of ollama ps shows how much runs on the GPU. If it says "100% CPU", update your graphics driver or pick a smaller variant that fits entirely into VRAM (video memory).
“ollama” or “python” is not found
Close the terminal window and open it again so it recognizes the newly installed program. For Python on Windows: reinstall it and check Add python.exe to PATH.
Windows: "Running scripts is disabled on this system"
Run the script with exactly the command above: powershell -ExecutionPolicy Bypass -File try_windows.ps1. This allows execution for this one time only.
Error message when loading the model (version, "trust_remote_code", unknown architecture)
Brand-new models often need the latest software version. With Ollama: update Ollama. With Python: pip install -U transformers in the activated environment. The Model Card usually lists the exact version.
Tokenstrom generates the scripts and notebook automatically. Please take a quick look before running them – they're plain, readable text. Mind the model's license: apache-2.0.
Ask the AI assistant
Questions about this article, installation or background? Our assistant knows this page and runs on our own machine, not in someone else's cloud.
The Assistant is an AI (Gemma 4) and can make mistakes – please check important details against the source. Questions are used only to generate an answer and are deleted after one hour. Please don't enter any personal data.
Comments
What do you think? Already tried it? Write it here, no sign-up needed.
Comments are public. Links aren't allowed, and we delete abusive posts. You can delete your own comment at any time in this browser. Learn more in the Privacy Policy.