Tencent Hy-SimulMT: open Chinese–English live translator with 3-second delay

Tencent has released Hy-SimulMT, an open model that translates spoken Chinese and English live, while the person is still talking. Tencent says the delay is around three seconds and the quality matches experienced interpreters. The weights are available under Apache-2.0, but all benchmarks come from Tencent itself.

Model releases Try it yourself Oct 26, 2026 5 min read Source: r/LocalLLaMA

The key points in brief

Hy-SimulMT is a simultaneous translation model from Tencent Hunyuan for speech between Chinese and English, in both directions. According to the model card, it was released on October 26, 2026 (the weights appeared on October 22, according to OrcaRouter).
The model is a Mixture-of-Experts with about 30.4B parameters (A3B, roughly 3B active per token by OrcaRouter's calculation) under the Apache-2.0 license.
On the RealSI long-audio test, Tencent reports a StreamLAAL of about 3 seconds, the lowest in the comparison.
On short-audio benchmarks it scores 90.3 on FLEURS, 75.5 on CoVoST2, 81.2 on ACL6060 and 84.7 on RealSI-Sentence. On CoVoST2, Qwen3.8-LiveTranslate leads clearly with 82.1.
In a small test with 18 recordings and six interpreters (CATTI Level 1 and 2), quality is said to be on par, often with lower latency.
There are six checkpoints (standard and Interleave, each as BF16, FP8 and GGUF). llama.cpp support has not yet been merged into the official repo.

What this is about

Tencent Hunyuan has released Hy-SimulMT, a model for simultaneous translation of speech between Chinese and English, in both directions. "Simultaneous" means the model takes in audio continuously and translates while the person is still speaking. Think of a conference-room interpreter, not a translator who waits for the whole sentence. It outputs translated text. With a connected text-to-speech service, it can also produce spoken output, optionally in the speaker's cloned voice.

The model card on Hugging Face gives October 26, 2026 as the release date. According to the OrcaRouter blog, the six repositories showed up as early as October 22. That post also describes a very quiet release, with no press release and no paper. OrcaRouter speculates that the timing fits the APEC leaders' meeting in Shenzhen, where Tencent's Hy Translate app is named as the official AI translation tool. That is explicitly OrcaRouter's guess, not a statement from Tencent.

What's in the package

The family includes six checkpoints:

Variant	Output	Format
Hy-SimulMT-30B-A3B	translation only	BF16, FP8, GGUF
Hy-SimulMT-30B-A3B-Interleave	recognized source text plus translation	BF16, FP8, GGUF

According to OrcaRouter, this is a Mixture-of-Experts (MoE) model (many specialized sub-networks, only a few of which are active per token) with 48 layers and 128 experts, using eight routed experts plus one shared expert per token. In total it has about 30.4 billion parameters (the values learned during training). Tencent doesn't explain the "A3B" in the name. Working from the config gives roughly 3 billion active parameters, but that is OrcaRouter's estimate, not a vendor figure. Audio input runs at 16 kHz and is processed in windows of at most 30 seconds. The license is Apache-2.0; third-party components keep their own terms.

There are also two tools on GitHub: SimulInfer (an inference service with a web interface, file upload, microphone and WebSocket) and SimulTrain (training based on the released checkpoints). Speech output and voice cloning need a separately configured TTS (text-to-speech) service. So "speech-to-speech" describes a system built around the model, not a single model that outputs audio.

Numbers and benchmarks

The key metric is StreamLAAL, the average delay between the spoken source and the translated output in streaming mode (lower is better). It is not the time to the first spoken output. On the long-audio benchmark RealSI, Tencent says Hy-SimulMT reaches about 3 seconds, the lowest in the comparison. Its GEMBA score (an automated quality measure for translations) is above the open-source models compared and four of five commercial APIs. The card doesn't say which API comes out ahead.

On short audio clips (higher is better):

Model	FLEURS	CoVoST2	ACL6060	RealSI-Sentence
Hy-SimulMT	90.3	75.5	81.2	84.7
Qwen3.8-LiveTranslate	88.4	82.1	80.5	84.4
Gemini3.5-LiveTranslate	92.4	74.4	82.6	82.2
Seed-LiveInterpret2.0	85.8	67.7	74.8	80.0
GPT-Realtime-Translate	77.7	50.8	76.4	76.7
SeamlessStreaming	72.8	60.9	49.8	56.9

Tencent stresses that among open-source models, it leads on FLEURS, ACL6060 and RealSI-Sentence, and that its 84.7 on RealSI-Sentence is the highest of all systems. OrcaRouter points out that CoVoST2 is the weakest column (6.6 points behind Qwen) and that Gemini is 2.1 points ahead on FLEURS.

Comparison with human interpreters

Tencent had six interpreters translate 18 Chinese and English recordings (about 6 minutes each, published online from September 2026, covering ten fields from medicine to law): three with a CATTI Level 1 certificate and three with Level 2. The best interpreter at each level counted. Tencent's result: quality on par with experts, often with lower latency. Tencent itself notes that this applies only to these samples and interpreters, and says nothing about speech quality or voice similarity. OrcaRouter notes that comparing against the best interpreter is a high bar for quality, but doesn't reflect the average for latency.

Two Chinese university experts praise the system: the model starts speaking and corrects itself afterward, much like interpreters do when they self-correct. They say it is moving from a "comprehension aid" to "usable communication" in low-risk situations. For strict liability or important decisions, Tencent says human interpreters remain important.

What you need to try it

Official route: Linux, Python 3.12+, Git, an NVIDIA GPU and vLLM 0.31 or later with HYV3ForCausalLM support. The command bash deploy.sh --install-server --model tencent/Hy-SimulMT-30B-A3B-Interleave sets everything up and starts the model (port 8800) and the web interface (port 8080).
Memory: Tencent gives no minimum requirement. By OrcaRouter's calculation, BF16 weighs about 61 GB, so an 80 GB card is the realistic lower bound. The FP8 and GGUF variants (Q4/Q8) make smaller cards plausible. These are estimates.
llama.cpp/Ollama/LM Studio: Support has not yet been merged into official llama.cpp. Tencent recommends waiting for a release. There is a bridge (serve_llamacpp.py, port 8094) for an already running audio-capable llama.cpp server.
Remote microphone access needs HTTPS. Speech output needs its own TTS service. Alternatively, there is the Hy Translate app for Android and iOS.

Our take

An openly downloadable simultaneous translator with about three seconds of delay is unusual. Comparable systems have mostly been commercial APIs. For developers who want to run live captions or meeting translation locally without the cloud, this is interesting, but only for Chinese and English.

There are caveats. All numbers come from Tencent, with no independent verification and no technical report. The interpreter comparison is small. According to OrcaRouter, the model metadata lists 36 languages, but only Chinese–English is documented. Whether other languages work for speech is unconfirmed. There is no hosted API and no pricing. Real-world use also depends on the missing llama.cpp support and on hardware: if you don't have a large NVIDIA card, you're best off waiting for community support.

Terms explained simply

Simultaneous translation: Translating while the person is still speaking, instead of waiting for the end of the sentence. It saves waiting time but is harder, because the meaning often only becomes clear later.
StreamLAAL: A metric for the average delay between speech and translation in streaming. Lower is better; it is not the time to the first spoken output.
GEMBA: An automated method for rating translation quality, in which a language model judges the translation. Higher scores mean better quality.
Mixture-of-Experts (MoE): The model consists of many specialized sub-networks (experts), of which only a few work on each token. This saves compute compared with a dense model of the same total size.
Voice cloning: The translated speech output sounds like the original speaker's voice. Here, the system needs a separate TTS service for this.
GGUF: A file format for quantized (shrunken) models that tools like llama.cpp, Ollama and LM Studio can read.
FP8: A number format with only 8 bits per weight. The model needs about half as much memory as in BF16.
CATTI: China's national certification exam for translators and interpreters; Level 1 is higher than Level 2.

Sources

r/LocalLLaMA huggingface.co
Google News (US) orcarouter.ai
Google News (BR) orcarouter.ai
Google News (US) news.google.com
Google News (TW) news.google.com
Hugging Face Trending huggingface.co

#tencent#translation#speech#open-weights#moe#hunyuan

Read the original source

Try it yourself: Hy-SimulMT-30B-A3B

Explained step by step, no prior knowledge needed – on your own computer or free in your browser.

What can I do with it?

Hy-SimulMT is a simultaneous speech translator. It listens to Chinese or English and translates into the other language while the person is still speaking. It works like a conference-room interpreter who doesn't wait for the end of the sentence. Locally, you get translated text. With a separately set-up TTS service (text-to-speech), you also get spoken output, optionally in the speaker's cloned voice. According to Tencent, the delay (StreamLAAL) is around 3 seconds. That is the lowest value in the comparison on the long-audio test RealSI. On short clips, the model is on par with the best systems compared. On RealSI-Sentence it has the top score at 84.7, while Qwen leads on CoVoST2. All numbers come from Tencent, with no independent verification. The limits: only Chinese and English are documented. It's not a chat model but a tool for audio. For important decisions or where liability is involved, human interpreters remain the better choice.

Who is it worth it for?

Trying it is worthwhile for developers and tinkerers with a powerful NVIDIA card on Linux who want to test live captions or meeting translation locally without the cloud. Curious users who want to follow Chinese or English talks can try it too. It makes less sense if you need other languages, only expect a simple Ollama or LM Studio chat, or don't have a big GPU. In that case, the Hy Translate app for Android and iOS is the easier route.

Ideas: What you can do with it

Live-caption English talks

You feed in an English conference talk via microphone or as a file and choose en-zh. The model streams Chinese text with a lag of about 3 seconds. That lets you check whether live captions would work for a meeting.

Understand Chinese videos and interviews

You upload a Chinese recording, such as an interview or a technical talk, and choose zh-en. With the Interleave variant, you see the source text and the English translation side by side. That helps with comprehension, but it's no substitute for a reliable translation.

Practice business conversations with partners in China

You speak sentences for a business meeting into the microphone and watch how the translation comes across. Tencent sees the model more for low-risk situations. For contracts or anything legally important, you still need humans.

Dub a podcast or short video with voice

You set up a TTS service and have a recording translated and spoken in a cloned voice. The result is a spoken version in the other language that still sounds like the original speaker. It's a good tinkering project, but not a finished product.

Build your own live translation

As a developer, you use SimulInfer's WebSocket to send audio from your own apps and get text back. You can build subtitle overlays or meeting tools that run without the cloud. The source gives no ready-made examples for this.

Test your own domains and keep training

With SimulTrain, you can build on the released checkpoints. First, check how the model handles your field, such as medicine or law. This is more for advanced users with plenty of hardware.

Size30.4 B parameters

Download61 GB (about 81 min at 100 Mbit/s)

Licenseapache-2.0

TypeTranslator

Hugging Facetencent/Hy-SimulMT-30B-A3B

What your computer needs

Graphics card (NVIDIA) at least 74 GB VRAMCalculated, not measured: the weights are 60.8 GB (BF16), so 60.8 × 1.2 + 1 ≈ 74 GB. The vendor gives no figure. The FP8 variant needs about half of that.

Mac (Apple Silicon M1–M5) from 128 GB unified memoryThe GPU and CPU share the memory – the model can use about 70% of it

Storage approx. 63 GB freeModel download plus programs

There are two models, each in BF16, FP8 and GGUF. "Hy-SimulMT-30B-A3B" outputs only the translation. "…-Interleave" also shows the recognized source text. For testing and checking quality, Interleave is handy because you can compare the original and the translation. The official example command also uses Interleave. By OrcaRouter's estimate, BF16 needs about 61 GB for the weights alone, so an 80 GB card is the realistic lower bound. With smaller cards, try FP8, although Tencent gives no minimum size for it. The GGUF variants (Q4, Q8) are meant for smaller hardware. But they only run once llama.cpp supports the model, which officially isn't the case yet.

Official route: Linux, Python 3.12+, Git, an NVIDIA GPU with a CUDA driver, and vLLM 0.31 or later with HYV3ForCausalLM support.
Tencent gives no minimum requirement for VRAM (video memory). By OrcaRouter's estimate, BF16 weighs about 61 GB, and FP8 and GGUF make smaller cards plausible. These are estimates, not measurements.
The model has about 30.4 billion parameters. Only a few experts are active per token (Mixture-of-Experts). Memory needs still depend on the total size, not on the roughly 3 billion active parameters.
Mac, CPU-only use, Ollama and LM Studio: llama.cpp support is not yet in the official repository. Tencent recommends waiting for a release.
Audio input runs at 16 kHz and is processed in windows of at most 30 seconds.
For the `serve_llamacpp.py` bridge (port 8094), you need an already running, audio-capable llama.cpp server.

How do I find out what my computer has?

Windows: Press Ctrl + Shift + Esc → Task Manager → Performance tab → click GPU on the left. At the bottom you'll see Dedicated GPU memory (that's your VRAM). You'll find system memory under Memory.
Mac: Apple menu at the top left → About This Mac. You'll see your chip (e.g. M2) and memory (e.g. 16 GB). On Apple chips, graphics and processor share this memory.
Linux: Type nvidia-smi in the terminal (shows your graphics card and its memory) and free -h for system memory.

Guide

This model runs with its developers' own program (Tencent-Hunyuan/Hy-SimulMT on GitHub). The commands below come from the Model Card and the developers' instructions.

Windows 10 / 11

Windows 10 / 11: not officially tested. The source doesn't mention Windows. The model needs a lot of video memory (about 74 GB), which normal PCs don't have. Use the Hy Translate app (Android/iOS) or the smaller FP8 or GGUF variants instead.

Check the prerequisites. You need Python 3 (ideally a recent version), Git and an NVIDIA graphics card with a current driver. You also need about 65 GB of free disk space. The source doesn't give exact versions.
```
python --version
git --version
nvidia-smi
```
Download the program. Get the program (SimulInfer) from GitHub into a new, empty folder.
```
git clone https://github.com/Tencent-Hunyuan/Hy-SimulMT
cd Hy-SimulMT
```
Create an environment. A virtual environment keeps the packages separate from the rest of your PC. Activate it afterward.
```
python -m venv .venv
.venv\Scripts\Activate.ps1
```
Install packages. First install the download tool. The other packages for the service are listed in the README of the GitHub project. Install them exactly as described there.
```
pip install -U huggingface_hub
```
Download the model. The download is 60.8 GB and takes a long time. It ends up in the folder model.
```
hf download tencent/Hy-SimulMT-30B-A3B --local-dir model
```
Start the service and test it. Start the SimulInfer service with the launch command from the GitHub project's README and point it to the folder model. This command wasn't in our source. For spoken output and voice cloning, you also need a configured TTS service (text-to-speech).
Result. The model is stored locally in the folder model. The translation service runs once you start it following the GitHub README, and translates speech between Chinese and English.

macOS

macOS: not officially tested. The source doesn't mention macOS, and no Apple support is mentioned. At about 74 GB of memory needed, the model is too big for most Macs. Try the Hy Translate app (iOS) or the GGUF variant instead.

Check the prerequisites. You need Python 3, Git (included with the Xcode tools) and about 65 GB of free disk space. The source gives no exact versions and no Mac support.
```
python3 --version
git --version
xcode-select --install
```
Download the program. Get the program (SimulInfer) from GitHub into a new, empty folder.
```
git clone https://github.com/Tencent-Hunyuan/Hy-SimulMT
cd Hy-SimulMT
```
Create an environment. A virtual environment keeps the packages separate from the rest of your Mac. Activate it afterward.
```
python3 -m venv .venv
source .venv/bin/activate
```
Install packages. First install the download tool. The other packages for the service are listed in the README of the GitHub project. Install them exactly as described there.
```
pip install -U huggingface_hub
```
Download the model. The download is 60.8 GB and takes a long time. It ends up in the folder model.
```
hf download tencent/Hy-SimulMT-30B-A3B --local-dir model
```
Start the service and test it. Start the SimulInfer service with the launch command from the GitHub project's README and point it to the folder model. This command wasn't in our source. For spoken output and voice cloning, you also need a configured TTS service.
Result. The model is stored locally in the folder model. The translation service runs once you start it following the GitHub README, and translates speech between Chinese and English.

Linux

Linux: not officially tested. The source doesn't explicitly name an operating system. Linux with an NVIDIA graphics card is the usual route for models like this. But you need about 74 GB of video memory, for example an 80 GB card. You can go smaller with the FP8 or GGUF variants.

Check the prerequisites. You need Python 3 (ideally a recent version), Git, a current NVIDIA driver and about 65 GB of free disk space. The source gives no exact versions.
```
python3 --version
git --version
nvidia-smi
```
Download the program. Get the program (SimulInfer) from GitHub into a new, empty folder.
```
git clone https://github.com/Tencent-Hunyuan/Hy-SimulMT
cd Hy-SimulMT
```
Create an environment. A virtual environment keeps the packages separate from the rest of the system. Activate it afterward.
```
python3 -m venv .venv
source .venv/bin/activate
```
Install packages. First install the download tool. The other packages for the service are listed in the README of the GitHub project. Install them exactly as described there.
```
pip install -U huggingface_hub
```
Download the model. The download is 60.8 GB and takes a long time. It ends up in the folder model.
```
hf download tencent/Hy-SimulMT-30B-A3B --local-dir model
```
Start the service and test it. Start the SimulInfer service with the launch command from the GitHub project's README and point it to the folder model. This command wasn't in our source. For spoken output and voice cloning, you also need a configured TTS service.
Result. The model is stored locally in the folder model. The translation service runs once you start it following the GitHub README, and translates speech between Chinese and English.

Try it

Easy: Slowly speak two or three English sentences into the microphone, such as "Good morning, my name is Anna and I work in a hospital." Choose the direction en-zh and see how quickly the text appears.

Easy: Upload a short English audio file (e.g. audio.wav, under 30 seconds), choose en-zh and read the translation.

Medium: Give a short talk in Chinese, such as an introduction to your company. Choose zh-en and watch whether the model starts early and corrects itself later.

Medium: Use the Interleave variant to test an interview recording with two speakers. Compare the source text and the translation side by side.

Advanced: Upload a multi-minute English conference recording on a specialist topic such as medicine, law or AI. Choose en-zh and check whether technical terms, numbers and names are right.

Advanced: Set up a TTS service and have a short talk translated in a cloned voice. Check how well the voice matches the original.

Tips for better results

Start with short, clear recordings under 30 seconds. Audio input works in windows of at most 30 seconds at 16 kHz. A good microphone and little background noise help.
Use the Interleave variant for checking. With the source text and translation side by side, you can see right away if the model misheard you.
Expect about 3 seconds of delay. That's the average delay, not the time to the first spoken output. So the first output may come earlier or later.
The model starts speaking early and corrects itself later, just like interpreters do. So judge a translation only once the sentence is finished.
Speak at a calm pace with clear pronunciation. Always double-check names, numbers and technical terms yourself.

If something goes wrong

vLLM must be version 0.31 or later with HYV3ForCausalLM support. With older versions, the model won't start.

The command `bash deploy.sh --install-server --model tencent/Hy-SimulMT-30B-A3B-Interleave` installs dependencies. If you run it again, everything gets reinstalled. If you already have your own vLLM/CUDA environment, use the --server-python option instead.

By default, the model runs on port 8800 and the web interface on port 8080. If a port is taken, startup fails. Then pick a different one with --port or --model-port.

Microphone access from another machine on the network only works over HTTPS. Test the microphone first directly on the machine running the service.

No speech output or cloned voice coming out? Then you probably lack the TTS service. It has to be set up and configured separately.

The model can't simply be loaded in Ollama or LM Studio. llama.cpp support is still missing from the official repository, so the GGUF files don't help there for now.

Languages other than Chinese and English are not documented. The model metadata does mark 36 languages, but whether they work for speech is unconfirmed.

“out of memory,” a crash, or your computer freezes

The model is too big for your VRAM (video memory). Pick a smaller variant from the table above and close other programs, especially games and browsers with lots of tabs.

It runs, but extremely slowly

Your CPU is probably doing the work instead of your graphics card. In Ollama, the PROCESSOR column of ollama ps shows how much runs on the GPU. If it says "100% CPU", update your graphics driver or pick a smaller variant that fits entirely into VRAM (video memory).

“ollama” or “python” is not found

Close the terminal window and open it again so it recognizes the newly installed program. For Python on Windows: reinstall it and check Add python.exe to PATH.

Windows: "Running scripts is disabled on this system"

Run the script with exactly the command above: powershell -ExecutionPolicy Bypass -File try_windows.ps1. This allows execution for this one time only.

Error message when loading the model (version, "trust_remote_code", unknown architecture)

Brand-new models often need the latest software version. With Ollama: update Ollama. With Python: pip install -U transformers in the activated environment. The Model Card usually lists the exact version.

Tokenstrom generates the scripts and notebook automatically. Please take a quick look before running them – they're plain, readable text. Mind the model's license: apache-2.0.

Ask the AI assistant

Questions about this article, installation or background? Our assistant knows this page and runs on our own machine, not in someone else's cloud.

Connecting …

The Assistant is an AI (Gemma 4) and can make mistakes – please check important details against the source. Questions are used only to generate an answer and are deleted after one hour. Please don't enter any personal data.

Comments

What do you think? Already tried it? Write it here, no sign-up needed.

Loading comments …

Comments are public. Links aren't allowed, and we delete abusive posts. You can delete your own comment at any time in this browser. Learn more in the Privacy Policy.