Nvidia's open Nemotron 3.5 Transcribe handles transcription, translation, speakers

What this is about

On October 28, 2026, Nvidia released Nemotron 3.5 Transcribe, an open-weights speech model available on Hugging Face and ModelScope. It turns speech recordings into text, but it does more than plain transcription. It labels speakers (diarization), translates, and answers questions about what's in a recording. You control it with text prompts, much like a chatbot. Nvidia is also releasing the Granary v2 training dataset, the pipeline behind it, and guides for fine-tuning.

What the model can do

Given a meeting recording, the same model produces different results depending on your prompt. For example: "Transcribe the recording and label each speaker," "Translate the audio into English," or "What decisions were made?" According to Nvidia:

• Transcription in 40 languages
• Translation for 31 languages to or from English
• Speaker labeling in 13 languages, up to eight speakers
• Recordings up to one hour long
• Word-level timestamps

You can put technical terms like "HbA1c" or "metformin" into the prompt as a hint list. The model is supposed to use them only if they are actually audible. Feature coverage varies by language, and questions about audio content (audio Q&A) are trained on English only. Nvidia advises spot-checking names, speaker changes and answers against the recording. The model is built for recorded audio, not live streaming.

Architecture and speed

The model pairs a speech encoder with a hybrid Mamba-Transformer mixture-of-experts language model. In a mixture-of-experts (MoE) model, many specialized sub-networks exist, but only a few work on each token. Of 30 billion parameters (the values learned during training), only about 4 billion are active. Speaker cues from the Sortformer system are fed into the language model together with the acoustic features, so it marks speaker changes while generating the text.

With the upcoming NVFP4 quantization (a heavily compressed number format), it processed 100 hours of audio in under three minutes on an RTX PRO 6000 Blackwell Server Edition (30-second clips, high parallelism). Quantized versions and local inference with NeMo-Speech.cpp are due soon, for example for RTX Spark, DGX Spark or Jetson AGX Thor. You can already run the checkpoint with vLLM, and Baseten, DeepInfra and Simplismart offer hosted endpoints.

Benchmarks

WER (word error rate) is the share of words recognized incorrectly. Lower is better. On Artificial Analysis, the model has roughly one-sixth the active parameters (about 4B) of the model it ties with (24B). The Decoder reports 2.81% versus 2.77% for Voxtral Small. On David AI, it edges out Nvidia's own Nemotron 3 Diarization (18.38%). On VoiceArena, it drops only 8.32% of words (median about 31%). According to Nvidia, it ranks ahead of several widely used commercial speech APIs on the overall Artificial Analysis list.

Granary v2 and domain adaptation

Granary v2 was built with NeMo Curator. According to Nvidia, the published manifest covers 463,062 hours in 48 languages from 16 corpora (its predecessor had 25 languages). The Decoder, by contrast, reports about 844,000 hours. The figures don't match, and Nvidia's own number refers to the manifest. Nvidia also says the dataset is not the model's full training mix. Each recording comes with three transcript styles (e.g. "I paid, uh, twenty-five euros." – "I paid, uh, €25." – "I paid €25."), plus labels, translations and quality flags. You need to download the source audio separately under its own licenses.

In a medical case study with recordings of Indian doctors (Eka Care, 3,619 clips), a more precise prompt cut the error rate from 30.8% to 23.8%, and fine-tuning brought it to 11.8%. On separate, unseen speakers it fell from 22.44% to 12.17%, beating an open medical specialist model (13.79%) and a widely used general-purpose model (14.92%). Training used about 153 hours of synthetic speech, 29 hours of Common Voice and 63 hours of MultiMed, and took roughly 14 GPU hours on a GB300. The trade-offs: on general English (LibriSpeech), the error rate rose slightly from 1.54% to 1.69%, and the specialist model stayed more accurate on drug brand names.

What you need to try it

The source gives no concrete VRAM figures. According to Nvidia, the model runs on a single GPU, from DGX Spark to the data center. At 30B parameters (4-bit), roughly 18–20 GB of VRAM is realistic (our own estimate). Nvidia says the quantized versions are still to come. The easiest way in is the Hugging Face demo or a hosted endpoint. To run it locally you need vLLM and plenty of disk space. Whether it runs on a Mac or on CPU alone is unclear; NeMo-Speech.cpp has been announced.

Our take

An open model that handles transcription, speaker labeling and translation in one step simplifies pipelines that used to need several models. Nvidia's specialized diarization model from September still had to be combined with a transcription model. Compared with Mistral's Voxtral family, Nvidia is betting on recordings rather than real time. According to The Decoder, Mistral is targeting voice agents with Voxtral 3. The caveats: most benchmarks come from Nvidia, the dataset figures contradict each other, real-time support is missing, and the medical adaptation shows trade-offs. The sources don't state the license.