Nvidia's Nemotron-Labs-Sigma writes text by diffusion, not word by word

Nvidia has released Nemotron-Labs-Sigma, an open language model with 8 billion parameters (plus a smaller 3-billion-parameter version). Instead of writing text one token at a time, it pulls whole blocks out of noise in parallel, much like an image generator does. That could mean much faster text output, and it makes the model an interesting one for researchers.

Model releases Try it yourself Nov 16, 2026 6 min read Source: Reddit r/LocalLLaMA
Illustrative image, AI-generated

What's going on

Nvidia has released a new model family on Hugging Face called Nemotron-Labs-Sigma. It surfaced in the r/LocalLLaMA community on November 16, 2026. According to the post title, it comes in two sizes, with 8 billion and 3 billion parameters. So far only the 8B version is documented in any detail, on its Model Card at huggingface.co/nvidia/Nemotron-Labs-Sigma-8B.

The interesting part isn't the size. It's how the model works. Sigma is a continuous diffusion language model. Almost every well-known chatbot, from ChatGPT to Llama, is autoregressive: it writes one token (a piece of a word) at a time, and each new token depends on all the ones before it. Sigma generates entire blocks of text at once instead. It starts with random noise and refines it step by step until readable text emerges. Image generators like Stable Diffusion work the same way.

Nvidia is releasing two versions: a base model that's only pretrained, and an Instruct version for chat and following instructions: Nemotron-Labs-Sigma-8B-instruct. According to the Model Card, the model is "ready for commercial and non-commercial use."

How diffusion works for text

Diffusion makes intuitive sense for images. Pixels are numbers, so you can add noise to them gradually and then take it away again. Text is different. It's made of discrete symbols, words and tokens, with no smooth transitions between them. That's why many existing text diffusion models are discrete: they mask out words, a bit like a fill-in-the-blanks exercise, and fill the gaps in gradually.

Nvidia takes a different route with Sigma. Each token is represented as a point in a continuous embedding space, a kind of coordinate system where similar tokens sit close together. In Sigma this space has just 16 dimensions, which is very small. The model starts with Gaussian noise, meaning randomly distributed numbers, and nudges those points step by step toward where meaningful tokens are. According to Nvidia, the denoising network and the token embeddings are trained jointly via likelihood optimization: the model learns to assign the highest possible probability to its training texts.

Blocks, steps and the KV cache

Sigma combines both worlds. Text is generated in parallel blocks. Within a block, all tokens are refined at the same time over an adjustable number of denoising steps. You can choose between two well-known samplers from image diffusion:

  • DDPM (Denoising Diffusion Probabilistic Model): the classic stochastic sampler, which usually needs many steps.
  • DDIM (Denoising Diffusion Implicit Model): a deterministic variant that gets by with far fewer steps.

On top of that there's self-conditioning. The model gets its own estimate from the previous step as an extra input, which makes the refinement more stable.

Once a block is finished, it gets "committed." The tokens already generated are then reused through a perfectly ordinary KV cache. That's the buffer where language models store text they've already processed so they don't have to recompute it every time. So from block to block, Sigma behaves like a conventional model. Inside a block, it behaves like an image generator.

Why this could be faster

The appeal is in the trade-off between speed and quality. An autoregressive model needs 100 passes to produce 100 tokens. Sigma can produce a whole block in a handful of denoising steps. Nvidia says fewer steps cost "a little quality" but deliver "large speedups." The excerpt of the Model Card we have doesn't include concrete speed or benchmark numbers. The highlights only mention reasoning performance that's "competitive." That sentence is cut off in the source, so it's unclear what exactly the model is being compared to.

Nvidia also points out that the smooth, continuous representation makes the model well suited for:

  • Few-step distillation: you train a version that delivers almost the same results with very few steps, in the extreme case just one. In image models, this has sped up generation many times over.
  • Trajectory steering: you deliberately guide the denoising path, for example to push the text in a particular direction. It's similar to how image generators are steered with guidance signals.

Who the base model is for

Nvidia is explicit that the base model is not an assistant. It's meant as a starting point for research into parallel decoding, fine-tuning (further training on your own data) and distillation. If you want to chat or give instructions, use the Instruct version. The "Labs" in the name makes it clear this is a research project from Nvidia's Nemotron family, not a finished product.

What you need to try it

The source gives no official hardware requirements. The following are estimates:

  • GPU: For the 8B version at full precision (BF16), you'll need roughly 16 to 18 GB of video memory (VRAM), so think RTX 4090, 5090 or a workstation card. The 3B version should get by with about 6 to 8 GB, so it'll run on mid-range cards too.
  • Quantized versions: These are smaller versions that store numbers at lower precision. None are known to exist yet. Because the architecture is new, popular tools like Ollama, LM Studio and llama.cpp very likely don't support the model yet. You'll need Python with the Hugging Face transformers library and probably the model code that ships with it.
  • RAM and disk space: Plan for at least 32 GB of RAM and around 20 GB of free disk space for the 8B weights.
  • Mac, CPU, Colab: Running on Apple Silicon or CPU only is uncertain, since Nvidia research code is usually built around CUDA. The 3B version should be testable in Google Colab with a T4 or L4 GPU.

The bigger picture

Diffusion language models have been seen for a while as a possible way around the bottleneck of word-by-word generation. Until now, most well-known examples came from the discrete masking camp. Nvidia openly releasing a continuous approach at 8 billion parameters is notable for research. Combining parallel blocks with a KV cache looks pragmatic, because it keeps using proven inference techniques.

A lot is still open, though. Without published benchmarks and speed measurements, there's no way to say how Sigma stacks up against conventional models of the same size, such as Llama or Qwen variants. "Competitive" is the vendor's own claim. For end users, the model is more of an experiment for now than a replacement for proven local chat models. But if you're curious about where fast text generation is heading, this is one of the first open models of its kind at a meaningful size.

Sources

#nvidia#nemotron#diffusion#language model#open-weights
Read the original source

Try it yourself: Nemotron-Labs-Sigma-8B

Explained step by step, no prior knowledge needed – on your own computer or free in your browser.

What can I do with it?

Nemotron-Labs-Sigma doesn't generate text word by word. It works block by block through gradual denoising, much like an image generator. Running it locally, you can watch how quality and speed change as you adjust the number of denoising steps. The base model continues existing text and is meant for research, fine-tuning and distillation, not conversation. For chat and instructions there's the Instruct version. Nvidia calls its reasoning performance "competitive," but independent benchmarks are still missing. Don't expect a leap in quality over mature local chat models like Qwen or Llama. The appeal is the new technique and the potential speed advantage.

Who is it worth it for?

Worth it for developers, students and researchers who want to understand or build on new generation methods, and for tinkerers with a powerful Nvidia GPU and some Python experience. If you just want a good local chat assistant, you're better off with established models in Ollama or LM Studio.

Ideas: What you can do with it

Speed experiment

Run the same prompt with 4, 16 and 64 denoising steps and time each run. You'll see right away how much quality you give up for more speed.

Learning project: diffusion

Print the intermediate output after each denoising step, if the code allows it. You'll watch words gradually emerge from noise and understand diffusion better than any textbook could teach you.

Your own fine-tune

Fine-tune the base model on your own texts, such as product descriptions or articles from your field. Then check whether block-wise generation gets you usable results faster for formats like these.

Text continuation for drafts

Give the model the opening of an email, a blog post or a story and let it keep writing. What you get is a rough draft to edit.

Compare with a conventional model

Ask the Instruct version and a conventional model of the same size, like Llama or Qwen, the same questions. That gives you your own sense of where diffusion stands today.

Size8.5 B parameters
Download5.1 GB (about 7 min at 100 Mbit/s)
Licensecustom license (see Model Card)
TypeLanguage model (text chat)

What your computer needs

Graphics card (NVIDIA) at least 7 GB VRAMe.g. RTX 3060 Ti, RTX 4060, RTX 5060, RX 7600
Mac (Apple Silicon M1–M5) from 16 GB unified memoryThe GPU and CPU share the memory – the model can use about 70% of it
CPU only (no graphics card) Runs smoothly without a graphics cardan estimated 9 words per second on a current desktop PC. This requires at least 16 GB of RAM.
Storage approx. 7.1 GB freeModel download plus programs
Google Colab runs for freeThe free Tesla T4 GPU with 15 GB is enough

Values estimated from the model size – the Model Card may have more precise figures.

Which variant is right for me?

Many models come in several sizes or "quantizations." These are compressed versions: smaller and faster, but slightly less accurate. Rule of thumb: pick the largest variant whose memory requirement fits in your VRAM (video memory). The scripts below choose this for you automatically.

VariantDownloadNeeds approx.Compatible graphics cards
4-bit (Q4) Default 5.1 GB7 GBe.g. RTX 3060 Ti, RTX 4060, RTX 5060, RX 7600
8-bit (Q8) 9.1 GB12 GBe.g. RTX 3060 12 GB, RTX 4070, RTX 5070, RX 7700 XT
16-bit (Original) 17 GB20 GBe.g. RTX 3090, RTX 4090, Tesla P40

For chat and instructions, use the Instruct version (nvidia/Nemotron-Labs-Sigma-8B-instruct). For research and fine-tuning, use the base model. The 8B version needs roughly 16 to 18 GB of VRAM in BF16. The 3B version, which also exists according to the Reddit post title, is the better choice for cards with 8 to 12 GB or for Google Colab. No quantized versions (GGUF) are known.

  • An estimated 16–18 GB of VRAM for the 8B version in BF16. The source gives no official figures.
  • The 3B version should get by with about 6–8 GB of VRAM.
  • An Nvidia GPU with CUDA is strongly recommended. It's unclear whether the model runs on Apple Silicon or CPU only.
  • More denoising steps mean more compute time but make little difference to memory use.
  • Plan for about 20 GB of disk space for the weights.
How do I find out what my computer has?
  • Windows: Press Ctrl + Shift + Esc → Task Manager → Performance tab → click GPU on the left. At the bottom you'll see Dedicated GPU memory (that's your VRAM). You'll find system memory under Memory.
  • Mac: Apple menu at the top left → About This Mac. You'll see your chip (e.g. M2) and memory (e.g. 16 GB). On Apple chips, graphics and processor share this memory.
  • Linux: Type nvidia-smi in the terminal (shows your graphics card and its memory) and free -h for system memory.

Guide

Google Colab (in your browser, no installation)

Google Colab is a free service from Google. It gives you a computer with a graphics card in your browser for a few hours. All you need is a Google account. For this model: runs for free – The free Tesla T4 GPU with 15 GB is enough.

  1. Download the notebook. Click the button and save the file.

    Download colab.ipynb

  2. Open Colab. Go to colab.research.google.com and sign in with your Google account.
  3. Upload the notebook. In the window that appears, click Upload on the left (or choose File → Upload notebook at the top) and select the file colab.ipynb.
  4. Turn on the GPU. In the top menu, go to Runtime → Change runtime type → under Hardware accelerator, select T4 GPU → Save.
  5. Start everything. Runtime → Run all (keyboard shortcut Ctrl + F9). If you see the warning “This notebook was not authored by Google,” click Run anyway.
  6. Wait. The cells run one after another. A green checkmark appears to the left of each cell when it's done. Installation and download take about 6 minutes here.
  7. Try it out. The result appears below the last cell. To test something else, change the text in the cell and run it again with Shift + Enter.
  8. Clean up. When you're done: Runtime → Disconnect and delete runtime. This saves your free quota.

Good to know: The free GPU is limited (usually a few hours per day). After about 90 minutes of inactivity, Colab disconnects, and you'll have to start everything over.

Windows 10 / 11

This model runs on Python. Our script handles the setup automatically: it creates an isolated Python environment, installs the required packages (PyTorch, approx. 3 GB) and downloads the model (5.1 GB (about 7 min at 100 Mbit/s)).

  1. Update your graphics driver (NVIDIA): install the latest driver. Without an NVIDIA card, it runs on the CPU – much more slowly.
  2. Install Python. Download Python 3.12 from python.org and run the installer. Important: At the bottom of the first window, tick Add python.exe to PATH, then click Install Now.
  3. Create a folder and download the files. Create a folder, e.g. Documents\AI-Test, and save both files into it:

    try_windows.ps1 demo.py

  4. Open a terminal in the folder. Open the folder in File Explorer, right-click an empty spot → Open in Terminal.
  5. Run it. Paste this command and press Enter:
    powershell -ExecutionPolicy Bypass -File try_windows.ps1
    Installation and download are running now. The first time, this can take 10–30 minutes.
  6. Result. The window shows the model's answer to a test question.
  7. Try your own input: Open demo.py in an editor, change the sample text, and save. Then run this in the terminal:
    .\.venv-nemotron-labs-sigma-8b\Scripts\Activate.ps1
    python demo.py

macOS

Macs with an Apple chip (M1 to M5) are ideal. Older Intel Macs are very slow.

  1. Open Terminal. Press Cmd + Space, type Terminal, and press Enter.
  2. Install the developer tools (first time only; includes Python):
    xcode-select --install
    Click Install in the window. If you see "already installed," everything is already in place.
  3. Download the files – both end up in your Downloads folder:

    try_mac.sh demo.py

  4. Run it:
    cd ~/Downloads
    bash try_mac.sh
    The first time, installation and download take 10–30 minutes.
  5. Result. The window shows the model's answer to a test question.

Linux

  1. Check your GPU. nvidia-smi should show your card. If it doesn't, install the driver on Ubuntu and restart:
    sudo ubuntu-drivers install
  2. Install the Python basics (Ubuntu/Debian):
    sudo apt update && sudo apt install -y python3-venv python3-pip git
  3. Put the files in a folder and open a terminal there:

    try_linux.sh demo.py

  4. Run it:
    bash try_linux.sh
  5. Result. The window shows the model's answer to a test question.

Try it

The capital of France is
Write a short story about a robot who wants to become a gardener.
Explain in three sentences how a diffusion model works.
A train leaves at 9:15 a.m. and takes 2 hours and 50 minutes. When does it arrive? Work it out step by step.
Summarize the pros and cons of parallel text generation versus word-by-word generation in a table.

Tips for better results

  • Start with the DDIM sampler and a medium number of steps, then lower it gradually to see when quality starts to drop noticeably.
  • Give the base model the beginning of a text instead of asking it a question, for example the start of a sentence or a paragraph in the style you want.
  • Compare the same input with few and many steps and time it. That trade-off between speed and quality is the whole point of the model.
  • Use the Instruct version for chat tests and stick to the chat format described on its Model Card.

If something goes wrong

Because of the new architecture, the model probably isn't supported by Ollama, LM Studio or llama.cpp. Use Python with transformers and follow the instructions on the Model Card.

You'll probably need custom model code. Load it with trust_remote_code=True, and check what's in the repo first.

The base model doesn't respond like a chatbot. It continues text. For questions and instructions you need the Instruct version.

If you run out of VRAM (CUDA out of memory), switch to the 3B version or shorten the output length.

Generation depends on sampler settings (DDPM or DDIM, number of steps). The wrong values produce garbled output.

“out of memory,” a crash, or your computer freezes

The model is too big for your VRAM (video memory). Pick a smaller variant from the table above and close other programs, especially games and browsers with lots of tabs.

It runs, but extremely slowly

Your CPU is probably doing the work instead of your graphics card. In Ollama, the PROCESSOR column of ollama ps shows how much runs on the GPU. If it says "100% CPU", update your graphics driver or pick a smaller variant that fits entirely into VRAM (video memory).

“ollama” or “python” is not found

Close the terminal window and open it again so it recognizes the newly installed program. For Python on Windows: reinstall it and check Add python.exe to PATH.

Windows: "Running scripts is disabled on this system"

Run the script with exactly the command above: powershell -ExecutionPolicy Bypass -File try_windows.ps1. This allows execution for this one time only.

Error message when loading the model (version, "trust_remote_code", unknown architecture)

Brand-new models often need the latest software version. With Ollama: update Ollama. With Python: pip install -U transformers in the activated environment. The Model Card usually lists the exact version.

Tokenstrom generates the scripts and notebook automatically. Please take a quick look before running them – they're plain, readable text. Mind the model's license: custom license (see Model Card).

Ask the AI assistant

Questions about this article, installation or background? Our assistant knows this page and runs on our own machine, not in someone else's cloud.

Connecting …

The Assistant is an AI (Gemma 4) and can make mistakes – please check important details against the source. Questions are used only to generate an answer and are deleted after one hour. Please don't enter any personal data.

Comments

What do you think? Already tried it? Write it here, no sign-up needed.

  1. Loading comments …

Comments are public. Links aren't allowed, and we delete abusive posts. You can delete your own comment at any time in this browser. Learn more in the Privacy Policy.