Nemotron 3.5 Bolt: NVIDIA's compact vision-language model arrives on Ollama
NVIDIA has released Nemotron 3.5 Bolt, a compressed vision-language model that's now in the Ollama library. It has 74 billion parameters, of which only 9 billion are active per token, and it's aimed at coding, reasoning and tool use.
What it's about
NVIDIA has released Nemotron 3.5 Bolt, a compact vision-language model that can run locally through the Ollama library. A vision-language model understands not just text but also images. According to the description, Bolt is meant for local coding, reasoning and tool use. Ollama lists the capabilities "vision", "tools" and "thinking". The page shows around 8,940 downloads and was last updated about a week ago. You start the model with ollama run nemotron-3.5-bolt.
How the model is built
Bolt has 74 billion parameters in total. Roughly speaking, those are the knobs the model tuned during training. But only 9 billion of them are active per token (a token is a chunk of text, about a syllable or word fragment). That comes from the mixture-of-experts (MoE) design: the model consists of many specialized sub-networks, the "experts", and only some of them are consulted for each token. That saves compute, but the memory requirement still tracks the total size.
The backbone combines two architectures: Mamba-2 (a so-called state-space model that handles long inputs efficiently) and the classic Transformer. For images, it uses a C-RADIO vision encoder, which translates image content into a form the language model can work with.
Derived from Nemotron 3.5 Super
According to NVIDIA, Bolt was developed from Nemotron 3.5 Super through model compression and further training. Super has 120 billion parameters (12 billion active), so Bolt is considerably smaller.
Quantization
The released GGUF format (a widely used file format for local models) uses quantization-aware distillation. Quantization means storing the numbers in a model with fewer bits so it needs less memory. In distillation, a smaller, more heavily compressed model is trained to imitate the behavior of a larger one. Here, that training explicitly accounts for quantization. Specifically:
- Expert weights: IQ2_XXS (very aggressive, roughly 2-bit quantization)
- Other layers: NVFP4 and Q8_0
The page gives no file size. According to Ollama, memory use depends on context size, image inputs and speculative decoding. Speculative decoding is a technique in which a small helper model makes suggestions that the large model verifies, which speeds up generation.
What you need to try it
NVIDIA provides instructions for running it with CUDA, meaning on NVIDIA GPUs. Apple Silicon / Metal is currently not supported, so Macs with M-series chips are out for now. The source gives no concrete VRAM (video memory) figures. As a rough estimate, with 74 billion parameters and very heavy quantization, a GPU with a lot of memory or a multi-GPU setup makes sense. For reliable numbers, check NVIDIA's documentation. System RAM and disk space aren't quantified either. Ollama offers launch commands for integrations such as ollama launch claude, opencode, hermes and openclaw with this model.
License
Bolt is released under the OpenMDW License Agreement, version 1.1. For training details and evaluation results, Ollama and the page point to NVIDIA's model card. The source cites no benchmarks.
Our take
Bolt is interesting because NVIDIA has deliberately shrunk a large model for local use: few active parameters, aggressive quantization, image understanding and tool support. For developers with NVIDIA hardware who want to run local coding agents, it's a candidate. What remains open is how much quality the 2-bit quantization costs and how Bolt stacks up against rival models. The source provides no numbers on either. The lack of Metal support narrows the audience. All performance claims come solely from NVIDIA or the Ollama description.
Sources
- Ollama-Bibliothek ollama.com
- MarkTechPost marktechpost.com
- Google News (DE) news.google.com
- Google News (BR) news.google.com
- AI Times (KR) aitimes.com
- Hugging Face Trending huggingface.co
Try it yourself: Nemotron 3.5 Bolt
Explained step by step, no prior knowledge needed – on your own computer or free in your browser.
What can I do with it?
You can use Bolt locally for programming tasks, logical reasoning and agentic workflows with tools. It also understands images, such as screenshots or diagrams.
Who is it worth it for?
Worth a look for developers with powerful NVIDIA hardware who want to test local coding agents. Mac users with Apple Silicon should skip it for now.
What your computer needs
Which variant is right for me?
Many models come in several sizes or "quantizations." These are compressed versions: smaller and faster, but slightly less accurate. Rule of thumb: pick the largest variant whose memory requirement fits in your VRAM (video memory). The scripts below choose this for you automatically.
| Variant | Download | Needs approx. | Compatible graphics cards |
|---|---|---|---|
e4b |
7.2 GB | 9 GB | e.g. RTX 3060 12 GB, RTX 4070, RTX 5070, RX 7700 XT |
e4b-q4_K_M |
7.9 GB | 10 GB | e.g. RTX 3060 12 GB, RTX 4070, RTX 5070, RX 7700 XT |
e4b-q8_0 |
9.8 GB | 12 GB | e.g. RTX 3060 12 GB, RTX 4070, RTX 5070, RX 7700 XT |
e4b-bf16 |
17 GB | 20 GB | e.g. RTX 3090, RTX 4090, Tesla P40 |
74b-a9b Default |
25 GB | 29 GB | e.g. RTX 5090 |
The Ollama page lists only one variant (with no size given). The GGUF uses IQ2_XXS for experts and NVFP4 and Q8_0 for other layers.
- NVIDIA provides CUDA instructions; an NVIDIA GPU is the intended platform.
- Apple Silicon / Metal is currently not supported.
- Memory use depends on context size, image inputs and speculative decoding.
- The source gives no concrete VRAM figures; with 74B parameters, expect to need a lot of memory.
How do I find out what my computer has?
- Windows: Press Ctrl + Shift + Esc → Task Manager → Performance tab → click GPU on the left. At the bottom you'll see Dedicated GPU memory (that's your VRAM). You'll find system memory under Memory.
- Mac: Apple menu at the top left → About This Mac. You'll see your chip (e.g. M2) and memory (e.g. 16 GB). On Apple chips, graphics and processor share this memory.
- Linux: Type
nvidia-smiin the terminal (shows your graphics card and its memory) andfree -hfor system memory.
Guide
Google Colab (in your browser, no installation)
Google Colab is a free service from Google. It gives you a computer with a graphics card in your browser for a few hours. All you need is a Google account. For this model: runs for free – The free Tesla T4 GPU with 15 GB is enough.
- Download the notebook. Click the button and save the file.
- Open Colab. Go to colab.research.google.com and sign in with your Google account.
- Upload the notebook. In the window that appears, click Upload on the left (or choose File → Upload notebook at the top) and select the file
colab.ipynb. - Turn on the GPU. In the top menu, go to Runtime → Change runtime type → under Hardware accelerator, select T4 GPU → Save.
- Start everything. Runtime → Run all (keyboard shortcut Ctrl + F9). If you see the warning “This notebook was not authored by Google,” click Run anyway.
- Wait. The cells run one after another. A green checkmark appears to the left of each cell when it's done. Installation and download take about 12 minutes here.
- Try it out. The result appears below the last cell. To test something else, change the text in the cell and run it again with Shift + Enter. The last cell starts a proper chat with an input field.
- Clean up. When you're done: Runtime → Disconnect and delete runtime. This saves your free quota.
Good to know: The free GPU is limited (usually a few hours per day). After about 90 minutes of inactivity, Colab disconnects, and you'll have to start everything over.
Windows 10 / 11
The easiest way is Ollama, a free program that launches AI models with a single command.
- Update your graphics driver (NVIDIA cards only): Install the NVIDIA app or the latest driver. For AMD cards, use the latest Adrenalin driver.
- Install Ollama. Download the file OllamaSetup.exe from ollama.com/download/windows, double-click it, and choose Install. Afterward, a small llama icon appears in the taskbar at the bottom right.
- Open a terminal window. Press the Windows key, type PowerShell and press Enter. A blue or black window opens.
- Start the model. Copy this command ("Copy" button), paste it into the window with a right-click, and press Enter:
The first time, Ollama downloads the model (25 GB (about 33 min at 100 Mbit/s)). After that, it's instant.ollama run nemotron-3.5-bolt:74b-a9b - Start typing. Once
>>> Send a messageappears, type your question and press Enter. Type/byeto end the chat.
Not enough memory? Pick a smaller variant from the table above, e.g. ollama run nemotron-3.5-bolt:e4b.
Prefer automatic? A script that picks the right variant for you
- Download both files into the same folder: try_windows.ps1 demo.py
- Open the folder in File Explorer, right-click an empty spot and choose Open in Terminal.
- Paste this command and press Enter:
powershell -ExecutionPolicy Bypass -File try_windows.ps1
If needed, the script installs Ollama via winget, measures your VRAM (video memory) and downloads the matching variant.
macOS
Macs with an Apple chip (M1 to M5) are ideal. Older Intel Macs are very slow.
- Install Ollama. Download it from ollama.com/download/mac, open the file, and drag Ollama into your Applications folder. Launch Ollama once – a llama appears in the menu bar at the top.
- Open Terminal. Press Cmd + Space, type Terminal, and press Enter.
- Start the model. Paste the command (Cmd + V) and press Enter:
The first time, the model is downloaded (25 GB (about 33 min at 100 Mbit/s)).ollama run nemotron-3.5-bolt:74b-a9b - Start typing. Type your question and press Enter. Quit with
/bye.
Linux
- Check your GPU.
nvidia-smishould show your card. If it doesn't, install the driver on Ubuntu and restart:sudo ubuntu-drivers install - Install Ollama:
curl -fsSL https://ollama.com/install.sh | sh - Start the model:
Exit withollama run nemotron-3.5-bolt:74b-a9b/bye. Useollama psto check whether the model is running on the GPU. - Or let it pick the right variant automatically:
bash try_linux.sh
Try it
Write a Python function that reads a CSV file and removes duplicate rows, and explain the code.Describe what you see in this screenshot of an error message and suggest a fix.Plan step by step how to deploy a REST API on Docker, and list the commands I need.If something goes wrong
Install the latest Ollama version, since the model is new.
It doesn't run on Macs with Apple Silicon right now, because Metal isn't supported.
With a large context or images, memory use rises sharply; reduce the context if needed.
“out of memory,” a crash, or your computer freezes
The model is too big for your VRAM (video memory). Pick a smaller variant from the table above and close other programs, especially games and browsers with lots of tabs.
It runs, but extremely slowly
Your CPU is probably doing the work instead of your graphics card. In Ollama, the PROCESSOR column of ollama ps shows how much runs on the GPU. If it says "100% CPU", update your graphics driver or pick a smaller variant that fits entirely into VRAM (video memory).
“ollama” or “python” is not found
Close the terminal window and open it again so it recognizes the newly installed program. For Python on Windows: reinstall it and check Add python.exe to PATH.
Windows: "Running scripts is disabled on this system"
Run the script with exactly the command above: powershell -ExecutionPolicy Bypass -File try_windows.ps1. This allows execution for this one time only.
Error message when loading the model (version, "trust_remote_code", unknown architecture)
Brand-new models often need the latest software version. With Ollama: update Ollama. With Python: pip install -U transformers in the activated environment. The Model Card usually lists the exact version.
Tokenstrom generates the scripts and notebook automatically. Please take a quick look before running them – they're plain, readable text. Mind the model's license: custom license (see Model Card).
Ask the AI assistant
Questions about this article, installation or background? Our assistant knows this page and runs on our own machine, not in someone else's cloud.
The Assistant is an AI (Gemma 4) and can make mistakes – please check important details against the source. Questions are used only to generate an answer and are deleted after one hour. Please don't enter any personal data.
Comments
What do you think? Already tried it? Write it here, no sign-up needed.
Comments are public. Links aren't allowed, and we delete abusive posts. You can delete your own comment at any time in this browser. Learn more in the Privacy Policy.