Clover-35B-A3B: Diffusion language model beats Qwen3.6 at agent tasks, Apache 2.0
DiffuSpace, a team linked to the Dream models, has released Clover-35B-A3B-Preview as an open diffusion language model. It reportedly beats the equally sized Qwen3.6-35B-A3B on agent benchmarks and generates much faster. The weights are available under the Apache license.
What this is about
The DiffuSpace team released its first model on October 28, 2026: Clover-35B-A3B-Preview. It is a diffusion language model. Common language models generate text one word at a time (more precisely, one token at a time). A diffusion model produces several tokens at once by gradually cleaning up an initially "noisy" block of text. That is supposed to be much faster. DiffuSpace comes from the circle around the NLP lab at the University of Hong Kong, which introduced Dream 7B in April 2025. According to the team, the Dream models have over 2.7 million downloads on Hugging Face. Clover is meant to show that diffusion models can keep up at larger scale, too. The weights are freely available under the Apache license.
Key model facts
Clover is a mixture-of-experts (MoE) model. It has 35 billion parameters (the adjustable values learned during training), but only about 3 billion are active per token, because only some of the "experts" are called on each time. It supports a thinking mode and a context window of 128K tokens. Weights and code are available on Hugging Face (Diffuspace/Clover-35B-A3B-Preview), GitHub and ModelScope. According to the maker, inference runs with SGLang and llama.cpp. You can also request online access at chat.diffuspace.ai. The team explicitly thanks Qwen for the architecture and training ecosystem as a starting point, which suggests Clover builds on Qwen foundations.
Architecture
The novelty lies in a rework of Gated DeltaNet (GDN), an alternative to classic attention with a compressed "memory." The bidirectional GDN treats the past and the current block differently: context that has already been generated is condensed into a recurrent state, while the tokens in the block currently being worked on exchange information through bidirectional softmax attention. The memory readout takes part in that same softmax as its own unit of information, so the model learns when to rely on history and when on the block. Added to this are a learnable block-size calibration and an extra convolution branch for the right-hand context, initialized to zero.
Benchmarks (maker's figures)
According to DiffuSpace, Clover scored highest among the compared models on six agent benchmarks. The comparison included the diffusion models LLaDA2.2 flash and Mercury 2.5, as well as Qwen3.6-35B-A3B and Qwen3.5-35B-A3B.
| Benchmark | Clover |
|---|---|
| SWE-bench Verified | 73.6 |
| Terminal-Bench 2.1 | 51.7 |
| τ³ Overall | 65.3 |
| BFCL-v4 | 67.9 |
| τ³ Banking | 14.4 |
| GPQA Diamond | 83.8 |
| MMLU-Pro | 84.8 |
| ZebraLogic | 95.8 |
| AA-LCR | 72.0 |
Against lightweight models (MiniCPM-V 4.7, LFM2.5, Nemotron 3.5 Lightning, Gemma 4 26B-A4B), Clover wins six of nine tests. Compared with MiniCPM-V 4.7, it leads by 15.7 points on Terminal-Bench 2.1 and by 28.0 points on SWE-bench Verified.
Speed
On an NVIDIA GB300, Clover reaches 680 tokens/s in BF16 at batch size 1 on internal agent data, according to the maker. On an H200 it manages 515 tokens/s on agent tasks, compared with 365 tokens/s for a Qwen model using the DFlash2 acceleration method. On an RTX 5070 Ti with 16 GB of VRAM, an INT4 version ran at 101.8 tokens/s on MATH-500, while comparison models such as MiniCPM and Qwen reached 48.5 tokens/s. These are the team's own measurements, partly in internal scenarios.
Demos and decision model
The demos show browser agents. In a route-finding task, Clover needed 28 steps and 169.9 s, while Qwen3.6 Flash failed after 30 steps. In a WebMCP test with 49 tasks, the median execution time was 1.52 s versus 8.86 s for Gemini 3.6 Flash. It also generates interactive web pages, such as a simulated desktop in a single HTML file or 3D learning pages.
A decision model has also been announced. It answers several related questions in a single pass (up to 8). In an internal test with 500 groups and 2,420 questions, it got 95.0% correct answers in joint mode and 89.8% when answering individually. The model will be released later.
What you need to try it
According to the maker, an INT4 version runs on a graphics card with 16 GB of VRAM (tested: RTX 5070 Ti). BF16 needs much more memory and compute, and the measurements were made on data-center GPUs. Expect roughly 20 GB or more of storage for a quantized version. You need SGLang or a suitable llama.cpp version. The source says nothing about Mac, CPU or Colab.
Takeaway
Clover is one of the first larger open diffusion language models with strong agent scores. It's interesting if you want fast agent loops or local inference. The caveats: the numbers come from the maker, the comparison partly uses its own scenarios, and the model carries the "Preview" label. Long-term stability is still to be improved. Whether the results can be independently reproduced, and how broad tool support will become, remains to be seen.
Sources
- r/LocalLLaMA diffuspace.ai
- InfoQ China (CN) infoq.cn
Try it yourself: Clover-35B-A3B-Preview
Explained step by step, no prior knowledge needed – on your own computer or free in your browser.
What can I do with it?
Clover is a language model that doesn't write text token by token (a token is a building block of text, such as a word or part of a word). It fills in a whole block at once and cleans it up step by step. This is called diffusion. According to the maker, that makes output faster. Locally, you can use it to answer questions, write text and code, and turn data into structured form, such as JSON. It can also build interactive web pages from a description, for example a simulated desktop in an HTML file. It has a thinking mode in which it reasons first and then answers. It also comes with a 128K-token context, which is the amount of text that your question and the answer may take up together. According to the maker, it is especially strong at agent tasks, where the AI operates tools such as a browser or a terminal. However, the figures come from the maker and partly from its own tests. The model carries the "Preview" label. Whether it holds up against familiar alternatives in everyday use remains to be seen.
Who is it worth it for?
It's worth a try for tinkerers, developers and the curious with a powerful graphics card who want to experience a new operating principle. You should be ready to use a terminal and set up software like SGLang or llama.cpp. It's probably not for beginners who just want to chat, or for anyone without a suitable graphics card. The source says nothing about Mac, CPU-only operation or Colab.
Ideas: What you can do with it
You describe in one or two sentences what the page should do, such as a quiz or a timer. Clover delivers a single HTML file that you open in your browser. Then you ask it for changes, for example different colors or a new feature.
You have a topic explained as an interactive page, such as solar eclipses in 3D. According to the maker, Clover can generate pages like this. You quickly check the facts and use the page in class or at home.
You enter a news item or a long email and ask for it to be broken down into individual tasks as JSON. That gives you data you can reuse in spreadsheets or scripts. It's best to specify the desired format with an example.
You have it write small scripts, explain them or hunt for bugs. According to the maker, Clover scores strongly in coding and terminal benchmarks. Test any code yourself before you use it.
If you have experience, you can build Clover into an agent loop in which it operates tools or a browser. The maker shows demos such as route finding. Speed matters most here, because many steps run in sequence.
With 128K context, very long texts fit into one request, such as a report or a transcript. You ask for summaries or answers to questions about it. Expect higher memory needs.
What your computer needs
Values estimated from the model size – the Model Card may have more precise figures.
Which variant is right for me?
Many models come in several sizes or "quantizations." These are compressed versions: smaller and faster, but slightly less accurate. Rule of thumb: pick the largest variant whose memory requirement fits in your VRAM (video memory). The scripts below choose this for you automatically.
| Variant | Download | Needs approx. | Compatible graphics cards |
|---|---|---|---|
4-bit (Q4) Default |
21 GB | 24 GB | e.g. RTX 3090, RTX 4090, Tesla P40 |
8-bit (Q8) |
37 GB | 42 GB | e.g. 2× RTX 3090/4090 or RTX A6000 |
16-bit (Original) |
69 GB | 78 GB | e.g. multiple professional GPUs (2× A6000, H100) or DGX Spark |
For a graphics card with 16 GB of VRAM (video memory), the maker offers an INT4 version. INT4 means the weights are heavily compressed, which saves memory and can cost some quality. It was tested on an RTX 5070 Ti. Expect roughly 20 GB or more of disk space. According to the Model Card, the full BF16 version needs about 69 GB of memory for the weights alone, plus room for running it. It only makes sense with a data-center GPU or a lot of video memory. For running and testing, the Model Card recommends the official SGLang implementation.
- According to the maker, INT4 runs on an RTX 5070 Ti with 16 GB of VRAM. There, 101.8 tokens/s was measured on MATH-500.
- According to the Model Card, BF16 needs about 69 GB for the weights, plus memory for running the model.
- The top figures (680 tokens/s on a GB300, 515 tokens/s on an H200) come from data-center GPUs. You won't reach those at home.
- A long context costs extra memory. Up to 131,072 tokens apply to your question and the answer combined.
- For the Python route, the Model Card lists an NVIDIA GPU with CUDA and PyTorch 2.10 or newer.
- The source says nothing about Mac, CPU-only operation and Colab.
How do I find out what my computer has?
- Windows: Press Ctrl + Shift + Esc → Task Manager → Performance tab → click GPU on the left. At the bottom you'll see Dedicated GPU memory (that's your VRAM). You'll find system memory under Memory.
- Mac: Apple menu at the top left → About This Mac. You'll see your chip (e.g. M2) and memory (e.g. 16 GB). On Apple chips, graphics and processor share this memory.
- Linux: Type
nvidia-smiin the terminal (shows your graphics card and its memory) andfree -hfor system memory.
Guide
Google Colab (in your browser, no installation)
Google Colab is a free service from Google. It gives you a computer with a graphics card in your browser for a few hours. All you need is a Google account. For this model: Colab Pro only – needs an A100 with 40 GB.
- Download the notebook. Click the button and save the file.
- Open Colab. Go to colab.research.google.com and sign in with your Google account.
- Upload the notebook. In the window that appears, click Upload on the left (or choose File → Upload notebook at the top) and select the file
colab.ipynb. - Turn on the GPU. In the top menu, go to Runtime → Change runtime type → under Hardware accelerator, select T4 GPU (with Colab Pro: L4 or A100) → Save.
- Start everything. Runtime → Run all (keyboard shortcut Ctrl + F9). If you see the warning “This notebook was not authored by Google,” click Run anyway.
- Wait. The cells run one after another. A green checkmark appears to the left of each cell when it's done. Installation and download take about 11 minutes here.
- Try it out. The result appears below the last cell. To test something else, change the text in the cell and run it again with Shift + Enter.
- Clean up. When you're done: Runtime → Disconnect and delete runtime. This saves your free quota.
Good to know: The free GPU is limited (usually a few hours per day). After about 90 minutes of inactivity, Colab disconnects, and you'll have to start everything over.
Windows 10 / 11
This model runs on Python. Our script handles the setup automatically: it creates an isolated Python environment, installs the required packages (PyTorch, approx. 3 GB) and downloads the model (21 GB (about 28 min at 100 Mbit/s)).
- Update your graphics driver (NVIDIA): install the latest driver. Without an NVIDIA card, it runs on the CPU – much more slowly.
- Install Python. Download Python 3.12 from python.org and run the installer. Important: At the bottom of the first window, tick Add python.exe to PATH, then click Install Now.
- Create a folder and download the files. Create a folder, e.g.
Documents\AI-Test, and save both files into it: - Open a terminal in the folder. Open the folder in File Explorer, right-click an empty spot → Open in Terminal.
- Run it. Paste this command and press Enter:
Installation and download are running now. The first time, this can take 10–30 minutes.powershell -ExecutionPolicy Bypass -File try_windows.ps1 - Result. The window shows the model's answer to a test question.
- Try your own input: Open
demo.pyin an editor, change the sample text, and save. Then run this in the terminal:.\.venv-clover-35b-a3b-preview\Scripts\Activate.ps1 python demo.py
macOS
Macs with an Apple chip (M1 to M5) are ideal. Older Intel Macs are very slow.
- Open Terminal. Press Cmd + Space, type Terminal, and press Enter.
- Install the developer tools (first time only; includes Python):
Click Install in the window. If you see "already installed," everything is already in place.xcode-select --install - Download the files – both end up in your Downloads folder:
- Run it:
The first time, installation and download take 10–30 minutes.cd ~/Downloads bash try_mac.sh - Result. The window shows the model's answer to a test question.
Linux
- Check your GPU.
nvidia-smishould show your card. If it doesn't, install the driver on Ubuntu and restart:sudo ubuntu-drivers install - Install the Python basics (Ubuntu/Debian):
sudo apt update && sudo apt install -y python3-venv python3-pip git - Put the files in a folder and open a terminal there:
- Run it:
bash try_linux.sh - Result. The window shows the model's answer to a test question.
Try it
What is diffusion? Explain it in five sentences for kids.Write me a packing list for a weekend of camping, sorted by category.Summarize this text in three bullet points and name the key takeaway: [paste text]Create a simulated desktop in a single HTML file with draggable windows, a calculator, a notepad and a clock.Build an interactive 3D page that explains why there isn't a solar eclipse every month.Tips for better results
- Keep the Model Card defaults, such as block_size=32 and threshold=0.95. They are already set sensibly.
- Use thinking mode (enable_thinking=True) for tricky tasks like logic or code. For simple questions you can leave it off, which is faster.
- Give clear requirements for programming tasks: language, file format and what the result should do. For HTML pages, add that everything should be in a single file.
- Set max_new_tokens as a hard cap on length. The Model Card uses 24576 in its example.
- Break big tasks into small steps. That helps with a preview whose long-term stability the maker says is still to be improved.
If something goes wrong
Regular diffusion models don't run in every piece of software. Use SGLang or a suitable, up-to-date llama.cpp version. The Model Card recommends SGLang.
On the Hugging Face route, the Model Card requires exactly Transformers 5.19.0 and PyTorch 2.10 or newer. You also have to set trust_remote_code=True so the model code gets loaded. Review the code first, because this runs third-party code.
The Python route via Hugging Face downloads the BF16 weights (about 69 GB). That won't work on a 16 GB card. Use the INT4 version via the official guide instead.
Very long answers can start to repeat themselves. Countermeasures are already built into the defaults. Change them only if you really need to.
Requests that exceed the 128K context limit are rejected. This includes the space for the last block of the answer.
The speed figures are the maker's own and partly from internal tests. It may be slower on your hardware. The text can also differ slightly depending on the software, even with identical settings.
The model is a preview. On long tasks, results and stability can vary. The decision model has not been released yet.
“out of memory,” a crash, or your computer freezes
The model is too big for your VRAM (video memory). Pick a smaller variant from the table above and close other programs, especially games and browsers with lots of tabs.
It runs, but extremely slowly
Your CPU is probably doing the work instead of your graphics card. In Ollama, the PROCESSOR column of ollama ps shows how much runs on the GPU. If it says "100% CPU", update your graphics driver or pick a smaller variant that fits entirely into VRAM (video memory).
“ollama” or “python” is not found
Close the terminal window and open it again so it recognizes the newly installed program. For Python on Windows: reinstall it and check Add python.exe to PATH.
Windows: "Running scripts is disabled on this system"
Run the script with exactly the command above: powershell -ExecutionPolicy Bypass -File try_windows.ps1. This allows execution for this one time only.
Error message when loading the model (version, "trust_remote_code", unknown architecture)
Brand-new models often need the latest software version. With Ollama: update Ollama. With Python: pip install -U transformers in the activated environment. The Model Card usually lists the exact version.
Tokenstrom generates the scripts and notebook automatically. Please take a quick look before running them – they're plain, readable text. Mind the model's license: apache-2.0.
Ask the AI assistant
Questions about this article, installation or background? Our assistant knows this page and runs on our own machine, not in someone else's cloud.
The Assistant is an AI (Gemma 4) and can make mistakes – please check important details against the source. Questions are used only to generate an answer and are deleted after one hour. Please don't enter any personal data.
Comments
What do you think? Already tried it? Write it here, no sign-up needed.
Comments are public. Links aren't allowed, and we delete abusive posts. You can delete your own comment at any time in this browser. Learn more in the Privacy Policy.