Qwenputer: A RISC-V processor running inside Qwen 3.5's weights

What this is about

Under the name Qwenputer, developer Divyansh Agrawal has published an unusual experiment: an unmodified Qwen 3.5 4B (a language model with about 4 billion parameters, i.e. learned tuning knobs) with a working 32-bit RISC-V CPU compiled into its weights. RISC-V is an open processor architecture. The CPU runs entirely inside the model's forward passes and can execute arithmetic as well as C and MicroPython code, with no external program, tool or sandbox. The model is available on Hugging Face and ModelScope, and the code is on GitHub.

The trick: Gated DeltaNet as exact working memory

Qwen3-Next and Qwen 3.5 use Gated DeltaNet in three out of four attention layers. It's a linear RNN (a network with a running memory state) used as the attention mechanism. Each head keeps a state matrix as its memory. If you set the gate (which controls forgetting) to 1, the write strength beta to 1 and use one-hot keys (a vector with exactly one 1 that points to a row like a memory address), every write overwrites exactly one row and every query returns the exact value. That turns the layers into real RAM. Unlike the KV cache of normal attention, its size stays the same regardless of conversation length. Schlag et al. (2021) had already shown that the delta rule works as a memory. According to the author, what's new is making it exact and addressable in a production model.

A CPU without training

A compiler sets every weight directly. The CPU is described as a circuit (read registers, add, branch conditionally), and each step becomes part of a transformer layer: attention heads do lookups, feed-forward units compute, and DeltaNet heads hold the RAM. Each instruction corresponds to a few generated tokens (ADDR, VAL, STEP). These tokens are the CPU's execution trace.

The CPU implements RV32IM plus the Zba extension and a small bignum unit for large numbers, and takes up 11 layers ahead of Qwen's 32. It passes the official riscv-tests, its programs match the qemu emulator, and it executed 18 million instructions without a single error. It is not yet a CPU that can run Linux. Qwen's original weights stay unchanged, while the compute weights sit in extra dimensions and layers. Qwen uses the CPU without any further training by relying on the familiar tool-call format. A small driver called RvChat loads the firmware into the DeltaNet state.

Results

On NUPA (long integer arithmetic), Qwenputer rises from 48.6% to 100%, on par with the Python tool. Around 230,000 output positions were checked against a reference machine with no deviation. Normal text stays close to the base model (KL 6e-4 against a noise floor of 5e-4).

A file system inside the model

There is also an experimental file system with the operations put, append, get, list and delete. File names are hashed to one of 1,024 directory slots, the text lives in other heads, eight tokens per line. It holds up to 512 files, about 53,000 tokens of text. The state can be saved and loaded into a new conversation. With it, Qwen reproduces updated facts correctly 90% of the time, versus 30% when you paste in old chats. Python-based memory tools reach 83.3% versus 78.9% (as stated in the original). The code is on the rv-store branch.

What you need to try it

• A single GPU with 24 GB of VRAM.
• The checkpoint is 18.8 GB (about twice the size of base Qwen, mostly zeros as padding), so plan for a good 20 GB of disk space.
• Tools: uv and the Hugging Face CLI; a Linux/Python environment.
• There is a live demo on ModelScope. The author tested throughput on an L4 GPU (about 3,200 CPU tokens per second).
• Running on a CPU or Mac isn't specified.

Limits

This is an experiment. It needs the RvChat driver and doesn't yet run with a simple generate() call. It's slow because every CPU instruction costs several tokens, so big calculations take seconds instead of milliseconds. Next, the author plans to have Qwen write Python and run it on the MicroPython inside its own weights, plus a faster CPU. Two follow-up articles cover AI safety: how hybrid models could hide their reasoning in the RNN state and evade CoT monitoring, and how compiled weights could serve as monitors that can't be switched off.

Our take

Qwenputer shows that a language model can run exact, deterministic computation internally, an approach that could make tool calling partly unnecessary. In practice it's still slow and clunky, and all the numbers come from the author, not independent verification. The benchmarks are limited to arithmetic, where Python tools already work perfectly. The most exciting part is the research angle: exact control over the RNN state as a testbed for interpretability and safety.