A computer in the weights: why hobbyists are compiling CPUs into language models

What this is about

Language models can chat about math, but they botch a twenty-digit multiplication. The usual fix is a tool: the model writes a bit of Python, an external program runs it and passes back the result. A small but very active hobbyist scene is taking a different route and building the computer directly into the model.

Three projects have appeared within just a few months. In March 2026, the startup Percepta unveiled a transformer that runs WebAssembly, a standardized machine format that programs written in C, for example, can be compiled into. At the end of October came Qwenputer by Divyansh Agrawal, a 32-bit RISC-V processor inside Qwen3.5-4B. Now there's Qwenx86 from Hugging Face user Quazim0t0: 13 extra layers sit in front of the 32 layers of Qwen3.5-4B and form a 64-bit x86 machine. The Model Card stresses that these layers contain no learned weights; every value is calculated by a compiler.

Our thesis: this isn't ready to be an everyday tool. But it's a serious research direction that could show whether models will one day do exact computation natively instead of through a tool.

How a program becomes weights

Normally, a transformer learns its weights during training. But you can also calculate them by hand if you know what each layer is supposed to do. Gail Weiss, Yoav Goldberg and Eran Yahav laid the groundwork in 2021 with RASP, a small programming language whose instructions map exactly onto the two building blocks of a transformer. Attention, the mechanism a token uses to look things up in earlier tokens, becomes the operation "find the matching position and read its value". The MLP, the small computing network in each layer, becomes the operation "compute something from what's here". In 2023, DeepMind built a real compiler for it with Tracr, which translates RASP programs into transformer weights. It was intended as a lab for interpretability research, because with a compiled model you know exactly which algorithm it runs. Later came "Looped Transformers as Programmable Computers" (ICML 2023), which wired a simple computer into a 13-layer transformer running in a loop, and Google's ALTA (2024), which also handles loops.

Today's projects use the same basic tricks:

• Attention as memory access. An attention head compares a query against all stored keys and reads the value at the best match. Choose the keys cleverly and exactly one address always wins. Percepta stores each key as the point (2j, −j²). Query with (i, 1) and you get the score 2ij − j², which peaks exactly at j = i. That turns a similarity search into exact array access.
• MLP as an arithmetic unit. Neurons with thresholds behave like logic gates. In Qwenputer, the MLP computes additions over four-bit chunks with carry, multiplies in five stages and works out divisions via sign comparisons. According to the design document, the eleven layers roughly split into fetching the instruction (layers 1–4), executing it (5–9) and writing back the result (10–11).
• Gated DeltaNet as memory. Alongside regular attention, Qwen3.5 uses Gated DeltaNet layers, a kind of fixed-size scratchpad that gets selectively overwritten with each token. Qwenputer writes using "one-hot" keys, where exactly one position is active. That turns each row of the scratchpad into a memory cell that can be overwritten precisely.

Each processor step is a small block of tokens the model generates itself, known as trace or machine tokens. When Qwen writes a <tool_call> in its usual format, the built-in machine takes over the call. The machine tokens are invisible to Qwen's own layers; only the result reappears as normal text. Qwen's original weights remain unchanged. According to the Model Card, Qwenx86's most likely next token matched the original at 48 out of 48 tested positions.

What Qwenx86 can do, according to the Model Card

The core of Qwenx86 is an evolved version of Qwenputer's RISC-V core, extended with x86 flags and conditional jumps. It manages 64 KB of RAM, 256 KB of code ROM and 128 KB of read-only data, all in the DeltaNet state. Indirect jumps and SSE instructions are missing. x86 code is translated into internal micro-ops beforehand. The checkpoint also ships firmware for six tools: an exact fraction calculator, a stripped-down MicroPython, a small C compiler, a notes store, 647 physical constants plus the periodic table, and 1,003 Python documentation entries.

A few caveats matter. The 1,000-out-of-1,000 arithmetic result comes from version 2's "Memory" module. According to the Model Card, it runs alongside the model and is driven by Python code, not by Qwen itself. The GSM8K test covers only 28 problems, so the difference is three problems. With Qwenputer, the developer says GSM8K even ended in a tie, because on word problems the bottleneck is reasoning, not arithmetic. Qwenputer's own numbers are revealing too: on DeepMind's mul_big problem set, accuracy rises from 55.0 to 97.5 percent; with a real Python tool it's 100. So the built-in machine computes exactly. The remaining errors happen when Qwen phrases the problem wrong or misreads the result, and the same is true with an external tool.

Running the numbers: what does a built-in computer cost?

This is where it gets sobering. We derived throughput from the published figures:

For a C program, the Qwenx86 card cites 0.1 to 6 million machine tokens. At around 4,000 tokens per second, by our math that's roughly 25 seconds to 25 minutes. A Python call on your own machine typically takes milliseconds for the same task. Aditya Narayana did the Doom run using a version of Doom designed for microcontrollers that draws the frame in 25 strips. It shows a single verified frame, not gameplay, and Qwen's language layers weren't even loaded. It's starkest with the qwendoom project, which inflated Qwenputer to 4 MB of RAM and 328 layers. A single game frame there costs about 3.7 million tokens and 17 minutes. The full demo with 7,155 frames would take about two months; a regular C emulator does it in 21 seconds. Roughly speaking, that's a factor of about 250,000.

Then there's the hardware. The compact Qwenx86 file is 8.75 GB and needs about 9 GB of VRAM on a CUDA GPU, so a 16 GB card will do. The merged Qwenputer takes up 18.7 GB in bf16; the developer recommends 24 GB. The QwenPU variant with a vector unit needs 40 GB or more. On top of that, the fast throughput figures only work with a custom Triton executor, meaning hand-written GPU code. Via the standard Hugging Face path, the chip manages only about 250 tokens per second, according to the design document.

What's the case for it anyway?

Qwenputer drew 147 comments of intense discussion on Hacker News. One user found it "really not that interesting"; another considered the claim of "no external runtime" "a little disingenuous", since a host program drives the execution. Another criticism also has merit: a calculator that scores 97.5 percent in tests seems odd. With Percepta, the objection was that running instructions on a real CPU will always be faster.

But there are serious arguments in favor too, some from the same discussions:

• No sandbox: No foreign code runs on the host machine. The computer can only write to its 64 KB of RAM, which shrinks the attack surface. Prompt injection, meaning smuggling in outside instructions via text, is still possible, as commenters pointed out.
• Verifiability: Every computation step exists as a token trace and can be checked step by step against a reference. Qwenputer verified around 230,000 positions this way with no deviation; its CHIP-8 predecessor verified 3.9 million.
• No interruptions: Frequent tool calls disrupt batching of many requests on servers. If the computation runs inside the model, that detour disappears.
• A model as a single file: The calculator, reference library and notes store live in the checkpoint and run anywhere the model runs.

Percepta goes furthest on speed. A geometric data structure called HullKVCache cuts the cost per memory access from linear to logarithmic, from "scan every entry" to "keep halving". The team cites more than 30,000 tokens per second on a CPU and a hard Sudoku solved in under three minutes. The code has been released as transformer-vm (Apache-2.0). According to the README, the weights are "computed analytically", with 35 WebAssembly instructions built in directly. An independent reimplementation by blogger Oskar Austegard confirmed the logarithmic addressing in a toy version. But he criticizes the term "convex hull" as misleading and leaves open whether the whole thing beats a real WebAssembly interpreter in practice.

Who else is working on this?

The well-known foundations come from research: Tracr from Google DeepMind, ALTA from Google, Looped Transformers from Wisconsin and Princeton, plus Percepta, backed by General Catalyst. The current wave, Qwenputer, Qwenx86, the Doom spin-offs and the file system and NPU variants, comes from individuals who say they do this in their spare time. We found no evidence that big labs are building compiled compute units into production models. Work like "Compiling Agentic Workflows into LLM Weights" (2026) relies on training and so takes a different approach.

The bottom line

Our take: Qwenx86 and Qwenputer aren't tools for your daily work. If you need exact computation, a Python tool call is faster, cheaper and at least as accurate. But the projects demonstrate two things cleanly and verifiably. First, a complete processor, tested against real emulators, can be translated into the standard building blocks of an ordinary language model without measurably hurting its language abilities. Second, the accuracy gain comes from the compute unit; the remaining errors happen at the interface between language and computation.

The really exciting questions remain open. Can compiled components be built in so that a model learns to use them during training? Can compute units be bolted on like software libraries, as Percepta sketches out? And will it ever pay off compared to a real CPU? So far, almost all figures come from the developers themselves. There's no peer review, and the most impressive demos run on custom executors rather than standard software. As a research direction, it's still worth following.