llama-modchip runs 284B-parameter DeepSeek in 64 GB of RAM, no GPU required

What's going on

Very large language models usually run on expensive GPU servers. The open-source project llama-modchip, from a developer or group called Synchopate, wants to prove there's another way. It aims to run DeepSeek-V4-Flash, a model with 284.33 billion parameters, on a regular computer with 64 GB of RAM. No GPU, and with a 256,000-token context window. The context window is how much text the model can keep "in view" at once. 256K works out to roughly several hundred book pages.

The Korean portal GeekNews and the news site news.lavx.hu both covered the project. According to lavx, the code, a setup script, command-line tools and prebuilt Linux binaries live in the GitHub repository synchopate/llama-modchip. You'll also find links to model downloads, optimization notes and an architecture document there.

How the trick works

DeepSeek-V4-Flash is a Mixture-of-Experts (MoE) model. It's built from many specialized sub-networks ("experts"), and only a few of them do any work for each token. That makes it faster than a dense model of the same size, where everything computes all the time. Memory is still the problem, though, because all the experts have to be stored somewhere.

llama-modchip goes after this in two places:

• Compression: A custom method called "SwiGLU-H2" and a homegrown quantization format called "Fractal-64" are supposed to shrink the model a lot. Quantization means storing the model's numbers at lower precision, a bit like a photo that gets by with fewer color shades. The project says this cuts the DeepSeek file from 76.87 GiB (compared with the well-known, very aggressive 2-bit IQ2_XXS format) to 56.56 GiB. SwiGLU is a specific building block in the layers of modern language models. The reports don't explain how the "H2" variant works.
• Optimized CPU routines: The computation is split into small tiles (tiling) sized to fit the processor's registers and its L1, L2 and L3 caches, the tiny, extremely fast memory right on the chip. The code uses AVX-512 instructions, which let modern processors crunch many numbers at once. The project says this made prefill 77 percent faster than its own earlier version.

According to GeekNews, the system doesn't need NVMe swap. Nothing gets offloaded to the SSD, which would otherwise slow things down badly.

The numbers

All measurements come from the project itself. The test machine is an AMD Ryzen 9 9955HX with 64 GB of DDR5 memory.

Keep in mind that prefill (reading the prompt) and generating the answer are two separate phases. As lavx points out, the higher prefill number tells you nothing about how fast replies show up in a chat. The number that matters there is the roughly 9 to 15 tokens per second, which is about a comfortable reading pace. "Top-6" probably means the number of experts active per token, but the sources don't say for sure.

The project also lists smaller MoE models. For NVIDIA's Nemotron-3.5-30B-A3B it reports 25 to 60.4 tokens/s on a 16-core Zen 5 system, depending on the test and method, and as much as 111.62 tokens/s on a Threadripper.

Stockfish-PVS and RouteCrack: faster by guessing

For the extra speed, llama-modchip uses speculative decoding. A small, fast helper proposes the next few tokens, and the big model only checks them instead of working out every token itself. When the helper guesses right, you save time. The project calls its version "Stockfish-PVS", apparently a nod to the Stockfish chess engine and its Principal Variation Search algorithm. It comes with a tiny 1.64 MB draft component called RouteCrack. On the Nemotron model, version 2 reportedly hits a 96.4 percent acceptance rate. lavx notes that the gains depend heavily on the model, hardware, prompt and acceptance rate, so the peak numbers aren't typical speeds you should expect on other systems.

The repository title also mentions a "13 ms Claude Code restore" feature. The available reports don't explain what that actually is.

The catches

Tight memory: According to lavx, a 64 GB system gives you about 59.6 GiB before the operating system and other programs take their share. With peak usage of up to 58.9 GiB, there's almost no headroom. So there's no guarantee it will run reliably on every 64 GB machine. Even an open browser could tip it over.

Custom format: Fractal-64 files use their own tensor type (ID 40). According to the repository, you can't load them in stock llama.cpp, Ollama or LM Studio, only in the project's patched runtime with its own kernels. If you're used to those familiar front ends, you'll have to switch.

Unclear quality: The project does publish "quality gate" and similarity scores for its quantized models, but nobody has checked them independently. With compression this aggressive, the key question is how much accuracy the model loses. You should test that yourself for your own use case.

Hardware: The optimizations depend on AVX-512. Current AMD Zen processors support it, but not every CPU does. Testing was done on a high-end laptop processor with fast DDR5 memory.

The bigger picture

If the claims hold up, this would be remarkable: a 284-billion-parameter-class model with a huge context window on a machine with no graphics card, at a usable chat speed. Until now that has meant multiple GPUs, a lot more RAM, or expensive workstations or Macs with large unified memory. Projects like llama.cpp have shown how much headroom CPU inference still has, and llama-modchip pushes that further with aggressive compression and hand-tuned kernels.

Still, some skepticism is in order. Every number is self-reported, the memory budget is cut extremely close, and the proprietary file format ties you to a single, still-young piece of software. If you're a tinkerer with the right hardware, it's an exciting experiment. Before it counts as a dependable everyday option, it needs to prove itself, ideally through independent tests of both speed and answer quality.