Kimi K3, a 2.78-trillion-parameter model, runs on a CPU with 8 GB of RAM

What this is about

The GitHub project kimi-k3-in-c by Fareed Khan (and other contributors) shows that even a model with 2.78 trillion parameters can run on ordinary hardware, as long as you're willing to wait a very long time. It is an inference engine for Moonshot AI's Kimi K3, written in portable C99: no BLAS library (a library for fast matrix math), no AI framework, no GPU. According to the README, the entire engine is 176 KB. Measured peak usage is 8.24 GB of RAM with a 1.56 TB checkpoint on disk. The project is licensed under Apache-2.0, is at version 1.1.0, and runs on Linux (reference), macOS/arm64, and Windows.

Important: this isn't a new model. It's a tool for running an existing one. You download the Kimi K3 weights separately from Hugging Face.

The trick: memory becomes a dial

Kimi K3 is a Mixture-of-Experts model. It consists of a very large number of sub-networks ("experts"), of which only a few are active per token. According to the README, 16 of 896 experts are selected per step. The engine exploits this radically. Four decisions about where which bytes live are meant to take the model from a server cluster to a laptop:

1. Experts stay 4-bit: The roughly 1.45 TB of expert weights are never fully loaded into memory. They are multiplied directly from their packed 4-bit form (MXFP4), which is how they ship anyway.
2. KDA attention: An attention mechanism whose memory doesn't grow with text length.
3. MLA: Instead of 96 attention heads, a single shared compressed vector ("latent") is stored, which keeps the cache small.
4. Streaming the trunk: The dense 93 layers (the "trunk") are repacked once into a 109 GB file, so each layer can be loaded with a single read. You decide how many layers stay in RAM permanently. According to the author, this turns a fixed floor into a "dial".

There is also an LRU cache for experts (the most recently used ones stay in memory). The consequence, per the README: the same model runs with 8 GB and with 224 GB, and produces byte-identical output at every budget. More memory only makes it faster.

The numbers: how slow is "slow"?

Time per token depends on RAM (measured on a machine with 124 cores and a fast NVMe SSD):

The demo runs make it concrete. On the "laptop" preset, the engine generated eight tokens for "The capital of France is" in 261.5 s (32.69 s/token on average, recorded on a slower disk) at 8.24 GB peak RAM. On the "server" preset, 28 tokens for "def fibonacci(n):" took 299.3 s (10.69 s/token) at 127.92 GB. Compared with version 1.0.0, compute per token is said to be about 8x lighter, a follow-up question in chat 3.9x faster, and long prompts about half as expensive.

The presets at a glance (trunk/expert cache budget in GB):

Features and verification

Besides raw text continuation, there is a chat mode using Kimi K3's official XTML format, including a thinking block and conversation history as a JSONL file. Version one is text-only, with no tools or images. Thinking is on by default (effort "max"). According to the author, --no-think is the biggest speed lever: in one run, 119 of 150 tokens in a five-word answer were the thinking block. Sampling (temperature, top-p, seed) is supported, and the default is greedy decoding.

For verification there is a test ladder: make test runs without a model in under a minute and compares the engine against a PyTorch reference model on a 13-layer test model with the same tensor graph. Results were also checked on the real checkpoint. A script verifies a complete download byte by byte, since an incomplete download produces wrong tokens rather than error messages. Contributors provided macOS, Windows, and NEON ports, chat mode, and faster MXFP4 kernels.

What you need to try it

The bottleneck is storage: about 1.7 TB free (1.56 TB checkpoint plus the 109 GB packed trunk), ideally on a fast local NVMe SSD. According to the author, "storage is the whole game". RAM: from 8 GB, every preset runs. CPU: AVX2 + FMA on x86-64 or NEON on ARM. Compiler: GCC ≥ 9 or Clang ≥ 10, plus make or CMake, and Python 3.9+ for the download and packing scripts. The download takes hours and needs a Hugging Face token. You don't need a graphics card. Apple Silicon Macs are supported, and Google Colab is unrealistic given the storage requirements. You can check the engine beforehand, without a download, using make and make test.

Background: Kimi K3 and K3.1

According to TechRepublic, Moonshot AI introduced Kimi K3 with 2.8 trillion parameters, 1M tokens of context, and vision as the "world's first open 3T model". K3 ranked first in Arena.AI's Frontend Code Arena there. Overall, according to Moonshot, it trailed the strongest proprietary systems. The API costs $3 per million input tokens and $15 per million output tokens. The weights were due to be released there on July 27.

According to Unite.ai, Kimi K3.1 Pro followed on October 24, 2026. It is a retrained K3 with a larger post-training effort and 1M context. Moonshot cites, among other figures, 65.7% on Terminal-Bench 4.0 (GPT-6 Astra 58.2%, Claude Opus 5.5 66.4%) and 92.7% on BenchCAD. For the open weights, the company says it removed cyber data from training, and the weights are due by November 4, 2026. Geeky Gadgets (citing a YouTube channel) repeats these numbers in a very promotional tone and also quotes "per task" prices that are hard to verify, so treat them with caution. Whether the C engine supports K3.1 isn't stated in the README at hand.

Our take

The project is less an everyday solution than a proof of concept: 26.5 seconds per token means even a short answer takes hours. It's interesting for developers and tinkerers who want to understand how giant models can be brought to small machines through streaming, 4-bit experts, and caching, and for cases where a model simply needs to run reproducibly on unremarkable local hardware. For interactive work you'd need 128 GB+ of RAM, and even then 5.6 s per token is far from cloud speed. All measurements come from the project itself and haven't been independently confirmed. Speed depends heavily on the SSD. Byte-identical output at every memory budget is a strong quality promise, because no additional quantization of the trunk is used. The author explains at length why in the section "Why the trunk is not quantised".