Analysis

Giant AI models off an SSD: why "runs on a laptop" often means 0.1 tokens/s

Within just a few days, several engines have appeared that load models with hundreds of billions of parameters piece by piece from an SSD. We put the projects side by side and did the math on how fast this can physically go. The upshot: without a good cache and a fast NVMe SSD, it's mostly a demo. With the right model, though, it becomes a genuinely useful tool.

In a short span, we've run four headlines: Kimi K3, with 2.78 trillion parameters, runs on a CPU with 8 GB of RAM. colibri gets GLM-5.2 (744 billion parameters) running on a laptop with 25 GB. Betelgeuse hits 22 tokens per second on an RTX 3060. And a Strata patch starts agent sessions 20 times faster. All of them use the same trick. Mixture-of-experts (MoE) models consist of hundreds of subnetworks, the "experts," …

• Without a cache, GLM-5.2 has to read around 11.4 GB of experts from disk for every token. Even a PCIe 5 SSD at 14.7 GB/s tops out at roughly 1.3 tokens per second in theory.
• Measured speeds for GLM-5.2 range from 0.05 to 2.06 tokens/s depending on the machine. An independent Better Stack test on an M2 Max with an external SSD got 0.1 tokens/s, with more than two minutes before the first token appeared.
• In the C99 engine, Kimi K3 takes 26.5 seconds per token with 8 GB of RAM and 5.6 seconds with more than 128 GB. In the trace analysis, the LRU cache hit only 36 percent of the time.

Revolut says it cut AI coding costs 90% with GLM. Here's what the math shows

Revolut claims it cut coding inference costs by about 90% by making self-hosted GLM its default instead of Claude Code. We run the numbers: what's in the $0.30 per million tokens, what's missing, and at what team size self-hosting pays off. We also look at what independent benchmarks and security findings say about GLM 5.3.

British fintech company Revolut said at the AI Engineer conference in New York (October 2026) that it cut the cost of AI-assisted programming by about 90% in less than a year. The reason: self-hosted GLM models from Chinese lab Z.ai recently became the default, while Claude Code and OpenAI Codex are kept only for exceptions. The Register and Techmeme picked up the figure on November 3, and it has circulated as a …

• According to The Register, Revolut runs 56 Nvidia B200 GPUs (seven DGX B200 systems) for about 2,000 developers and cites roughly $0.30 per million tokens as a blended price.
• Per reports, the "90%" comes from the headline and the talk (AI Engineer NYC, October 2026): the bill is about one tenth of the earlier Anthropic and OpenAI spend. Revolut doesn't disclose usage volume or staffing costs.
• Our recalculation: at rental prices of $3.69 to $6.82 per GPU-hour, 56 B200s running around the clock cost about $1.8 to $3.3 million per year, so the $0.30 price implies trillions of tokens per year.