Bonsai 27B squeezes a 27B model into 3.9 GB, but the memory math has a catch

What's going on

If you run AI language models on your own machine, you'll hit one limit fast: your graphics card's memory (VRAM). A 27-billion-parameter model needs around 54 GB just for its weights at standard 16-bit precision (FP16). That won't fit on any consumer GPU. US startup PrismML, a spinout from researchers at the California Institute of Technology (Caltech), went after exactly this problem. Its model Bonsai 27B crams the weights of a 27B model into about 3.9 GB.

Bonsai 27B isn't a newly trained model, though. It's an extremely compressed version of Alibaba's Qwen3.6-27B, with the architecture left unchanged. According to ad-hoc-news, it launched in July 2026 in two flavors: a 1-bit version at 3.9 GB and a ternary version at 5.9 GB. Tech site MakeUseOf has now put the 1-bit version through a thorough test and come away with mixed feelings. Meanwhile, PrismML has already shipped a successor, Bonsai 2 27B.

The trick: one bit per weight

Normally, every number in a model is stored with 16 bits. Bonsai uses the Q1_0_g128 format, a form of quantization: the model gets smaller because its numbers are stored less precisely. Here, that precision is about as coarse as it gets. Each weight is reduced to a single sign bit, with 0 meaning negative and 1 meaning positive. Every 128 weights share one FP16 scaling factor that determines how large the value actually is. All told, that works out to about 1.125 bits per weight. PrismML says that's roughly 14.2 times smaller than FP16.

PrismML also takes a swipe at the competition: a popular "2-bit" build of Qwen3.6-27B, it says, actually averages 2.8 bits per weight and takes up 9.4 GB.

One question remains open: PrismML won't say how it decides which way each weight's sign points. According to MakeUseOf, an independent analysis confirmed that the published weights are genuine and work well. But it also found that none of the white papers explain the method. If you simply quantize a model down to 1 bit yourself using the public format, you get a model that fails completely. The real know-how lives in the unpublished process.

How much performance survives

PrismML ran 15 benchmarks in thinking mode, where the model reasons at length before answering. All figures come from the company:

So the model keeps about 89.5 percent of the original's performance. Math and code hold up well. It gets noticeably weaker at following instructions and at tool calling, where the model independently invokes external tools like search functions or programs. PrismML counters that conventional 2-bit builds fail in sneakier ways. IQ2_XXS, for example, scores 88.93 on the MMLU-Redux knowledge test but only 57.5 on the AIME26 math competition. Weaknesses like that don't necessarily show up in general-purpose tests.

If you have a bit more memory, you can opt for the ternary version. Ternary means each weight can take one of three values: negative, zero or positive. According to MakeUseOf, this version keeps 94.6 percent of the original's performance.

4 GB of weights doesn't mean 4 GB of memory

This is where MakeUseOf gets blunt: the weights take up only 3.79 GB, but your graphics card has to hold more than that. There's the KV cache, the model's short-term memory of the conversation so far, plus runtime buffers. PrismML itself lists these peak figures:

• 4,000 tokens of context, FP16 KV cache: 5.2 GB
• 100,000 tokens of context, FP16 KV cache: 11.6 GB
• 100,000 tokens with a 4-bit KV cache: 6.86 GB
• Full 262,000-token context window with a 4-bit KV cache: 9.4 GB

Long contexts stay this cheap because of the architecture: only 16 of the model's 64 layers keep a full attention cache. Still, MakeUseOf says you'll realistically want a card with 6 or 8 GB for comfortable use.

The tester tried it on a laptop with a GTX 1650 Ti (4 GB). LM Studio immediately suggested partial GPU offloading and put just four layers on the graphics card. Trying to load the whole model onto the GPU at 4,096 tokens of context failed with the error "failed to allocate buffer for kv cache". It only worked at 50 percent offload. That used about 3.6 GB of VRAM, with the rest computed on the processor (CPU).

Fast on GPUs and Macs, slow on CPUs

PrismML reports 26 tokens per second on an M4 Pro Mac, 44.2 on an M5 Pro and 66.4 on an M5 Max. With the optional DSpark add-on, an H100 data center card jumps from 104.8 to 143.8 tokens per second. DSpark uses speculative decoding: a small helper guesses several words ahead and the big model just checks them. According to ad-hoc-news, the model ran on an iPhone 17 Pro at around eleven tokens per second.

Independent tests are more sobering. A DataCamp tester measured about 92 tokens per second on an RTX 5070 Ti, with the model using around 11.9 GB of VRAM at 100,000 tokens of context. For work as a coding agent, though, he still found a regular Q4 version of Qwen3.6-27B more reliable. On a Ryzen 5 5600 with no graphics card, Big Iron got 6.7 tokens per second for generation and just 13 tokens per second for prompt processing. On homelab questions, Bonsai scored 32 out of 40, while the much smaller Qwen 3-4B scored 35. Because llama-server caches prompts, memory use climbed to 16.3 GB.

It was even slower on MakeUseOf's laptop: with half the model on the GTX 1650 Ti and the rest on a Core i5-9300H, it produced about 4 tokens per second, and just 2.4 with no GPU at all. With an input of around 1,600 tokens, it took almost a minute before the first word appeared. Bonsai correctly answered a simple question about when a train would arrive, but spent more than three minutes thinking about it.

The successor: Bonsai 2 27B

On September 17, 2026, PrismML announced Ternary Bonsai 2 27B in a press release. It's based on Qwen3.8 27B, has 27.8 billion parameters, understands text and images and, according to the company, was trained on Google's TPU v5. The file is 5.9 GB, more than nine times smaller than the original. Across 20 benchmarks, PrismML says the model scores 83.9 versus 85.4 for the original. That's 98.2 percent; the first ternary Bonsai came in at around 95 percent. It's said to hit up to 143 tokens per second on an RTX 5090. The model is free to download under the open Apache 2.0 license.

"Bonsai 27B proved that capable models don't have to be confined to the cloud," says PrismML founder and CEO Babak Hassibi.

PrismML's backers include Khosla Ventures, Cerberus, Google and Samsung. According to ad-hoc-news, the company has raised the equivalent of around €14.7 million.

What you need to try it

• Graphics card: In theory, 4 GB of VRAM is enough for the weights, but in practice you should have 6 to 8 GB. With a 4-bit KV cache and a very long context, you'll need up to 9.4 GB. Cards with 8 or 12 GB are ideal, since a regular 27B model wouldn't run on them at all.
• Mac: Apple Silicon works well (Metal support). PrismML reports 26 tokens per second even on an M4 Pro.
• CPU only: It works, but it's very slow (2 to 7 tokens per second, with long waits on long inputs). Plan for at least 16 GB of RAM.
• Disk space: About 5 GB. In LM Studio it's 4.73 GB, because the vision component (0.93 GB) is downloaded too.
• Software: LM Studio (which lists 4 GB RAM as the minimum) or a recent llama.cpp. PrismML's own fork is faster on CPUs.

The bigger picture

Bonsai 27B shows how far extreme quantization has come: a 27B-class reasoning model on an ordinary laptop was unthinkable not long ago. But the headline "4 GB" figure only applies to the weights, not to the memory you actually need. Almost all the benchmarks come from PrismML itself, and the method remains secret. Independent tests show that a standard 4-bit version is more reliable for agentic coding tasks if you have enough memory. PrismML itself concedes that long agentic coding tasks spanning many files aren't this model's strong suit. If you're getting started today, take a look at Bonsai 2 27B too: it's 2 GB larger, but according to the company it loses far less quality.