Ternary Bonsai 2 squeezes a 9B vision model into just 2.42 GB

What's going on

AI company Prism ML (also written PrismML) released the Ternary Bonsai 2 model family on October 15, 2026, in 2B, 4B and 9B sizes. The models are built for everyday devices: smartphones, laptops and machines without a powerful GPU. Most of the attention is going to the largest variant, Ternary-Bonsai-2-9B. It's based on Alibaba's Qwen3.5-9B and understands both text and images, which makes it a vision-language model. Its language model has around 8.95 billion parameters and takes up just 2.42 GB on disk in the default version. In the usual 16-bit format (FP16), it would be about 17.9 GB.

All three models are available for download under the permissive Apache 2.0 license in the "Bonsai 2 Small" collection on Hugging Face. According to alphasignal.ai, the 9B repository had more than 896,000 downloads and 3,700 likes at the time of reporting. The site points out, though, that Hugging Face also counts repeated and automated downloads.

How 9 billion parameters fit into 2.42 GB

The trick is ternary quantization. Quantization shrinks a model by storing its numbers at lower precision. Normally, each weight is a 16-bit floating-point number. In Bonsai 2, it can only take one of three values: −1, 0 or +1. To avoid losing too much information, each group of weights shares a common FP16 scaling factor. That factor sets how "loud" the three values are within the group. According to alphasignal.ai, the method covers almost everything: the embeddings (which turn words into numbers), the attention projections, the MLP layers and the so-called LM head, which picks the next word at the end.

On top of that comes a Hadamard rotation. This mathematical rotation is baked into the stored weights. At inference time, the software applies the matching inverse rotation to the intermediate values (activations). This spreads individual extreme outliers across many dimensions. Without this step, those outliers would cause large errors when rounded to three values. Custom compute routines (kernels) read the packed weights directly, without first building a full FP16 copy in memory.

The architecture is unchanged from Qwen3.5. That includes hybrid attention: some layers use efficient linear attention, the rest use classic, compute-heavy attention. That's what makes the large 262,144-token context window affordable in the first place, meaning the amount of text the model can take in at once. The model also includes MTP speculative decoding. The model predicts several tokens at once and then verifies them, which speeds up output. There's also an optional small prediction helper called the "DFlash2 Drafter".

Three file formats, one important trap

Prism ML ships the 9B language model in three GGUF variants. GGUF is the file format used by llama.cpp and many apps built on top of it.

The vision component ships separately and comes on top of these sizes. According to alphasignal.ai, the 7.4x factor applies only to the language weights on disk. At runtime the model needs more memory, because the context cache, compute buffers and runtime overhead add up. If you max out the full 262,000-token context window, you'll need "significantly" more memory, according to the source, and should expect noticeably longer wait times.

OrcaRouter issues a warning in an analysis of the big sibling, Ternary Bonsai 2 27B, and it likely applies to the whole family: stock llama.cpp rejects Prism's own formats as unknown. But it loads an older ternary format without any error message and then produces fluent-sounding nonsense, because the matching activation rotation is missing. So with the wrong software, you don't get a crash. You get confidently wrong answers.

The whole family: from smartphone to desktop

According to GIGAZINE, Prism ML released the three small models together. They use less than one-ninth of the memory of their base models:

Prism ML says it specifically optimized the models as local assistants. They're meant to follow instructions reliably, avoid getting stuck in endless loops, call the right tools (tool calling) and skip unnecessary output. In the company's own benchmarks, GIGAZINE reports, the models land only slightly below their base models, and in some tests the compressed versions even come out ahead. These numbers haven't been independently verified.

On speed, Prism ML quotes up to 34 tokens per second for the 9B model on the iPhone 18 Pro Max and up to 120 tokens per second for the 2B model. A demo video shows the 9B model handling agentic tasks on the iPhone, meaning multi-step tasks involving tool use. Adrien Grondin, developer of the free Locally AI app, wrote on X:

"The new Bonsai 2 2B hits up to 120 tokens/s on the iPhone 18 Pro. It's better than Qwen 3.5 2B (4-bit), runs twice as fast and needs almost half the memory."

Locally AI already supports all three models on iPhone and iPad.

What the big sibling tells us

Back in September 2026, Prism ML released Ternary Bonsai 2 27B, a 5.9 GB version of Qwen3.8-27B. Prism ML says it reaches 98.2% of the original's performance (83.9 vs. 85.4 points averaged over 20 benchmarks). OrcaRouter broke down these vendor numbers in more detail. On math, coding and instruction following, the gap is tiny. On knowledge questions and image understanding, it's bigger. On long, multi-step software tasks, though, the model drops off sharply: 60.8 instead of 80.6 points on SWE-bench Verified, and 52.8 instead of 69.7 on Terminal-Bench 2.1. The sources don't include comparable detailed numbers for the 9B variant. But the finding hints at where ternary models may hit their limits.

What you need to try it

• GPU: For the 2.42 GB file, a card with about 4 GB of VRAM is enough if you use a moderate context. Long contexts or image input need more.
• No GPU: According to alphasignal.ai, the small files also run on CPU-only systems and Apple Silicon. For CPU inference, 8 GB of RAM is a reasonable minimum.
• Smartphone: The Locally AI app runs it directly on iPhone and iPad, no PC needed.
• Storage: 2–3 GB for the language model, plus the separate vision file.
• Software: PTQ1_0 and PQ2_0 require Prism ML's llama.cpp fork. With stock llama.cpp or tools built on it, only the Q2_0 file works.

The big picture

Ternary Bonsai 2 shows how far model compression has come: a 9B model that understands images and has a huge context window fits into just under 2.5 GB. That makes local assistants possible on devices that could previously only handle much weaker models, especially smartphones. The permissive Apache 2.0 license also allows commercial use.

There are weak spots, though. All performance numbers come from Prism ML itself. The analysis of the 27B version suggests the average score hides losses on demanding agent and coding tasks. The special software requirement is a real hurdle, and with the wrong app you risk silently wrong results. In the end, only your own testing will tell you whether the model works for your tasks.