Mirage: 22 MB vision model detects objects, reads text and segments in one pass

What's going on

Startup Cactus Compute has unveiled Mirage, a very small vision model for smartphones, wearables, robots and other consumer devices. Mirage has 51.3 million parameters and, using the company's own 2-bit quantization scheme "Cactus Quants," comes in at only around 22 MB. Quantization means shrinking a model by storing its numbers at lower precision. For comparison: typical vision-language models (VLMs, models that understand both images and text) usually take up several gigabytes.

Despite its size, Mirage is said to describe scenes, detect objects, read text (OCR), generate segmentation masks (pixel-accurate object outlines), work out spatial relationships and make simple decisions, all in a single pass. On a Samsung Galaxy S26, that takes 74 milliseconds on the NPU, the phone's AI accelerator chip, and 107 milliseconds on the regular processor, according to the company. The measurement covers the entire pipeline from image to result.

Henry from Cactus Compute introduced the model on the r/LocalLLaMA subreddit. Weights and config are on Hugging Face, the source code is on GitHub. Mirage runs on the same engine as the company's sibling models Needle and Whistle. The blog post includes a demo that runs directly in your browser: point your camera at a mug or a label, or upload a photo, and the detections appear as an overlay on the image.

What Mirage returns

Instead of free-form prose, your app gets a structured JSON record it can work with right away:

• Scene: What kind of place is this, and what are the lighting, surfaces and background like?
• Objects and attributes: names, bounding boxes, positions and attributes such as color, shape or material. The names come from the model's language component, so they aren't limited to a fixed list of classes the way classic object detectors are.
• Segmentation: a mask for every described object, as an outline, as individual pieces or as a bitmap.
• Text: written words in the image, each with a bounding box and confidence score. They're read by a dedicated text recognizer inside the model.
• Relationships and events: Which object is attached to which, what's on the table, what action is happening?
• Decisions and forms: a choice from predefined labels, a count within a range, a short text field or a box belonging to a selected object.

Together, names, attributes and relationships form a small scene graph, a network of objects and how they connect. An app can use it to jump from a sign to its pole, for example, or pull out every red object. There's also a short reasoning trace that shows how the model arrived at its description. Cactus itself points out, though, that this text is generated too and should be read critically.

Descriptions are available in English, German, French, Spanish, Italian, Dutch and Polish. The Python API lets you pick exactly which parts you need. Boxes are given on a 0 to 1000 scale relative to the image size. Parameters such as conf and text_conf set detection thresholds, while max_objects, max_text_lines and max_tokens cap the output. Set language="de" and you get the description in German. On the first call, the package downloads the engine and weights and caches them. The image analysis itself runs on your own device.

Letting text models see, no fine-tuning required

Cactus calls out the most interesting finding itself: you simply pass Mirage's JSON to a text-only model along with a question about the image. Mirage does the seeing, and the language model reads the observations as plain text. Neither model needs to be adapted.

The team tested this with 256 questions about 128 images from the VQAv2 dataset (Visual Question Answering). They measured the standard VQA consensus score from 0 to 100:

For Qwen3-0.6B, the jump is big. Control runs show that the right image data is what matters: given the record from a different image, the model scored only 22.2. Without the reasoning trace, it even hit 33.1. So Cactus found no reliable benefit from the reasoning; the structured data was enough.

With larger text models, Cactus says the results are mixed. The scores also blend correct answers with format compliance: the task required a JSON object with an "answer" text field, and any response that missed the format scored zero. Nemotron failed this on 214 of 256 answers, Qwen3-1.7B on 34. Cactus also lists perception errors. On a skateboarding photo, Mirage missed the skateboard and described cars, people and shoes. From the question alone, Qwen had correctly guessed "a skateboard"; after seeing the Mirage record, it answered "a car." What Mirage doesn't see, even the best text model can't know. Next, Cactus plans to fine-tune a text model specifically on Mirage records paired with questions and answers.

How the model is built

The 51.3 million parameters are split between a 7.8 million-parameter image encoder, a 40.8 million-parameter language decoder and 2.6 million parameters of heads for text recognition and spatial outputs. Henry explains the core idea like this: full sentences, connecting filler words and open-ended reasoning eat up capacity that a small model is better off spending on the image itself. That's why Cactus built structure into the task, with dedicated heads for spatial outputs, a dedicated text recognizer and short references linking names to image regions.

The encoder uses a building block called HConvGLU. Fixed Walsh-Hadamard transforms, a mathematical reshuffle that mixes all channels together, wrap three small learned convolutions with 65 channels. Each convolution stores 512 × 65 values, roughly eight times fewer than a full 512 × 512 matrix. The encoder works on 256 image cells and uses grouped-query attention, which shares computation to save memory. On top of that come four "register tokens" that act as scratch pads, plus a normalization that balances the channels. The detection and mask heads receive finer image features from the early layers so details are preserved.

The decoder has eight layers and comes from the same family as Needle and Whistle. It generates names, attributes, relationships and the reasoning trace, and points to detected boxes with short references. The engine merges all of that with the boxes, masks and text recognition into the JSON. That way the decoder has to generate fewer tokens. For quantization, most matrices are reduced to 2 bits. Some sensitive parts stay at 4 or 16 bits, which brings the file to 21.6 MB.

What you need to try it

The barrier to entry is unusually low: the browser demo on the Cactus blog runs locally in your browser, with no installation and no graphics card. For your own projects, there's a Python API that accepts file paths, image bytes, Pillow images or RGB arrays. Native apps load the mirage.cact file with cactus_load and analyze images with cactus_describe. You don't need a GPU: Mirage is built for phone processors and NPUs, and a few hundred MB of RAM plus a few MB of storage are enough. A separate repository lists exactly which platforms the engine supports. There's no GGUF version for llama.cpp or Ollama; Mirage requires the Cactus engine.

The bigger picture

Mirage shows how far visual understanding can be shrunk. Normally, detection, text recognition, segmentation and captioning take several models. Here, it all fits into 22 MB that run on a phone in a tenth of a second. That's interesting for apps that need to work offline and keep data on the device, such as assistive tools for blind and low-vision users, robots or camera glasses. The idea of teaching text models to see via structured observations is also appealing for small local systems.

All figures come from the company, though, and there are no independent tests yet. The VQA sample of 256 questions is small, and absolute scores around 31 out of 100 show the duo is still far behind large vision-language models. Cactus is upfront about weaknesses, including missed objects and mixed results with larger text models, and has published every answer from the experiment. The source doesn't mention a license for the weights. If you want to use Mirage commercially, check the license on Hugging Face.