WiCi Infer: free Mac app streams MoE experts from your SSD to run 800B models
Startup WiCi has released WiCi Infer, a free Mac app, along with InferSSD, an open-source engine. For Mixture-of-Experts models, they load only the parts needed right now from your SSD. WiCi says that lets giant models with around 800 billion parameters run on ordinary machines, though slowly.

What's going on
Startup WiCi is best known for the WiCi One, its "wireless graphics card." Now it has released software for local AI: WiCi Infer, a free desktop app for macOS with no subscription and no sign-up. Under the hood is InferSSD, an inference engine that WiCi says has been open source from day one. An inference engine is the program that actually runs a trained language model and generates its answers. According to Androidguys, you can get the app, the benchmarks and the download on WiCi's official website.
The pitch is a big one. Language models with around 800 billion parameters are supposed to run on computers whose RAM and VRAM (video memory) are far too small for them. And you shouldn't need to compress the models any further to get there.
The memory problem and how InferSSD gets around it
If you run a language model locally, you almost always hit the same wall. Normally the entire model has to fit into RAM or VRAM. For an 800-billion-parameter model, that's still hundreds of gigabytes even after heavy shrinking. Hardly any home computer has that much.
InferSSD takes advantage of how many modern large models are built: the Mixture-of-Experts (MoE) architecture. An MoE model is made up of many specialized sub-networks called "experts." For each word, or more precisely each token, a component called the router picks just a few of them. Think of a big medical center: dozens of specialists work there, but each patient only sees two or three.
That's where InferSSD comes in. WiCi says the engine doesn't keep every expert in memory. It loads only the ones it needs right now, straight from the SSD. It does this with three building blocks. Expert indexing keeps track of where each expert sits on the drive. Router-driven streaming loads experts based on what the router decides. Adaptive CPU/GPU scheduling keeps adjusting how work is split between the processor and the graphics chip. According to Androidguys, WiCi has also published a technical write-up explaining how the engine decides which weights stay in memory.
The basic idea isn't entirely new. llama.cpp can also read model files from disk via memory mapping. But when memory runs short, that gets painfully slow. WiCi wants to handle it in a more targeted, efficient way.
The numbers, according to WiCi
Every performance figure comes from WiCi's own benchmarks, and none has been independently verified yet. The comparison is against llama.cpp, the popular open-source program that many other local AI apps are built on.
| Hardware | Model | InferSSD vs. llama.cpp (decode) |
|---|---|---|
| MacBook Pro, M5 Max, 48 GB unified memory | Muse Spark Open (816B) | 6.2 times faster |
| NVIDIA DGX Spark, 128 GB unified memory | Muse Spark Open, 4K prompt | 2.14 vs. 0.12 tokens/s (17.8 times faster) |
"Decode" is the phase where the model writes its answer token by token. Keep these numbers in perspective. 2.14 tokens per second is much better than 0.12, but it's still only about one or two words per second, which is a slog to read along with. It's unlikely to be enough for long coding tasks. WiCi itself admits that results vary by model, hardware and settings. It says the gap to llama.cpp was clear in every configuration it tested, though.
The giant models WiCi lists as supported are DeepSeek V4.1 Flash with 763 billion parameters and Muse Spark Open with 816 billion. Smaller models are supposed to run well too: a 35-billion-parameter Qwen3 model reportedly works with as little as 8 GB of memory.
Features for developers and the curious
WiCi Infer comes with a model browser for finding and downloading supported language models, plus a built-in chat interface. For developers, the bigger deal is the OpenAI-compatible API. The app runs a local endpoint that's structured exactly like OpenAI's. Any application that lets you enter a custom model URL can then talk to the locally running model. WiCi names the coding assistants Claude Code and OpenCode as examples. According to the company, you don't have to change your existing workflow.
If you want more power, WiCi would like you to buy its hardware. WiCi One is a device that connects a full NVIDIA graphics card to compatible devices over Wi-Fi. It runs the same InferSSD engine and the same WiCi Infer interface. So the free app also doubles as an on-ramp to WiCi's hardware business.
What you need to try it
According to the source, WiCi Infer is currently available only for macOS. The report doesn't say whether Windows or Linux versions exist. That said, the benchmarks also ran on a DGX Spark, NVIDIA's compact AI computer. For smaller models like the 35B Qwen3 mentioned above, WiCi says 8 GB of memory is enough. For the 800B models, WiCi cites machines with 48 GB (MacBook Pro M5 Max) or 128 GB of unified memory.
Because experts are constantly read from the SSD, a fast drive is essential. You'll also need plenty of free space: even compressed, models this size take up several hundred gigabytes. The source doesn't say exactly how much. It also doesn't address whether all that reading wears out the SSD faster. In general, though, reads put far less strain on an SSD than writes.
Our take
WiCi Infer goes after a real problem. Almost all of the best open models are now MoE giants that won't fit in memory even on expensive workstations. Loading experts from the SSD on demand is an obvious and elegant approach. And because the engine is open source, the claims are easier to check.
The caveats remain. Every number comes from the company itself. Comparing against llama.cpp in a scenario where it's barely usable anyway makes the gap look especially big. Even with InferSSD, 800B models run at roughly two tokens per second. That's fine for experimenting and the odd hard question, but not for smooth day-to-day work. What matters more for everyday use is how well mid-sized MoE models perform on machines with little memory, and independent tests are still missing there. The business model is clear too: the free app is meant to pave the way to the paid WiCi One hardware.
Related videos
Sources
- Brave News androidguys.com
Ask the AI assistant
Questions about this article, installation or background? Our assistant knows this page and runs on our own machine, not in someone else's cloud.
The Assistant is an AI (Gemma 4) and can make mistakes – please check important details against the source. Questions are used only to generate an answer and are deleted after one hour. Please don't enter any personal data.
Comments
What do you think? Already tried it? Write it here, no sign-up needed.
Comments are public. Links aren't allowed, and we delete abusive posts. You can delete your own comment at any time in this browser. Learn more in the Privacy Policy.