Unsloth and AMD finally put local LLMs to work on the Ryzen AI NPU
The open-source team behind Unsloth has worked with AMD to run local language models on the AI accelerator (NPU) built into modern Ryzen processors. Until now, chatbots almost never used that part of the chip. The companies say it draws two to four times less power than a GPU.

What's going on
The team behind Unsloth, a popular open-source tool for fine-tuning and running language models, has shipped a new feature together with AMD: local AI models now run on the NPU of current AMD Ryzen processors. An NPU (Neural Processing Unit) is a dedicated block inside the processor, built specifically for AI workloads. Think of it as a specialist sitting next to the generalists, the CPU and the graphics unit. Unsloth announced the news on November 19, 2026, on the r/LocalLLM subreddit. Two other projects are involved: Lemonade and FastFlowLM, which handle the actual execution on the NPU.
AMD published two companion posts: an announcement in the AMD Newsroom and a technical article on the AMD developer blog that covers the desktop version of Unsloth.
What NPU support can do
According to Unsloth, you can now run language models directly on the NPU, with the feature set you'd expect from a modern AI assistant:
- Streaming responses: Text appears word by word instead of you waiting for the full answer.
- Tools and skills: The model can call tools and use predefined capabilities.
- MCP servers: The Model Context Protocol is an open standard for connecting AI models to external services and data sources, a kind of universal plug for AI tools.
- Web search and chat with local documents: You can hand the model your own files and ask questions about them without anything leaving your machine.
- Coding with local files: The model can work directly with code on your own computer.
The announcement also says you can hook up compatible apps and scripts, which then hand their AI requests off to the NPU. That's meant to free up memory and compute on the rest of the chip, for example keeping the graphics unit available for gaming, video editing or a second, larger model.
Why power draw is the big deal
Unsloth's main pitch is efficiency: NPUs are said to use two to four times less power than conventional GPUs at high compute loads. That's the vendors' own claim; the sources don't include independent measurements yet. For laptops, it still matters. If you run a local assistant on the go, an NPU puts far less strain on your battery, and the fan is more likely to stay quiet.
The Reddit announcement doesn't say how fast the models generate text on the NPU or which model sizes run well. There are no tokens-per-second comparisons with the integrated graphics. So it's still unclear whether the NPU can keep up on speed or mainly wins on power consumption.
Which chips are supported
The announcement names AMD's Strix, Gorgon and Krackan platforms plus others, as well as future AMD chips. These are code names for recent generations of Ryzen AI processors, found mostly in laptops and mini PCs. The Strix family also includes the powerful Ryzen AI Max+ 395 ("Strix Halo"). According to Techtimes, it combines 16 Zen 5 cores, an RDNA 3.5 graphics unit with 40 compute units and an XDNA 2 NPU on a single chip, all sharing up to 128 GB of LPDDR5x memory.
Until now, the NPU mostly sat idle
What makes this news interesting is that the NPU has barely mattered for language models so far. As recently as June 2026, Techtimes made it clear in a review of the Ryzen AI Max+ 395 that the advertised NPU performance of more than 50 TOPS (trillion operations per second) does not apply to language models. Popular tools like Ollama, llama.cpp and LM Studio sent LLM workloads to the graphics unit, not the NPU. Until then, the NPU mainly accelerated narrowly defined tasks such as video upscaling or image recognition. Techtimes explicitly advised buyers not to go by NPU TOPS when picking an AI PC for language models.
A hands-on test by Digital Reviews Network shows the same thing. The author used Unsloth Studio on an HP ZBook Ultra G1a with 64 GB of shared memory to train a model on her own writing style. While fine-tuning a Qwen3-4B model with QLoRA (the base model stays frozen and only small add-on modules are trained), the Radeon graphics unit ran at 95 to 100 percent utilization, while the NPU was "effectively idle." The training run took just under eleven minutes.
With the new support, the NPU finally has something to do, at least for inference, meaning running a finished model. Based on what's known so far, training still runs on the graphics unit.
Real-world experience with Unsloth on AMD
The Digital Reviews Network test also shows where Unsloth still stumbles on AMD machines. On Windows, the Unsloth Studio desktop installer initially failed because of a broken path that literally contained "$HOME". Installing via PowerShell with irm https://unsloth.ai/install.ps1 | iex worked, though. It correctly detected the integrated Radeon 8050S as "gfx1151" and automatically chose the right AMD/ROCm install path. ROCm is AMD's software platform for GPU compute, comparable to Nvidia's CUDA.
A second pitfall: with the larger Qwen3.8-27B, the auto setting for the context window (how much text the model can keep in view at once) silently picked the full 262,144 tokens. That pushed the estimated memory requirement to around 39.5 GiB, almost 19 GiB of it for the KV cache (the model's short-term memory during a conversation), and the model wouldn't start. Set manually to 65,536 or 131,072 tokens, it ran without problems. As the author puts it, the software often sets the practical limits of the hardware just as much as the hardware itself does.
The bigger picture
If you own a current Ryzen AI laptop or mini PC, this is a useful upgrade: a part of the chip you paid for, which chatbots have ignored until now, finally gets used. A power-efficient local assistant with document chat, web search and MCP support that leaves the graphics unit free is especially appealing on laptops.
But a lot is still unclear. The "two to four times more efficient" claim comes from the companies themselves. The announcement doesn't include figures on speed, supported model sizes or memory limits on the NPU. NPUs have so far been seen as better suited to smaller models. Independent tests will have to show whether large models run well on the NPU or still need the graphics unit. It's also unclear when tools like Ollama or LM Studio will follow. Until then, NPU support only works through Unsloth and its partner projects Lemonade and FastFlowLM. In the race against Apple, whose M-series chips have used their Neural Engine for a while, and against Intel and Qualcomm, which also put NPUs in laptops, AMD is closing some of the gap on the software side.
Sources
- Reddit r/LocalLLM reddit.com
- techtimes.com techtimes.com
- digitalreviews.net digitalreviews.net
Ask the AI assistant
Questions about this article, installation or background? Our assistant knows this page and runs on our own machine, not in someone else's cloud.
The Assistant is an AI (Gemma 4) and can make mistakes – please check important details against the source. Questions are used only to generate an answer and are deleted after one hour. Please don't enter any personal data.
Comments
What do you think? Already tried it? Write it here, no sign-up needed.
Comments are public. Links aren't allowed, and we delete abusive posts. You can delete your own comment at any time in this browser. Learn more in the Privacy Policy.