Old PS4 turned AI box: Qwen3.5-9B runs offline at up to 39 tokens per second

An open-source project called Garlic LLM turns a decade-old PlayStation 4 into an offline chatbot. A hobbyist developer got the Qwen3.5-9B language model running directly on the console's GPU, along with speech recognition and text-to-speech. To pull it off, he wrote most of the GPU code himself.

Tools Nov 16, 2026 6 min read Source: Bing News (de-DE)
Illustrative image, AI-generated

What's going on

The PlayStation 4 launched in 2013 and is long outdated as hardware goes. Developer Leonardo Alegría got an AI chatbot running on it anyway. His open-source project, Garlic LLM, runs Alibaba's 9-billion-parameter Qwen3.5-9B language model directly on the graphics processor (GPU) of an ordinary PS4. There's no cloud server, no extra PC and no Linux install on the console. The developer says the console produces 31 to 39 tokens per second. Tokens are the word fragments a language model uses to build its answers. On programming tasks, he measured up to 43 tokens per second. That's faster than most people read.

The German gaming site Fragster first reported on the project. Garlic LLM installs as a homebrew package, meaning unofficial, self-made software that doesn't come through the PlayStation Store.

What the PS4 can do with it

Once installed, the console works as a chatbot. Garlic handles more than typed text. It also understands spoken input, reads its answers aloud, and can write Python code and run it right on the console. The project ships two extra models for voice. Nvidia's Parakeet TDT 0.6B v3 handles speech recognition, and Chatterbox handles text-to-speech (turning text into spoken audio). Supported languages include German, English, Spanish, French, Italian, Portuguese and Russian.

After installation, everything runs offline. Nothing connects to OpenAI, Google, Anthropic or Alibaba, and no data leaves the console.

The memory problem: 9 billion parameters in 8 GB

Memory was the biggest hurdle. The PS4 has 8 GB of GDDR5 in total, shared between the processor (CPU) and the GPU. The operating system and the Garlic software itself take a slice of that too. At normal precision, a 9-billion-parameter model would be far too big.

So Alegría relies on aggressive quantization. The model's numbers get stored at lower precision, a bit like saving a photo with fewer color shades to make the file smaller. Most values are stored at 4 bits. Some of the large MLP matrices go down to 3 bits. MLP matrices are the big compute blocks in each layer of the model, and they take up a large share of its memory. The result is a model of about 4.5 GiB.

Garlic still offers a context window of around 16,384 tokens. That's how much text the model can keep in view at once, including conversation history, prompts and answers. It's enough for longer conversations, translations or small coding tasks. Modern cloud models handle many times that. For 2013 hardware, it's still impressive.

GPU control written by hand

The most unusual part is how the project is built. None of the usual AI tooling exists on a PS4. There's no Nvidia CUDA, the programming interface most AI software is built on, and no standard machine learning runtimes either. So Alegría wrote his own GPU kernels in OpenCL C. Kernels are small programs that run directly on the GPU. Garlic also drives the console's AMD GPU, which is based on the GCN architecture, with hand-written PM4 command packets. These are very low-level commands that tell an AMD GPU directly what to do. On a regular PC, drivers and frameworks take care of this. Garlic largely bypasses all of that.

The speed trick: speculative decoding

Quantization alone doesn't explain the speed. The key is speculative decoding. Qwen3.5 has a built-in multi-token prediction head, an add-on component that proposes several likely next tokens at once. The main model then checks those guesses in a single pass instead of computing each token one at a time. When the guesses are right, that saves a lot of time. According to the developer, it pushed speed from about 22 to 31–39 tokens per second. Code benefits most because it often follows predictable patterns.

Metric Developer's figure
Model Qwen3.5-9B
Model size after quantization approx. 4.5 GiB
PS4 memory 8 GB GDDR5 (shared by CPU and GPU)
Speed without speculative decoding approx. 22 tokens/s
Speed with speculative decoding 31–39 tokens/s
Peak on coding tasks up to 43 tokens/s
Context window approx. 16,384 tokens
Install package approx. 5.66 GB

What you need to try it

Garlic isn't as easy as grabbing an app from the store. You need a modded PS4 running GoldHEN. GoldHEN is a homebrew enabler that unlocks unofficial software on compatible firmware versions. Garlic itself contains no exploit (no security hole) and doesn't bypass the console's protections on its own. You have to set up the jailbreak yourself first. The developer says the project has been tested on the standard PS4 and the PS4 Slim. The PS4 Pro hasn't been officially tested yet. You download the full install package, including Qwen3.5-9B and the voice models, from Hugging Face. It's about 5.66 GB. The source code is released under the free GPLv3 license. The report doesn't give an exact download link.

Sony doesn't support homebrew. If you mod your console, you do it at your own risk. So this isn't really for regular PlayStation owners. It's aimed at tinkerers and the homebrew scene.

The bigger picture

Above all, Garlic LLM is an impressive proof of concept. It shows how far optimization can now push local AI. A 9-billion-parameter model running at usable speed on a twelve-year-old console is more than a gimmick. Because Alegría skips all the usual frameworks and drives the GPU himself, the project is also interesting for developers who want to run AI on unusual or old hardware.

The limits are clear, though. Qwen3.5-9B is a relatively small model. It can make up facts or fail at hard tasks. The heavy 3- and 4-bit quantization likely costs some quality on top of that, though nobody has measured it yet. All the speed figures come from the developer and haven't been independently verified. So the PS4 won't replace a modern AI PC. But if you have an old console in a closet that's already modded, you get a fully private, voice-controlled offline assistant.

Sources

#local ai#qwen#playstation#homebrew#quantization
Read the original source

Ask the AI assistant

Questions about this article, installation or background? Our assistant knows this page and runs on our own machine, not in someone else's cloud.

Connecting …

The Assistant is an AI (Gemma 4) and can make mistakes – please check important details against the source. Questions are used only to generate an answer and are deleted after one hour. Please don't enter any personal data.

Comments

What do you think? Already tried it? Write it here, no sign-up needed.

  1. Loading comments …

Comments are public. Links aren't allowed, and we delete abusive posts. You can delete your own comment at any time in this browser. Learn more in the Privacy Policy.