Spark-X2.6-4B: Compact on-device agent model under Apache 2.0
The SparkLLM team (XHToken) has released Spark-X2.6-4B, a small language model with about 4 billion parameters built to run on your own device. It is said to be much better than its predecessor at coding, agent tasks and long context, and the maker claims it sometimes matches 9B models.
What this is about
The SparkLLM team has released the model Spark-X2.6-4B on Hugging Face. It is a compact, general-purpose language model with about 4.1 billion parameters (roughly speaking, parameters are the knobs the model tuned during training). It is meant to run directly on devices, with no big data center needed. The license is Apache 2.0, and the weights are free to download. English and Chinese are supported.
According to the model card, the model arrives a little over a month after Spark-X2.5. The maker stresses that the architecture is exactly the same as Spark-X2.5-4B. So the progress doesn't come from a new design but from improved post-training. That's the phase after the base training, when the model is tuned for tasks, tool use and response behavior.
What the model is supposed to do
The maker cites improvements in coding, agent tasks, long context, multilingual ability and reasoning. "Agents" here means workflows where the model calls tools, plans in multiple steps and, for example, completes software tasks. The model card says it reaches top scores among open-source models of similar size and sometimes matches or beats models with around 9 billion parameters.
One standout feature is the adjustable thinking budget. There are three levels: none, low and high. Applications can use it to trade off reasoning quality, response time and compute cost. By default, thinking is enabled through the chat template; you can turn it off per request with enable_thinking: false.
Benchmarks and efficiency
The model card compares the model with leading on-device models of similar size across agents, code, math, general knowledge and knowledge tasks. Qwen3.5-4B and Gemma-E4B are named as comparisons for inference efficiency. Spark-X2.6-4B is said to be on par or better there while also being more capable. The concrete numbers appear only in charts, which aren't available in the text. These are manufacturer claims only, with no independent verification.
Tech and where it runs
The model card lists many runtimes:
- Transformers (version 4.57.1 required,
trust_remote_code=Trueneeded) - SGLang and vLLM as servers with an OpenAI-compatible API, also for Huawei Ascend NPUs; for vLLM, a Spark plugin must be installed as well
- MLX via the Spark-MLX-LLM project for Apple Silicon, Linux CPU and NVIDIA CUDA
- llama.cpp (from build b10828), Ollama (from v0.34.1, tag
SparkLLM/Spark-X2.6-4B) and LM Studio (from 0.4.0) - Unsloth Studio for GGUF inference; Llama-Factory is recommended for fine-tuning
The SGLang example sets a context length of 1,048,576 tokens. The card says this needs a corresponding amount of memory, so lower the value if necessary. Recommended sampling values: temperature 1.0, top_p 0.95.
What you need to try it
A 4-billion-parameter model runs in 4-bit quantization (heavily compressed weights) with roughly 3–4 GB of VRAM (video memory). At full precision (bf16), expect about 8–9 GB. Both are estimates; the card gives no figures. Long context also eats memory, so choose shorter context lengths. About 10 GB of free disk space is enough for the weights. On a Mac with Apple Silicon, use MLX; without a GPU, use the CPU variant of Spark-MLX-LLM, though it's slower. Ollama and LM Studio are the easiest way in.
Our take
Small on-device models are in demand because they offer privacy and low costs. Spark-X2.6-4B goes up against Qwen3.5-4B and Gemma-E4B. The permissive Apache 2.0 license and broad tool support are handy. On the critical side: all performance claims come from the maker, and the numbers exist only as a chart. The model card gives "spark2_5" as the architecture name and requires trust_remote_code, which means third-party code is executed when you load it. Weigh that consciously. The card also mentions GGUF, yet the repository itself contains no GGUF files. Whether quality holds up in everyday use, especially in German, is an open question, since only English and Chinese are listed.
Sources
- Hugging Face Trending huggingface.co
- r/LocalLLaMA huggingface.co
- Hugging Face Trending huggingface.co
- Bing News (en-GB) intelligentliving.co
Try it yourself: Spark-X2.6-4B
Explained step by step, no prior knowledge needed – on your own computer or free in your browser.
What can I do with it?
You can run a chat assistant locally that helps with programming, calls tools and solves tasks in multiple steps. Thinking can be switched on or off to trade speed for quality.
Who is it worth it for?
Interesting for developers and tinkerers with a small GPU or a Mac who want to test a lean agent model. If you mainly need good German, be careful, since only English and Chinese are listed.
What your computer needs
Which variant is right for me?
Many models come in several sizes or "quantizations." These are compressed versions: smaller and faster, but slightly less accurate. Rule of thumb: pick the largest variant whose memory requirement fits in your VRAM (video memory). The scripts below choose this for you automatically.
| Variant | Download | Needs approx. | Compatible graphics cards |
|---|---|---|---|
Q4_K_M Default |
3.5 GB | 5 GB | e.g. RTX 2060, RTX 3050 8 GB, RX 6600 |
Q8_0 |
4.4 GB | 6 GB | e.g. RTX 2060, RTX 3050 8 GB, RX 6600 |
There is one 4B model. For small GPUs or laptops, a 4-bit GGUF variant via Ollama or LM Studio is the best bet; with more memory, bf16 works via Transformers, vLLM or SGLang.
- About 3–4 GB VRAM at 4-bit, roughly 9 GB at bf16 (estimate).
- Very long contexts (up to 1,048,576 tokens in the example) need much more memory; reduce --context-length.
- Apple Silicon runs via Spark-MLX-LLM, and Linux CPU is also supported.
- Huawei Ascend NPUs are supported too.
How do I find out what my computer has?
- Windows: Press Ctrl + Shift + Esc → Task Manager → Performance tab → click GPU on the left. At the bottom you'll see Dedicated GPU memory (that's your VRAM). You'll find system memory under Memory.
- Mac: Apple menu at the top left → About This Mac. You'll see your chip (e.g. M2) and memory (e.g. 16 GB). On Apple chips, graphics and processor share this memory.
- Linux: Type
nvidia-smiin the terminal (shows your graphics card and its memory) andfree -hfor system memory.
Guide
Google Colab (in your browser, no installation)
Google Colab is a free service from Google. It gives you a computer with a graphics card in your browser for a few hours. All you need is a Google account. For this model: runs for free – The free Tesla T4 GPU with 15 GB is enough.
- Download the notebook. Click the button and save the file.
- Open Colab. Go to colab.research.google.com and sign in with your Google account.
- Upload the notebook. In the window that appears, click Upload on the left (or choose File → Upload notebook at the top) and select the file
colab.ipynb. - Turn on the GPU. In the top menu, go to Runtime → Change runtime type → under Hardware accelerator, select T4 GPU → Save.
- Start everything. Runtime → Run all (keyboard shortcut Ctrl + F9). If you see the warning “This notebook was not authored by Google,” click Run anyway.
- Wait. The cells run one after another. A green checkmark appears to the left of each cell when it's done. Installation and download take about 5 minutes here.
- Try it out. The result appears below the last cell. To test something else, change the text in the cell and run it again with Shift + Enter. The last cell starts a proper chat with an input field.
- Clean up. When you're done: Runtime → Disconnect and delete runtime. This saves your free quota.
Good to know: The free GPU is limited (usually a few hours per day). After about 90 minutes of inactivity, Colab disconnects, and you'll have to start everything over.
Windows 10 / 11
The easiest way is Ollama, a free program that launches AI models with a single command.
- Update your graphics driver (NVIDIA cards only): Install the NVIDIA app or the latest driver. For AMD cards, use the latest Adrenalin driver.
- Install Ollama. Download the file OllamaSetup.exe from ollama.com/download/windows, double-click it, and choose Install. Afterward, a small llama icon appears in the taskbar at the bottom right.
- Open a terminal window. Press the Windows key, type PowerShell and press Enter. A blue or black window opens.
- Start the model. Copy this command ("Copy" button), paste it into the window with a right-click, and press Enter:
The first time, Ollama downloads the model (3.5 GB (about 5 min at 100 Mbit/s)). After that, it's instant.ollama run hf.co/XHToken/Spark-X2.6-4B-GGUF:Q4_K_M - Start typing. Once
>>> Send a messageappears, type your question and press Enter. Type/byeto end the chat.
Not enough memory? Pick a smaller variant from the table above, e.g. ollama run hf.co/XHToken/Spark-X2.6-4B-GGUF:Q4_K_M.
Prefer automatic? A script that picks the right variant for you
- Download both files into the same folder: try_windows.ps1 demo.py
- Open the folder in File Explorer, right-click an empty spot and choose Open in Terminal.
- Paste this command and press Enter:
powershell -ExecutionPolicy Bypass -File try_windows.ps1
If needed, the script installs Ollama via winget, measures your VRAM (video memory) and downloads the matching variant.
macOS
Macs with an Apple chip (M1 to M5) are ideal. Older Intel Macs are very slow.
- Install Ollama. Download it from ollama.com/download/mac, open the file, and drag Ollama into your Applications folder. Launch Ollama once – a llama appears in the menu bar at the top.
- Open Terminal. Press Cmd + Space, type Terminal, and press Enter.
- Start the model. Paste the command (Cmd + V) and press Enter:
The first time, the model is downloaded (3.5 GB (about 5 min at 100 Mbit/s)).ollama run hf.co/XHToken/Spark-X2.6-4B-GGUF:Q4_K_M - Start typing. Type your question and press Enter. Quit with
/bye.
Linux
- Check your GPU.
nvidia-smishould show your card. If it doesn't, install the driver on Ubuntu and restart:sudo ubuntu-drivers install - Install Ollama:
curl -fsSL https://ollama.com/install.sh | sh - Start the model:
Exit withollama run hf.co/XHToken/Spark-X2.6-4B-GGUF:Q4_K_M/bye. Useollama psto check whether the model is running on the GPU. - Or let it pick the right variant automatically:
bash try_linux.sh # mit Python/transformers statt Ollama: MODE=hf bash try_linux.sh
Try it
Write a Python function that removes duplicate rows from a CSV file, and briefly explain it.Lay out step by step how I can test a small Flask app with a login.Summarize this long text in five bullet points, and think it through thoroughly.If something goes wrong
Transformers must be installed in exactly version 4.57.1, and trust_remote_code=True must be set.
llama.cpp needs build b10828 or newer, Ollama v0.34.1+, LM Studio 0.4.0+.
For vLLM, you must install the Spark plugin from the GitHub repo.
The example value max_new_tokens=131072 and the huge context length can blow up your memory; choose smaller values.
“out of memory,” a crash, or your computer freezes
The model is too big for your VRAM (video memory). Pick a smaller variant from the table above and close other programs, especially games and browsers with lots of tabs.
It runs, but extremely slowly
Your CPU is probably doing the work instead of your graphics card. In Ollama, the PROCESSOR column of ollama ps shows how much runs on the GPU. If it says "100% CPU", update your graphics driver or pick a smaller variant that fits entirely into VRAM (video memory).
“ollama” or “python” is not found
Close the terminal window and open it again so it recognizes the newly installed program. For Python on Windows: reinstall it and check Add python.exe to PATH.
Windows: "Running scripts is disabled on this system"
Run the script with exactly the command above: powershell -ExecutionPolicy Bypass -File try_windows.ps1. This allows execution for this one time only.
Error message when loading the model (version, "trust_remote_code", unknown architecture)
Brand-new models often need the latest software version. With Ollama: update Ollama. With Python: pip install -U transformers in the activated environment. The Model Card usually lists the exact version.
Tokenstrom generates the scripts and notebook automatically. Please take a quick look before running them – they're plain, readable text. Mind the model's license: apache-2.0.
Ask the AI assistant
Questions about this article, installation or background? Our assistant knows this page and runs on our own machine, not in someone else's cloud.
The Assistant is an AI (Gemma 4) and can make mistakes – please check important details against the source. Questions are used only to generate an answer and are deleted after one hour. Please don't enter any personal data.
Comments
What do you think? Already tried it? Write it here, no sign-up needed.
Comments are public. Links aren't allowed, and we delete abusive posts. You can delete your own comment at any time in this browser. Learn more in the Privacy Policy.