Microsoft releases Aion 1.0 Instruct: the small model coming to Windows, MIT-licensed

Microsoft has published Aion 1.0 Instruct, a 2.3-billion-parameter model, on Hugging Face. By the end of January 2027, an optimized version is due to run as local AI built right into Windows. It handles long documents, understands German and several other languages, and its MIT license lets you use it with almost no restrictions.

Model releases Try it yourself Nov 18, 2026 6 min read Source: Reddit r/LocalLLaMA
Illustrative image, AI-generated

What's going on

Microsoft has released the language model Aion 1.0 Instruct on Hugging Face: microsoft/Aion-1.0-Instruct-2.3B. It was built by the Applied Sciences Group in Microsoft's Windows & Devices division. At around 2.3 billion parameters, it's a small language model (SLM), meaning it's deliberately kept compact. The idea is that it runs on your own machine rather than in a data center. Microsoft calls this "on-device AI."

Microsoft describes the Hugging Face release as a preview. A further optimized version, combined with Microsoft's content moderation model, is set to become Windows' built-in language model by the end of January 2027. It will replace Phi Silica, the current Windows language model. Microsoft says it's releasing early for the sake of transparency and is explicitly asking for feedback. The model is under the MIT license, so you can use it commercially, modify it and redistribute it.

What the model is for

Aion Instruct targets everyday text tasks: understanding documents, analyzing content, answering questions, summarizing and rewriting text, and chatting. Its context window, the amount of text the model can keep in view at once, is 32,768 tokens. Microsoft says that's enough for multi-page documents, long email threads or technical specs. And because everything runs on your device, your data never leaves your machine.

The model supports English, German, Spanish, Portuguese, Italian, French, Chinese and Japanese. That said, multilingual data makes up only about 6% of the training data, and Microsoft admits quality may vary by language. Aion also supports tool calling: it can output structured calls to external functions, such as a calendar or search function, which a program then executes.

Microsoft is upfront about the limits. The model wasn't trained to hold a lot of world knowledge. It may miss details, make false statements and struggle with complex, multi-step reasoning. It may also be vulnerable to deliberate manipulation attempts. It is explicitly not meant for medical diagnoses or legal decisions.

Benchmarks

Microsoft ran most of its tests with EleutherAI's lm-evaluation-harness, a widely used evaluation tool. Where a judge model was needed, GPT-4o filled that role. These are vendor-reported numbers:

Area Benchmark Score
Knowledge MMLU 55.15%
Knowledge MMLU-Pro 33.35%
Knowledge MMMLU (multilingual) 45.51%
Knowledge GPQA 28.18%
Math GSM8K 74.91%
Math GSMPlus 49.56%
Math MATH-500 48.00%
Instruction following IFEval 83.90%
Instruction following IFBench 32.00%
Instruction following Multi-IF 64.70%
Translation chrF++ 43.18

The model is strongest at following instructions (IFEval) and at basic grade-school math (GSM8K). It's much weaker on knowledge questions, which is what you'd expect at this size. The Model Card doesn't include direct comparisons with other models. According to ad-hoc-news, Microsoft said at Build 2026 that Aion is smaller and faster than Phi-4-mini.

Under the hood

The model has 28 layers. Of these, 21 use sliding-window attention: they only look at the most recent 2,048 tokens, a bit like a reader who mainly remembers the last few pages. The other 7 layers see the full text. This mix saves memory and compute on long inputs. On top of that comes grouped-query attention (GQA) with 24 query heads and 8 key-value heads. Several queries share one cache, which saves memory too.

The vocabulary has 200,029 tokens, and the knowledge cutoff is December 2025. The training data builds on Phi Silica's: heavily filtered public documents plus newly generated synthetic data, meaning AI-written training text, covering math, coding, common-sense reasoning and general knowledge.

Background: the Aion family

According to ad-hoc-news, Microsoft unveiled the Aion 1.0 family at its Build developer conference in June 2026. Alongside the CPU-optimized Instruct model, it includes Aion 1.0 Plan with 14 billion parameters. That one is meant to handle complex reasoning and tool calls in automated workflows. At the time, the open weights were announced for July 2026. They actually arrived only now, in November.

The report says Build also brought new Windows AI APIs for CPUs, GPUs and NPUs, the dedicated AI chips in newer laptops. There was also an isolated environment for local AI agents called Microsoft Execution Containers (MXC), plus the Surface RTX Spark Dev Box developer machine.

What you need to try it

So far, Hugging Face only hosts the uncompressed version in the bfloat16 number format. The file is roughly 4.5 to 5 GB. On a GPU, plan for about 6 GB of VRAM (video memory) for this version. With the announced quantized versions, which shrink the model by storing numbers at lower precision, about 3 GB should be enough at 4-bit.

Because the model is optimized for CPUs, it also runs without a GPU on a regular PC with 8 to 16 GB of RAM, just more slowly. A Mac with Apple Silicon or a free Google Colab notebook will also do.

A few things to keep in mind when you get started:

  • Microsoft ships its own model code, so you need to set trust_remote_code=True when loading.
  • The model was exported with Transformers 5.x, so you need a current version of this Python library.
  • There are no ready-made GGUF files yet for tools like llama.cpp or Ollama.

The bigger picture

Aion 1.0 Instruct isn't a frontier model, and Microsoft doesn't pretend it is. It's interesting for other reasons. Microsoft is opening up the model that's set to run on many millions of Windows PCs from 2027, and under a very permissive license at that. Developers can start building apps now that will later tap into Windows' built-in AI.

The long context window, German support and tool calling make the model useful for local summaries and simple assistant tasks. It competes with other small open models in the two-to-four-billion-parameter range, such as Microsoft's own Phi-4-mini.

Some questions remain open. There are no independent tests yet. With only 6% multilingual training data, it's unclear how good the German answers really are. And the Windows version will be further optimized and paired with a moderation model, so the Hugging Face release isn't exactly the final product.

Sources

#microsoft#windows#small language model#on-device#open weights
Read the original source

Try it yourself: Aion-1.0-Instruct-2.3B

Explained step by step, no prior knowledge needed – on your own computer or free in your browser.

What can I do with it?

Aion 1.0 Instruct is a small Microsoft language model for text tasks that runs right on your own computer. It summarizes documents, rewrites text, answers questions about text you give it and handles simple chats. With a 32,768-token context window, it can process long email threads or multi-page reports in one go, without any data going online. It understands eight languages, including German, but only about 6% of its training data is non-English, so English answers are often better. It's strong at following instructions: Microsoft reports 83.9% on IFEval. For knowledge questions and complex, multi-step reasoning, though, it's of limited use, and large models like ChatGPT are clearly ahead there. It can also generate function calls (tool calling), which makes it a fit for simple local assistants.

Who is it worth it for?

The model is worth a look if you want a fast, lightweight, privacy-friendly model for summarizing and rewriting. It's also interesting for developers who want to get ready for the upcoming Windows AI, and for tinkerers with modest hardware. If you need a model with lots of world knowledge, solid coding help or deep logical reasoning, go for a bigger model.

Ideas: What you can do with it

Summarize long email threads

Paste a long email thread into the chat and ask for the key points and open tasks. Thanks to the 32,768-token context, a lot of text fits in at once, and nothing leaves your computer.

Rewrite text

Have job applications, complaints or club newsletters made more polite, shorter or simpler. You provide the original text and the tone you want, and get a revised version in seconds.

Answer questions about documents

Paste in a lease, a user manual or a report and ask targeted questions about it. The answers are more useful when they're based on the pasted text than when the model has to answer from memory.

Local mini assistant with tools

Developers can hook up their own functions via tool calling, such as a notes database or a calendar. The model decides when to call a function and outputs the call in JSON format.

Get ready for Windows AI

If you build Windows software, you can already test how the future system language model behaves. An optimized version is due to ship directly in Windows by the end of January 2027.

Size2.3 B parameters
Download1.4 GB (about 2 min at 100 Mbit/s)
Licensemit
TypeLanguage model (text chat)

What your computer needs

Graphics card (NVIDIA) at least 3 GB VRAMalmost any graphics card with 4 GB or more (e.g. GTX 1650, RTX 3050)
Mac (Apple Silicon M1–M5) from 8 GB unified memoryThe GPU and CPU share the memory – the model can use about 70% of it
CPU only (no graphics card) Runs smoothly without a graphics cardan estimated 33 words per second on a current desktop PC. This requires at least 8 GB of RAM.
Storage approx. 3.4 GB freeModel download plus programs
Google Colab runs for freeThe free Tesla T4 GPU with 15 GB is enough

Values estimated from the model size – the Model Card may have more precise figures.

Which variant is right for me?

Many models come in several sizes or "quantizations." These are compressed versions: smaller and faster, but slightly less accurate. Rule of thumb: pick the largest variant whose memory requirement fits in your VRAM (video memory). The scripts below choose this for you automatically.

VariantDownloadNeeds approx.Compatible graphics cards
4-bit (Q4) Default 1.4 GB3 GBalmost any graphics card with 4 GB or more (e.g. GTX 1650, RTX 3050)
8-bit (Q8) 2.5 GB4 GBalmost any graphics card with 4 GB or more (e.g. GTX 1650, RTX 3050)
16-bit (Original) 4.6 GB6.5 GBe.g. RTX 3060 Ti, RTX 4060, RTX 5060, RX 7600

So far, only the uncompressed bfloat16 version is available, and the model file is roughly 4.5 to 5 GB. It runs on GPUs with about 6 GB of VRAM or on the CPU with 8 to 16 GB of RAM. Microsoft has announced compressed versions. If you're short on memory or want to use llama.cpp or Ollama, wait for those. A 4-bit variant should get by with around 3 GB.

  • The current bfloat16 version needs about 5 GB of storage. On a GPU, plan for about 6 GB of VRAM.
  • Microsoft says the model is optimized to run on CPUs, so it works without a GPU too, just more slowly.
  • Long inputs of up to 32,768 tokens cost extra memory. Thanks to hybrid sliding-window attention, that overhead is smaller than with many other models.
  • A Mac with Apple Silicon and at least 8 GB of unified memory, or a free Google Colab notebook, is enough.
  • Quantized versions that need less memory are on the way. There are no GGUF files for llama.cpp or Ollama yet.
How do I find out what my computer has?
  • Windows: Press Ctrl + Shift + Esc → Task Manager → Performance tab → click GPU on the left. At the bottom you'll see Dedicated GPU memory (that's your VRAM). You'll find system memory under Memory.
  • Mac: Apple menu at the top left → About This Mac. You'll see your chip (e.g. M2) and memory (e.g. 16 GB). On Apple chips, graphics and processor share this memory.
  • Linux: Type nvidia-smi in the terminal (shows your graphics card and its memory) and free -h for system memory.

Guide

Google Colab (in your browser, no installation)

Google Colab is a free service from Google. It gives you a computer with a graphics card in your browser for a few hours. All you need is a Google account. For this model: runs for free – The free Tesla T4 GPU with 15 GB is enough.

  1. Download the notebook. Click the button and save the file.

    Download colab.ipynb

  2. Open Colab. Go to colab.research.google.com and sign in with your Google account.
  3. Upload the notebook. In the window that appears, click Upload on the left (or choose File → Upload notebook at the top) and select the file colab.ipynb.
  4. Turn on the GPU. In the top menu, go to Runtime → Change runtime type → under Hardware accelerator, select T4 GPU → Save.
  5. Start everything. Runtime → Run all (keyboard shortcut Ctrl + F9). If you see the warning “This notebook was not authored by Google,” click Run anyway.
  6. Wait. The cells run one after another. A green checkmark appears to the left of each cell when it's done. Installation and download take about 4 minutes here.
  7. Try it out. The result appears below the last cell. To test something else, change the text in the cell and run it again with Shift + Enter.
  8. Clean up. When you're done: Runtime → Disconnect and delete runtime. This saves your free quota.

Good to know: The free GPU is limited (usually a few hours per day). After about 90 minutes of inactivity, Colab disconnects, and you'll have to start everything over.

Windows 10 / 11

This model runs on Python. Our script handles the setup automatically: it creates an isolated Python environment, installs the required packages (PyTorch, approx. 3 GB) and downloads the model (1.4 GB (about 2 min at 100 Mbit/s)).

  1. Update your graphics driver (NVIDIA): install the latest driver. Without an NVIDIA card, it runs on the CPU – much more slowly.
  2. Install Python. Download Python 3.12 from python.org and run the installer. Important: At the bottom of the first window, tick Add python.exe to PATH, then click Install Now.
  3. Create a folder and download the files. Create a folder, e.g. Documents\AI-Test, and save both files into it:

    try_windows.ps1 demo.py

  4. Open a terminal in the folder. Open the folder in File Explorer, right-click an empty spot → Open in Terminal.
  5. Run it. Paste this command and press Enter:
    powershell -ExecutionPolicy Bypass -File try_windows.ps1
    Installation and download are running now. The first time, this can take 10–30 minutes.
  6. Result. The window shows the model's answer to a test question.
  7. Try your own input: Open demo.py in an editor, change the sample text, and save. Then run this in the terminal:
    .\.venv-aion-1-0-instruct-2-3b\Scripts\Activate.ps1
    python demo.py

macOS

Macs with an Apple chip (M1 to M5) are ideal. Older Intel Macs are very slow.

  1. Open Terminal. Press Cmd + Space, type Terminal, and press Enter.
  2. Install the developer tools (first time only; includes Python):
    xcode-select --install
    Click Install in the window. If you see "already installed," everything is already in place.
  3. Download the files – both end up in your Downloads folder:

    try_mac.sh demo.py

  4. Run it:
    cd ~/Downloads
    bash try_mac.sh
    The first time, installation and download take 10–30 minutes.
  5. Result. The window shows the model's answer to a test question.

Linux

  1. Check your GPU. nvidia-smi should show your card. If it doesn't, install the driver on Ubuntu and restart:
    sudo ubuntu-drivers install
  2. Install the Python basics (Ubuntu/Debian):
    sudo apt update && sudo apt install -y python3-venv python3-pip git
  3. Put the files in a folder and open a terminal there:

    try_linux.sh demo.py

  4. Run it:
    bash try_linux.sh
  5. Result. The window shows the model's answer to a test question.

Try it

Summarize this text in three bullet points: [paste text]
Make this email more polite and shorter: "I've been waiting two weeks for your reply, this is unacceptable."
Here's a long email thread. What tasks are still open, and who is responsible for each? [paste thread]
Translate this paragraph into German, then explain in one sentence which parts were hard to translate: [paste paragraph]
Read this technical specification and create a table of all the requirements it lists and their limits: [paste text]

Tips for better results

  • Include the text you want the model to work on directly in your prompt. As Microsoft itself notes, it often answers world-knowledge questions inaccurately when there's no source text.
  • Give clear, single tasks like "summarize" or "rewrite" instead of nested multi-step assignments. Break complex tasks into several steps.
  • Use the chat template via `apply_chat_template`. It also lets you set the current date and your own system prompt.
  • If answers in other languages feel clunky, have the model work in English first and translate afterward. Multilingual data makes up only about 6% of the training data.
  • Double-check facts and figures in the answers. According to Microsoft, small models tend to make false statements.

If something goes wrong

Microsoft ships its own model code. The model won't load without `trust_remote_code=True`.

The model was exported with Transformers 5.x. Older versions of the library fail to load it, so run `pip install -U transformers accelerate` first.

The Model Card's quickstart uses the ID `asgmodels/Aion-Instruct-v1`. Use the official repo ID `microsoft/Aion-1.0-Instruct-2.3B` instead.

There are no GGUF or other quantized files yet, so Ollama and LM Studio don't work out of the box. Transformers or vLLM are your best bet for now.

Each conversation turn ends with the `<|end|>` token. If you build your own chat UI, use the bundled chat template, or the model may not stop generating.

“out of memory,” a crash, or your computer freezes

The model is too big for your VRAM (video memory). Pick a smaller variant from the table above and close other programs, especially games and browsers with lots of tabs.

It runs, but extremely slowly

Your CPU is probably doing the work instead of your graphics card. In Ollama, the PROCESSOR column of ollama ps shows how much runs on the GPU. If it says "100% CPU", update your graphics driver or pick a smaller variant that fits entirely into VRAM (video memory).

“ollama” or “python” is not found

Close the terminal window and open it again so it recognizes the newly installed program. For Python on Windows: reinstall it and check Add python.exe to PATH.

Windows: "Running scripts is disabled on this system"

Run the script with exactly the command above: powershell -ExecutionPolicy Bypass -File try_windows.ps1. This allows execution for this one time only.

Error message when loading the model (version, "trust_remote_code", unknown architecture)

Brand-new models often need the latest software version. With Ollama: update Ollama. With Python: pip install -U transformers in the activated environment. The Model Card usually lists the exact version.

Tokenstrom generates the scripts and notebook automatically. Please take a quick look before running them – they're plain, readable text. Mind the model's license: mit.

Ask the AI assistant

Questions about this article, installation or background? Our assistant knows this page and runs on our own machine, not in someone else's cloud.

Connecting …

The Assistant is an AI (Gemma 4) and can make mistakes – please check important details against the source. Questions are used only to generate an answer and are deleted after one hour. Please don't enter any personal data.

Comments

What do you think? Already tried it? Write it here, no sign-up needed.

  1. Loading comments …

Comments are public. Links aren't allowed, and we delete abusive posts. You can delete your own comment at any time in this browser. Learn more in the Privacy Policy.