Axion 1.0: ByteDance shows three training stages are enough for strong AI agents
A team from ByteDance and several US universities used just two rounds of fine-tuning and one round of reinforcement learning to train an open agent that keeps pace with big commercial models on science benchmarks. What sets it apart: the data, prompts and full training code are open along with the model.

What it's about
AI models are no longer expected to just answer questions. They're supposed to research the web on their own, run code and work through multi-step tasks. These are called "agentic" capabilities. A new research paper argues that the training pipelines behind them keep getting more convoluted. Many developers stack training phase on top of training phase, so it's hard to tell which step actually helps, and reproducing the results gets expensive.
A team from ByteDance (TikTok's parent company), Duke University, the University of Maryland and the University at Buffalo is pushing back. With Axion 1.0, the researchers want to show that three steps are enough. As The Decoder reports, they back that claim with a competitive open model. According to explainx.ai, the paper carries the arXiv number 2611.03233. The team also released code, training data and a 35-billion-parameter checkpoint.
The models and what was released
Axion 1.0 is a model family in two sizes, both built as Mixture-of-Experts (MoE). The model is made up of many sub-networks ("experts"), and only a few of them do any work for each token. That makes it faster than its total size would suggest:
| Model | Total parameters | Active per token | Base model |
|---|---|---|---|
| Axion 1.0-35B | 35B | 3B | Qwen3.6-35B-A3B |
| Axion 1.0-122B | 122B | 10B | Qwen3.5-122B-A10B |
So Axion wasn't trained from scratch. It's post-training: extra training layered on top of existing Qwen models from Alibaba.
According to explainx.ai, the release has four parts:
- Paper and project page with the recipe and tables covering each training stage. When the report was written, it ranked third among the daily papers on the Hugging Face papers page.
- Code in the bytedance/axion1_0 repository: data generation and filtering, prompts, two-stage fine-tuning, asynchronous reinforcement learning, plus benchmark preparation and evaluation. At the time, the repo had 19 stars and 36 commits, so it's an early release without an established community.
- Training data as the "axion-training-data" dataset in the Axion1-0 Hugging Face organization.
- A 35B checkpoint in BF16 with about 35.1 billion parameters under the Apache 2.0 license.
One important detail: the README says the 35B checkpoint is "a community release, not an official Axion checkpoint." explainx.ai found no download link for 122B weights. If you want to use that model, you'll have to rebuild it yourself from the training instructions.
The recipe: learn to think, learn to research, learn through interaction
The authors argue that complexity belongs in the data, not in a long chain of training tricks. They deliberately skipped mid-training (an intermediate training phase on large amounts of data), DPO (learning from pairs of preferred and rejected answers) and on-policy distillation (learning from a stronger teacher model based on the student's own answers).
Stage I – reasoning on its own: 10,819 solution traces from science and math. The team takes public problems and rewrites them to be harder. If the base model can already solve a problem, it gets harder still, for example by combining several problems, changing the number ranges or adding case distinctions. The reference answer is checked after every rewrite. The paper calls this "model-aware data construction": difficulty is matched to what the base model can already do. Python is allowed here, web search is not.
Stage II – reasoning with tools: 7,544 solution traces totaling about 312 million tokens. Here the pipeline works backwards. Programs filter records from structured databases, pick a single unambiguous entity and compute the answer. Only then does a language model write the matching question. The model being trained has to find the evidence itself using tools. Any question that can be answered without research gets thrown out. The README lists the NASA Exoplanet Archive, OurAirports, OpenAlex, UniProt and Ensembl as sources. That way every task has a single verifiable answer, which is exactly what the next stage needs.
Stage III – reinforcement learning: Starting from the Stage II model, the team runs asynchronous GRPO. The model generates several attempts at a solution and gets rewarded for the better ones. "Asynchronous" means generation and training run in parallel. During this stage the model works with search, web page fetching, scientific literature search and Python. A correct final answer earns the full reward, and verifiably correct intermediate steps earn partial credit. Long runs are split into sessions, and the history so far gets compressed. There are two technical tricks. "Token-In-Token-Out" makes sure generation and training use exactly the same token IDs. "Rollout Routing Replay" makes sure the MoE model picks the same experts in both phases.
Benchmarks: strong, but with footnotes
The authors compare Axion with open and closed models. Here's an excerpt from the README table (asterisk: measured by the Axion authors themselves, otherwise official figures; "–" = not reported):
| Model | HLE | BMB | SEAL-0 | HiPhO | FS Olympiad | FS Research |
|---|---|---|---|---|---|---|
| Agents-A1 | 47.6 | – | 56.4 | 46.4 | 79.0 | 40.0 |
| Apodex 1.1 (Agent Team) | 56.1 | 35.3 | – | – | – | 63.3 |
| GPT-5.6 Sol | – | 47.1* | 53.2* | 42.9* | 79.0* | 63.3* |
| Gemini 3.1 Pro | 51.4 | 35.3* | 53.2* | 38.5* | 85.0* | 26.7* |
| Claude Fable 5 | 63.8 | 46.1 | – | – | – | – |
| Axion 1.0-35B | 52.2 | 35.3 | 62.2 | 45.7 | 86.0 | 56.7 |
| Axion 1.0-122B | 59.0 | 41.7 | 58.6 | 46.2 | 87.0 | 66.7 |
HLE stands for Humanity's Last Exam, a collection of extremely hard expert questions. BMB is the human-hard subset of BioMysteryBench, and FS stands for FrontierScience. According to the Model Card, Claude Opus 5 also scores 63.6 on HLE, GPT-6 Astra 57.2 and Muse-Spark-1.0 58.0.
explainx.ai reads the table soberly. Axion leads where there's data to compare: the 122B model on FrontierScience Olympiad and Research, the 35B model on SEAL-0. On HLE and BioMysteryBench, though, Claude models and several closed systems come out ahead. The comparison is also uneven. Many cells are empty, some scores were measured by the Axion authors themselves and others come from official reports. The project page itself warns that test conditions may differ. The HLE score of 59.0 was achieved "with restricted web page access." That matters, because HLE answers can otherwise sometimes be found online.
There's another catch. According to the project page, each Axion score is the best of three runs (best of five for BMB), not an average. The community rebuild of the 35B model is encouraging, though. According to the Model Card, it scores 52.8 (HLE), 35.3 (BMB), 60.4 (SEAL-0), 45.9 (HiPhO), 85.0 (FS Olympiad) and 55.0 (FS Research). That's close to the paper's numbers, even if it's just one reproduction.
What the training cost
According to the authors, both fine-tuning stages ran on 32 GPUs:
| Model | Stage I | Stage II | Total GPU hours |
|---|---|---|---|
| Axion 1.0-35B | 1 h 02 min | 1 h 46 min | approx. 90 |
| Axion 1.0-122B | 3 h 32 min | 3 h 58 min | 240 |
On top of that comes the reinforcement learning: about 15 hours for the 122B model (40 GPUs for rollouts, 16 for training, 40 steps) and about 30 hours for the 35B model (8 GPUs each, 20 steps). These figures don't include the cost of building the data, which takes a lot of calls to large language models. The README also flags a small discrepancy: the published 122B recipe lists a micro-batch size of 1, while the paper says 2.
What you need to try it
Only the 35B checkpoint has been released, and only in BF16. At that full precision it takes up about 70 GB of memory. With 4-bit quantization (shrinking the model by storing its numbers at lower precision), you'd need roughly 22–24 GB of VRAM. That means a card like the RTX 3090/4090, or a Mac with at least 32 GB of unified memory. The sources don't mention any ready-made GGUF files. Since only 3 billion parameters are active per token, offloading some experts to system RAM is also an option, but there are no concrete numbers for it. Plan on about 70 GB of disk space for the original. One important point: Axion only shows its strengths when it has tools like web search and Python. A plain chat interface doesn't give it those.
The bigger picture
Open weights are common these days. Open recipes aren't. Most releases ship a checkpoint and a model description and keep the data pipeline secret. Axion opens up its generators, filters, prompts and evaluation code, and that's where the real value lies. Other teams can now check whether the "three stages are enough" claim holds up. The low compute cost of the fine-tuning makes that realistic even for universities.
There are still limits. Much of the performance comes from the Qwen base models. The benchmarks cover scientific reasoning and information seeking, but they say little about coding agents, long computer-use sessions or product workflows. The comparison table is self-reported, has gaps and uses best scores instead of averages. And the only downloadable checkpoint is explicitly not an official one. As a research contribution, Axion is still notable. It suggests that carefully built, verifiable data does more than ever-longer training pipelines.
Sources
- The Decoder (EN) the-decoder.com
- explainx.ai explainx.ai
- ad-hoc-news.de ad-hoc-news.de
Try it yourself: Axion-1.0-35B
Explained step by step, no prior knowledge needed – on your own computer or free in your browser.
What can I do with it?
Axion 1.0-35B is trained for scientific reasoning and independent research. It solves hard math and science problems, and if you give it tools, it can search the web, read web pages, query the scientific literature and run Python code. In the authors' benchmarks, the 35B version beats every model in the comparison on SEAL-0 and scores 86 on FrontierScience Olympiad. On Humanity's Last Exam, though, its 52.2 puts it well behind frontier models like Claude Opus 5. Since only 3 billion parameters are active per token, it responds fairly quickly. It has limits: coding agents, computer use and creative writing weren't tested, and without tools hooked up, much of its agentic ability is gone. The checkpoint is a community rebuild, not an official release.
Who is it worth it for?
It's interesting for anyone experimenting with local research agents, and for researchers and students in math and science who have a 24 GB GPU or a high-memory Mac. If you just want a general chat assistant or coding help, you're probably better off with the Qwen base model or another all-rounder.
Ideas: What you can do with it
Enter a tough problem from school or university and have the model explain the solution step by step. It can check intermediate results with Python, which cuts down on calculation errors.
Connect the model to web search and ask specific factual questions, for example about airports, planets or scientific databases. You get an answer backed by the evidence it found, and your queries never go to a cloud chatbot.
Use Olympiad problems for practice and compare your solution with the model's. The authors report high scores on FrontierScience Olympiad.
Have the model use literature search to find studies on a topic and summarize them briefly. Be sure to check the sources yourself afterwards.
With the open code, data and evaluation scripts, you can apply the training recipe to your own field. The authors say fine-tuning the 35B model took only about 90 GPU hours.
What your computer needs
Which variant is right for me?
Many models come in several sizes or "quantizations." These are compressed versions: smaller and faster, but slightly less accurate. Rule of thumb: pick the largest variant whose memory requirement fits in your VRAM (video memory). The scripts below choose this for you automatically.
| Variant | Download | Needs approx. | Compatible graphics cards |
|---|---|---|---|
Q3_K_M |
17 GB | 20 GB | e.g. RTX 3090, RTX 4090, Tesla P40 |
Q4_K_M Default |
21 GB | 24 GB | e.g. RTX 5090 |
Q5_K_M |
25 GB | 28 GB | e.g. RTX 5090 |
Q6_K |
28 GB | 32 GB | e.g. 2× RTX 3090/4090 or RTX A6000 |
BF16 |
69 GB | 78 GB | e.g. multiple professional GPUs (2× A6000, H100) or DGX Spark |
Only the 35B version (3B active) is available to download, in BF16 at about 70 GB. At full precision it only runs on machines with a lot of VRAM. On regular hardware, 4-bit quantization makes sense (roughly 22–24 GB), but according to the sources none is officially available, so you may have to wait for the community. The 122B version isn't available to download. You'd have to retrain it yourself.
- The original BF16 checkpoint takes up about 70 GB of memory.
- With 4-bit quantization, roughly 22–24 GB of VRAM, e.g. an RTX 3090/4090 or a Mac with at least 32 GB of unified memory.
- Only 3B active parameters per token: offloading experts to system RAM is possible in principle, but there are no official numbers.
- Long agent runs with lots of search results need a large context, which costs extra memory.
- Plan on about 70 GB of free disk space for the download.
How do I find out what my computer has?
- Windows: Press Ctrl + Shift + Esc → Task Manager → Performance tab → click GPU on the left. At the bottom you'll see Dedicated GPU memory (that's your VRAM). You'll find system memory under Memory.
- Mac: Apple menu at the top left → About This Mac. You'll see your chip (e.g. M2) and memory (e.g. 16 GB). On Apple chips, graphics and processor share this memory.
- Linux: Type
nvidia-smiin the terminal (shows your graphics card and its memory) andfree -hfor system memory.
Guide
Google Colab (in your browser, no installation)
Google Colab is a free service from Google. It gives you a computer with a graphics card in your browser for a few hours. All you need is a Google account. For this model: Colab Pro only – needs the L4 GPU (22.5 GB); the free T4 is too small.
- Download the notebook. Click the button and save the file.
- Open Colab. Go to colab.research.google.com and sign in with your Google account.
- Upload the notebook. In the window that appears, click Upload on the left (or choose File → Upload notebook at the top) and select the file
colab.ipynb. - Turn on the GPU. In the top menu, go to Runtime → Change runtime type → under Hardware accelerator, select T4 GPU (with Colab Pro: L4 or A100) → Save.
- Start everything. Runtime → Run all (keyboard shortcut Ctrl + F9). If you see the warning “This notebook was not authored by Google,” click Run anyway.
- Wait. The cells run one after another. A green checkmark appears to the left of each cell when it's done. Installation and download take about 11 minutes here.
- Try it out. The result appears below the last cell. To test something else, change the text in the cell and run it again with Shift + Enter. The last cell starts a proper chat with an input field.
- Clean up. When you're done: Runtime → Disconnect and delete runtime. This saves your free quota.
Good to know: The free GPU is limited (usually a few hours per day). After about 90 minutes of inactivity, Colab disconnects, and you'll have to start everything over.
Windows 10 / 11
The easiest way is Ollama, a free program that launches AI models with a single command.
- Update your graphics driver (NVIDIA cards only): Install the NVIDIA app or the latest driver. For AMD cards, use the latest Adrenalin driver.
- Install Ollama. Download the file OllamaSetup.exe from ollama.com/download/windows, double-click it, and choose Install. Afterward, a small llama icon appears in the taskbar at the bottom right.
- Open a terminal window. Press the Windows key, type PowerShell and press Enter. A blue or black window opens.
- Start the model. Copy this command ("Copy" button), paste it into the window with a right-click, and press Enter:
The first time, Ollama downloads the model (21 GB (about 28 min at 100 Mbit/s)). After that, it's instant.ollama run hf.co/prithivMLmods/Axion-1.0-35B-GGUF:Q4_K_M - Start typing. Once
>>> Send a messageappears, type your question and press Enter. Type/byeto end the chat.
Not enough memory? Pick a smaller variant from the table above, e.g. ollama run hf.co/prithivMLmods/Axion-1.0-35B-GGUF:Q3_K_M.
Prefer automatic? A script that picks the right variant for you
- Download both files into the same folder: try_windows.ps1 demo.py
- Open the folder in File Explorer, right-click an empty spot and choose Open in Terminal.
- Paste this command and press Enter:
powershell -ExecutionPolicy Bypass -File try_windows.ps1
If needed, the script installs Ollama via winget, measures your VRAM (video memory) and downloads the matching variant.
macOS
Macs with an Apple chip (M1 to M5) are ideal. Older Intel Macs are very slow.
- Install Ollama. Download it from ollama.com/download/mac, open the file, and drag Ollama into your Applications folder. Launch Ollama once – a llama appears in the menu bar at the top.
- Open Terminal. Press Cmd + Space, type Terminal, and press Enter.
- Start the model. Paste the command (Cmd + V) and press Enter:
The first time, the model is downloaded (21 GB (about 28 min at 100 Mbit/s)).ollama run hf.co/prithivMLmods/Axion-1.0-35B-GGUF:Q4_K_M - Start typing. Type your question and press Enter. Quit with
/bye.
Linux
- Check your GPU.
nvidia-smishould show your card. If it doesn't, install the driver on Ubuntu and restart:sudo ubuntu-drivers install - Install Ollama:
curl -fsSL https://ollama.com/install.sh | sh - Start the model:
Exit withollama run hf.co/prithivMLmods/Axion-1.0-35B-GGUF:Q4_K_M/bye. Useollama psto check whether the model is running on the GPU. - Or let it pick the right variant automatically:
bash try_linux.sh # mit Python/transformers statt Ollama: MODE=hf bash try_linux.sh
Try it
Calculate the probability of rolling doubles at least once in five rolls of two dice, and explain each step.An exoplanet orbits its star in 3.5 days, and the star has 0.9 solar masses. What is the semi-major axis of the orbit? Verify it with Python.Explain the difference between mitosis and meiosis so a ninth-grader can understand it, and give a typical exam example for each.Research (using web search) which airport in Iceland has the longest runway, and cite your source.Use literature search to find three recent studies on a specific protein in UniProt and summarize their key findings.
Tips for better results
- For research tasks, run the model in an agent environment with search and Python tools, e.g. using the scripts in the bytedance/axion1_0 repository.
- For math and physics problems, explicitly allow it to use Python to check its work. That's how it was trained in Stage I.
- Phrase tasks so they have a single, verifiable answer. That's what the training targets.
- For long research runs, set a generous context window, but keep the extra memory cost in mind.
If something goes wrong
According to the README, the 35B checkpoint is a community release, not an official Axion checkpoint, so results may differ slightly.
The architecture is tagged qwen3_5_moe. Older versions of Transformers, vLLM or llama.cpp may not support it, so use current versions.
The sources don't mention ready-made GGUF files. For Ollama/LM Studio you may have to wait for community conversions or convert it yourself.
Without tools hooked up (web search, Python), the model behaves like a regular chat model. The agent benchmarks assume a tool environment.
The 122B version isn't available to download.
“out of memory,” a crash, or your computer freezes
The model is too big for your VRAM (video memory). Pick a smaller variant from the table above and close other programs, especially games and browsers with lots of tabs.
It runs, but extremely slowly
Your CPU is probably doing the work instead of your graphics card. In Ollama, the PROCESSOR column of ollama ps shows how much runs on the GPU. If it says "100% CPU", update your graphics driver or pick a smaller variant that fits entirely into VRAM (video memory).
“ollama” or “python” is not found
Close the terminal window and open it again so it recognizes the newly installed program. For Python on Windows: reinstall it and check Add python.exe to PATH.
Windows: "Running scripts is disabled on this system"
Run the script with exactly the command above: powershell -ExecutionPolicy Bypass -File try_windows.ps1. This allows execution for this one time only.
Error message when loading the model (version, "trust_remote_code", unknown architecture)
Brand-new models often need the latest software version. With Ollama: update Ollama. With Python: pip install -U transformers in the activated environment. The Model Card usually lists the exact version.
Tokenstrom generates the scripts and notebook automatically. Please take a quick look before running them – they're plain, readable text. Mind the model's license: apache-2.0.
Ask the AI assistant
Questions about this article, installation or background? Our assistant knows this page and runs on our own machine, not in someone else's cloud.
The Assistant is an AI (Gemma 4) and can make mistakes – please check important details against the source. Questions are used only to generate an answer and are deleted after one hour. Please don't enter any personal data.
Comments
What do you think? Already tried it? Write it here, no sign-up needed.
Comments are public. Links aren't allowed, and we delete abusive posts. You can delete your own comment at any time in this browser. Learn more in the Privacy Policy.