New AI index: open models lose 12–16 points, Mistral Large 4 drops to 25

What this is about

Benchmark provider Artificial Analysis has moved its Intelligence Index to version 5. The index combines many individual tests into a single overall score. According to ActuIA's research (as of October 28, 2026), the best model with downloadable weights, meaning freely available "open-weights" models, now scores 32 points. Anthropic's Claude Opus 5.5 scores 58. In the previous version, the gap was 12 points. The models themselves haven't changed, only the yardstick.

Claude Haiku 5.5, a closed Anthropic model, scores 36 points and also sits ahead of every open model. In the old version, MiMo-V2.6-Pro (Xiaomi), DeepSeek V4.1 Flash and GLM-5.3 all scored above 44.

What changed in the index

According to Artificial Analysis, 60% of the index now rests on evaluations whose test sets are private or whose answers are kept secret. The index has four groups, each weighted at 25%: knowledge work, coding, science and general knowledge.

• Named as private: FrontierCode 2.0 (a coding benchmark launched by Cognition, whose tasks "stay private to avoid contamination") and 40 hard questions from CritPt v2 (physics).
• AA-Omniscience v1.1 has been cleaned of its public questions.
• Three tests are dropped: GDPval-AA, SciCode and AA-LCR.
• Two are added: FrontierCode 2.0 and Terminal-Bench-Science (scientific tasks in the terminal).
• HLE-Diamond replaces Humanity's Last Exam.

The provider doesn't say exactly what makes up the 60%. On September 4, Artificial Analysis had already raised the private share to 40% with version 4.2, to stop models from being tuned specifically for the ranking. The provider mentions contamination, meaning test questions ending up in training data, only in connection with FrontierCode 2.0.

All open models lose heavily

ActuIA compared the open-model page (still on version 4.3.2) with the new ranking:

Kimi K3.1 Pro from Moonshot AI, whose weights have not been released, drops from 55 to 46 (−9). The best closed models stay between 50 and 58: Claude Opus 5.5 (58), GPT-6 Astra (57), GPT-6.1 Sol (55), Claude Sonnet 5.5 (54), GPT-6.1 Terra (51), Gemini 4 Argon (50). Opus 5.5 already scored 58 on October 23 and keeps that score in version 5.

Artificial Analysis doesn't explain why the open models fall so far. Possible factors include the new weighting (science counts for a quarter; on Terminal-Bench-Science, GPT-6 Astra completes 63% of tasks, DeepSeek V4.1 Flash 13%) and the removal of public questions. Without per-test results from both versions, you can't work out how much each effect contributes.

Three different gaps: 3, 12 or 26 points

Artificial Analysis summarizes version 5 by saying the gap for open models widens "from 3 to 12 points." In that framing, Kimi K3.1 Pro at 46 points counts as the best open model, even though its weights are "not yet released." The 3 already rested on that assumption: on October 23, the provider wrote that the gap would shrink "from 12 points to 3" once the weights are out. On October 28, they still weren't. On its own model page, Artificial Analysis even lists Kimi K3.1 Pro as a "Proprietary model," as it does Mistral Large 4.

If you count only weights that have actually been published, DeepSeek V4.1 Flash and GLM-5.3 at 32 points trail Opus 5.5 by 26 points. Before, the gap was 12. So it has more than doubled.

Mistral Large 4 in the field

At 25 points, Mistral Large 4 is level with Meta's Muse Spark Open (which was still one point ahead on October 19). Together they are the best-placed non-Chinese models with published or announced weights. DeepSeek and GLM are 7 points ahead, MiMo-V2.6-Pro 5, Ling 3.1 Flash and Step 5 4 each. Other European models have no score in version 5: Mistral Medium 3.5 and Mistral Small 4 are missing.

It's unclear whether Large 4 keeps its lead on agentic tasks. In version 4.3.2 it scored 27% on Terminal-Bench 4.0 (Muse: 8%) and 60% on AutomationBench (Muse: 50%). Both tests stay in the index with a combined weight of 15%, but the new model page shows no per-test results.

A closed model on low effort beats open ones

For each model, the index lists the reasoning effort level (max, xhigh, high). Claude Opus 5.5 at a low setting scores 35 points at an average of $0.98 per index task, 3 points more than the best model with published weights. At maximum effort, the same task costs $10.04. So any cost comparison between a downloadable model and an API model also depends on the effort level you choose.

Where Mistral Large 4's release stands

The Hugging Face repository lists October 31, 2026 as the estimated release date for the weights, but on the evening of October 28 it contained neither files nor a license. Mistral announced on October 6 that it would publish them "by the end of the month." The date of October 27, cited without a source by The Next Web and The New Stack, wasn't met. The license and the score of the final version remain open, since according to Mistral reinforcement training was still running.

Takeaways

The change makes methodological sense: private test sets make it harder to train on the test. It also shows how much leaderboards depend on composition and weighting. A uniform loss of 12 to 16 points across all open models suggests that public benchmarks may have flattered them until now, though that isn't proven. The headline figure of "12 points" is questionable, because it counts a not-yet-released model as open. Measured by weights that are actually available, the gap is 26. If you're weighing an open model against an API, you should also look at effort level and cost. All figures come from Artificial Analysis or from ActuIA's comparison. Per-test scores are missing.