Villanova2: European MoE model with open weights and fully documented training data

The Villanova.AI project has released the Villanova2 model family. Its flagship is a Mixture-of-Experts model with 15.5 billion parameters, of which only 3.9 billion are active per token. It was trained from scratch on English, Italian, German, Spanish and French. Both the weights and the training data are open, the license is Apache 2.0, and there are also chat and vision variants.

Model releases Try it yourself Nov 15, 2026 8 min read Source: Reddit rising: KI-Szene

The key points in brief

Villanova2-15B-A3B-Base has 15.5 billion parameters, 3.9 billion of them active per token (Mixture of Experts with 64 experts, 8 of which are used per token).
It was trained on 4.7 trillion tokens in five languages (EN, IT, DE, ES, FR) with a 131,072-token context window, using 64 Nvidia B200 GPUs for about 88,000 GPU hours.
Averaged across 39 benchmarks (5-shot), it scores 73.21. The developers say that makes it the strongest European MoE model, but it still trails Qwen3-14B-Base (75.56) and Gemma-4-26B-A4B (76.21).
The weights are released under Apache 2.0, and the training data is documented source by source. The family also includes a chat variant, a vision-language variant and a 2.4B model.
According to a Reddit post, GGUF files are available in the same Hugging Face collection. The project is funded through the EU's IPCEI-CIS program.

What it's about

The European Villanova.AI project has released the Villanova2 model family on Hugging Face. The flagship is Villanova2-15B-A3B-Base, a language model with 15.5 billion parameters, of which only 3.9 billion do any work for each token. It was trained from scratch, not built on top of someone else's model, and it covers five languages: English, Italian, German, Spanish and French. The developers' motto is "Built in EU, for EU."

What stands out is how open the project is. The weights are released under the permissive Apache 2.0 license, which allows commercial use. On top of that, every single training data source is documented. Models that are "open weight & data" like this are rare. Most vendors publish their weights but keep quiet about what they trained on. The work is funded through the EU's IPCEI-CIS program.

The Model Card at huggingface.co/VillanovaAI/Villanova2-15B-A3B-Base covers the base model. The r/LocalLLaMA post that announced the release says the same collection also contains GGUF files. That's the format used by local tools such as llama.cpp and LM Studio.

The family: six models in two sizes

Villanova2 consists of two base models, each with two derivatives:

Model	Type	Size
Villanova2-15B-A3B-Base	Base model, Mixture of Experts	15.5B total / 3.9B active
Villanova2-15B-A3B	Instruction-tuned (chat)	same as above
Villanova2-15B-A3B-VL	Image + text	same as above
Villanova2-2B-Base	Base model, dense	2.4B
Villanova2-2B	Instruction-tuned (chat)	2.4B
Villanova2-2B-VL	Image + text	2.4B

Both base models went through the same 4.7-trillion-token training run, with the same tokenizer and the same recipe. The only differences are size and architecture.

One important point: a base model is not a chatbot. All it has learned is to continue text. As the developers put it:

Prompting a base model means continuing it, not asking it.

If you type "Can you explain …?", you won't get a useful answer. For conversations, use the instruct variant Villanova2-15B-A3B. The base model is meant as a starting point for your own fine-tuning.

How the model is built

Villanova2-15B-A3B is a Mixture-of-Experts (MoE) model. Think of it as a team of specialists: for each token, a router sends the work to just a few experts. Each layer has 64 experts, and 8 of them are used per token (12.5 percent). A "shared" expert also runs every time. So the whole model has to fit in memory, but it computes about as fast as a model with just under 4 billion parameters.

The architecture comes from Qwen3-Next, a hybrid design by Chinese company Alibaba. Of the 28 layers, 21 use "linear attention" (Gated DeltaNet). This method compresses the text seen so far into a compact state, which saves a lot of compute on long inputs. Every fourth layer, 7 in total, uses classic full attention, where every token can look at every other token. These layers use GQA (Grouped Query Attention), where several query heads share the same key-value pairs to save memory. The context window is 131,072 tokens, which is how much text the model can take into account at once.

The tokenizer is new too. It splits text into building blocks and was trained specifically for the five languages plus code. With 131,072 entries, it's half the size of the first generation's Salamandra tokenizer, which was spread across more than 35 European languages. The developers say it's still more efficient: it needs 12 percent fewer tokens for the same text in the core languages and in code, and more than 150 percent fewer for non-Latin scripts. Numbers are stored in chunks of at most three digits, and indentation is preserved.

Training: one run, four phases

Training ran on 64 Nvidia B200 GPUs across 8 nodes, for about 88,000 GPU hours in total. The run finished without a restart and had four phases:

Phase	Tokens	Sequence length
Stable pretraining	0 → 4.0T	4,096
Annealing (cooldown on higher-quality data)	4.0 → 4.6T	4,096
Long context 32k	4.6 → 4.65T	32,768
Long context 128k	4.65 → 4.7T	131,072

Most of the data comes from publicly documented datasets: FineWeb-Edu and DCLM-Edu (educational English web text), FinePDFs, FineWeb-2 for the other four languages, FineWiki, math collections such as FineMath, code from Stack-Edu, scientific text from Common Pile, and synthetic data from Nvidia (Nemotron). The annealing phase added especially high-quality sources, including educational PDFs in the four languages, StackExchange questions, GitHub issues and Kaggle notebooks. For long context, the team used books from before 1929 and long scientific PDFs, among other sources.

Benchmarks: strong for Europe, not world-class

The developers tested the model on 39 benchmarks in five languages. In 5-shot mode, where the model sees five solved examples before each task, it averages 73.21 points. Here's an excerpt from the comparison table:

Model	Origin	Parameters	5-shot	DE
Gemma-4-26B-A4B	non-EU	26B / 3.8B	76.21	73.36
Qwen3-14B-Base	non-EU	14.8B	75.56	71.49
Qwen3-30B-A3B-Base	non-EU	30.5B / 3.3B	75.53	71.29
Ministral-3-8B-Base	EU	8.4B	74.29	71.70
Apertus-70B	non-EU	70.6B	73.09	70.91
Villanova2-15B-A3B-Base	EU	15.5B / 3.9B	73.21	70.36
ALIA-40b	EU	40.4B	71.31	68.34
EuroLLM-22B	EU	22.6B	70.61	68.43
Llama-3.1-8B	non-EU	8.0B	67.25	62.57

That means Villanova2 clearly beats other European open-data models like EuroLLM-22B and ALIA-40b, even though both are bigger. It's roughly level with Switzerland's Apertus-70B. The strongest models from China and the US are still better, though. According to the developers, most of the gap comes from English, where Villanova2 trails Qwen3-30B-A3B by 4.5 points. Mistral's French Ministral-3-8B, which only has open weights, also comes out ahead with 74.29 points.

The five languages are no more than 5 points apart. English scores 75.37, Italian 74.13, Spanish 72.16 and French 70.98. German comes last at 70.36. Compared with the first generation, Villanova-2B-Base-2603, it's a big leap: from 52.28 to 73.21 points.

Long context: honestly measured

In the RULER test, the model has to find hidden information in long texts. Villanova2 scores 92.1 at 4,000 tokens, 79.6 at 32,000 tokens and 58.5 at 128,000 tokens. By the strict definition, its "effective context length" is only 16,384 tokens. That's the longest length at which the score stays above the 85.6 threshold. The model still finds a single hidden fact almost every time even at 128k (85.8 to 99.6 percent). But when it has to search for several keys at once, performance drops as low as 17.4 percent in some cases. The developers' own advice: reliable up to 16k, usable with caution up to 64k, and lossy on hard retrieval tasks at 128k.

What you need to try it

Even though only 3.9 billion parameters are active, the whole 15.5-billion-parameter model has to fit in memory. At full BF16 precision, that's about 31 GB, so you'll need a large professional GPU or two consumer cards. 4-bit quantization shrinks the model by storing its numbers at lower precision. According to Reddit, GGUF files like that are in the collection, and with them roughly 10 to 12 GB of VRAM is enough, for example an RTX 3060 12 GB or an RTX 4070. Because each token needs so little compute, an MoE model like this also runs reasonably fast on Apple Silicon Macs with 16 GB or more of unified memory, or even on the CPU at a pinch. Depending on the variant, you'll need about 9 to 31 GB of disk space. The Hugging Face page also offers notebooks for Google Colab and Kaggle.

If you want to chat, use the instruct variant. Since the model is based on Qwen3-Next, you'll need recent versions of Transformers, vLLM or llama.cpp.

Our take

Villanova2 won't top any leaderboards. For Europe, though, it's a serious step: a fully documented, open-data model under the Apache license that's efficient to run and clearly beats EuroLLM and ALIA in its own league. If privacy or compliance rules require you to know what a model was trained on, this gives you an alternative that can match or beat Qwen3-8B or Llama-3.1-8B.

Some points are open or worth a critical look. All benchmarks come from the developers themselves, the technical report is still marked "coming soon," and there are no independent tests yet. Of all the five languages, German is the weakest. The architecture comes from Alibaba's Qwen3-Next, so "Built in EU" refers to the training and data, not the underlying design. And the 131,072-token context window is only truly reliable up to about 16,000 tokens in practice, which the developers openly admit. That transparency deserves credit compared with many competitors. The developers explicitly advise against using the model for hard math problems and safety-critical applications.

Terms explained simply

Mixture of Experts (MoE): A model made up of many sub-networks, the "experts." Only a small selection of them is used for each token, so the model runs faster than its total size would suggest.
Base model: A model that has only learned to continue text. It doesn't follow instructions yet and isn't a chatbot. Instead, it's the foundation for further training.
5-shot: A test method where the model sees five solved examples before each task. In 0-shot, it gets no examples at all.
Tokenizer: The tool that splits text into small building blocks (tokens). A good tokenizer needs fewer tokens for the same text, which saves compute.
RULER: A benchmark that tests how well a model can find and process information in very long texts.
Gated DeltaNet: A form of "linear attention" that compresses the text seen so far into a compact state. That makes long texts much cheaper to process.
GGUF: A file format for language models that tools like llama.cpp, LM Studio and Ollama can read. It usually contains a smaller, quantized version of the model.
LoRA/QLoRA: Lightweight fine-tuning methods: instead of the whole model, only small add-on modules are trained, so it works even on a single machine.

Sources

Reddit rising: KI-Szene huggingface.co

#villanova2#open-source#moe#europe#multilingual

Read the original source

Try it yourself: Villanova2-15B-A3B-Base

Explained step by step, no prior knowledge needed – on your own computer or free in your browser.

What can I do with it?

Villanova2-15B-A3B-Base is a base model. It continues text instead of answering questions. Give it the start of a letter, a story or an encyclopedia entry, or a list with a few examples, and it keeps writing in the same style, in English, Italian, German, Spanish or French. Because only 3.9 billion parameters are active per token, it's fairly fast for its size. In the developers' benchmarks, it lands roughly on par with Qwen3-8B, clearly ahead of Llama-3.1-8B and EuroLLM-22B, but behind Qwen3-14B and Gemma-4-26B-A4B. Its biggest strength is openness: an Apache 2.0 license and documented training data. That makes it a good foundation for your own fine-tuning. Its weak spots are reliable arithmetic, multi-step math and very long texts beyond 16,000 tokens. For regular chat, use the instruct variant Villanova2-15B-A3B.

Who is it worth it for?

It's worth a look for developers, researchers and tinkerers who want to fine-tune an openly documented European model themselves (for example with LoRA) or test text continuation in several European languages. If you just want a local chat assistant, go with the instruct variant Villanova2-15B-A3B. On raw performance, Qwen3 is often stronger.

Ideas: What you can do with it

Few-shot translation helper

Write three or four English–Italian sentence pairs one below the other and leave the last translation blank. The model continues the pattern and translates your new sentence. That gives you a small local translator for the five core languages, no chat mode needed.

Continue stories

Enter the opening paragraphs of a short story or fairy tale and let the model keep writing. A higher temperature makes the continuations more creative. It's a quick way to get ideas for where a plot could go.

Train your own domain assistant

With LoRA/QLoRA, you can fine-tune the base model on your own texts, such as manuals, meeting minutes or technical articles. Because the license and training data are open, this also works for companies that have to prove where their data comes from.

Product and form copy in a fixed format

Provide two finished product descriptions in a fixed template (name, features, short description), then just the name and features of a new product. The model fills in the description in the same style, in Spanish or French if you like.

Hands-on language model research

Because the training data is documented, you can investigate how the model's style or knowledge traces back to specific data sources. That makes it a good fit for theses or university courses on multilingual models.

Size15.5 B parameters (3.9 billion active)

Download9.3 GB (about 12 min at 100 Mbit/s)

Licenseapache-2.0

TypeLanguage model (text chat)

Hugging FaceVillanovaAI/Villanova2-15B-A3B-Base

What your computer needs

Graphics card (NVIDIA), fully in VRAM at least 12 GB VRAMe.g. RTX 3060 12 GB, RTX 4070, RTX 5070, RX 7700 XT – fastest; with less VRAM, see the next line

Graphics card + system memory (MoE offloading) from approx. 6 GB VRAM + 16 GB RAMdie Experten liegen im Arbeitsspeicher, nur der Kern auf der Grafikkarte (llama.cpp/LM Studio). Langsamer als komplett im VRAM, aber auf normalen PCs machbar

Mac (Apple Silicon M1–M5) from 24 GB unified memoryThe GPU and CPU share the memory – the model can use about 70% of it

CPU only (no graphics card) Runs smoothly without a graphics cardan estimated 10 words per second on a current desktop PC (MoE: only 3.9 billion parameters active per word). This requires at least 16 GB of RAM.

Storage approx. 11 GB freeModel download plus programs

Google Colab runs for freeThe free Tesla T4 GPU with 15 GB is enough

Values estimated from the model size – the Model Card may have more precise figures.

Which variant is right for me?

Many models come in several sizes or "quantizations." These are compressed versions: smaller and faster, but slightly less accurate. Rule of thumb: pick the largest variant whose memory requirement fits in your VRAM (video memory). The scripts below choose this for you automatically.

Variant	Download	Needs approx.	Compatible graphics cards
`4-bit (Q4)` Default	9.3 GB	12 GB	e.g. RTX 3060 12 GB, RTX 4070, RTX 5070, RX 7700 XT
`8-bit (Q8)`	17 GB	20 GB	e.g. RTX 3090, RTX 4090, Tesla P40
`16-bit (Original)`	31 GB	36 GB	e.g. 2× RTX 3090/4090 or RTX A6000

If you just want to try it out, grab a 4-bit GGUF file (e.g. Q4_K_M) from the Villanova2 collection. It needs about 10 to 12 GB of VRAM or unified memory on a Mac. With 16 GB or more, you can go for 5- or 6-bit and get slightly better quality. The full BF16 version at about 31 GB is meant for fine-tuning, and the developers recommend that precision for it. For chat instead of text continuation, download Villanova2-15B-A3B (instruct). For images, there's the VL variant. On weak hardware, the dense Villanova2-2B with 2.4 billion parameters is the lighter alternative.

The entire 15.5-billion-parameter model has to fit in memory, even though only 3.9 billion are active per token. That's about 31 GB in BF16 and about 9 to 11 GB in 4-bit.
With 4-bit quantization, a GPU with 12 GB of VRAM is enough. With 8 GB, it only works if part of the model is offloaded to system RAM, and then it's much slower.
Because each token needs little compute, it runs fairly quickly on Apple Silicon Macs with 16 GB or more of unified memory, and even on the CPU.
Long contexts take extra memory. But since only 7 of the 28 layers use full attention, memory use grows more slowly than with classic models.
According to the developers, full fine-tuning needs multiple nodes. On a single machine, LoRA/QLoRA is the practical route.

Gated access: You need a free Hugging Face account. Log in on the model page and accept the license. Then create an access token (type Read) at huggingface.co/settings/tokens and run hf auth login once before you start.

How do I find out what my computer has?

Windows: Press Ctrl + Shift + Esc → Task Manager → Performance tab → click GPU on the left. At the bottom you'll see Dedicated GPU memory (that's your VRAM). You'll find system memory under Memory.
Mac: Apple menu at the top left → About This Mac. You'll see your chip (e.g. M2) and memory (e.g. 16 GB). On Apple chips, graphics and processor share this memory.
Linux: Type nvidia-smi in the terminal (shows your graphics card and its memory) and free -h for system memory.

Guide

Google Colab (in your browser, no installation)

Google Colab is a free service from Google. It gives you a computer with a graphics card in your browser for a few hours. All you need is a Google account. For this model: runs for free – The free Tesla T4 GPU with 15 GB is enough.

Download the notebook. Click the button and save the file.
Download colab.ipynb
Open Colab. Go to colab.research.google.com and sign in with your Google account.
Upload the notebook. In the window that appears, click Upload on the left (or choose File → Upload notebook at the top) and select the file colab.ipynb.
Turn on the GPU. In the top menu, go to Runtime → Change runtime type → under Hardware accelerator, select T4 GPU → Save.
Start everything. Runtime → Run all (keyboard shortcut Ctrl + F9). If you see the warning “This notebook was not authored by Google,” click Run anyway.
Wait. The cells run one after another. A green checkmark appears to the left of each cell when it's done. Installation and download take about 7 minutes here.
Try it out. The result appears below the last cell. To test something else, change the text in the cell and run it again with Shift + Enter.
Clean up. When you're done: Runtime → Disconnect and delete runtime. This saves your free quota.

Good to know: The free GPU is limited (usually a few hours per day). After about 90 minutes of inactivity, Colab disconnects, and you'll have to start everything over.

Windows 10 / 11

This model runs on Python. Our script handles the setup automatically: it creates an isolated Python environment, installs the required packages (PyTorch, approx. 3 GB) and downloads the model (9.3 GB (about 12 min at 100 Mbit/s)).

Update your graphics driver (NVIDIA): install the latest driver. Without an NVIDIA card, it runs on the CPU – much more slowly.
Install Python. Download Python 3.12 from python.org and run the installer. Important: At the bottom of the first window, tick Add python.exe to PATH, then click Install Now.
Create a folder and download the files. Create a folder, e.g. Documents\AI-Test, and save both files into it:
try_windows.ps1 demo.py
Open a terminal in the folder. Open the folder in File Explorer, right-click an empty spot → Open in Terminal.
Run it. Paste this command and press Enter:
```
powershell -ExecutionPolicy Bypass -File try_windows.ps1
```
Installation and download are running now. The first time, this can take 10–30 minutes.
Result. The window shows the model's answer to a test question.
Try your own input: Open demo.py in an editor, change the sample text, and save. Then run this in the terminal:
```
.\.venv-villanova2-15b-a3b-base\Scripts\Activate.ps1
python demo.py
```

macOS

Macs with an Apple chip (M1 to M5) are ideal. Older Intel Macs are very slow.

Open Terminal. Press Cmd + Space, type Terminal, and press Enter.
Install the developer tools (first time only; includes Python):
```
xcode-select --install
```
Click Install in the window. If you see "already installed," everything is already in place.
Download the files – both end up in your Downloads folder:
try_mac.sh demo.py
Run it:
```
cd ~/Downloads
bash try_mac.sh
```
The first time, installation and download take 10–30 minutes.
Result. The window shows the model's answer to a test question.

Linux

Check your GPU. nvidia-smi should show your card. If it doesn't, install the driver on Ubuntu and restart:
```
sudo ubuntu-drivers install
```

Install the Python basics (Ubuntu/Debian):

sudo apt update && sudo apt install -y python3-venv python3-pip git

Put the files in a folder and open a terminal there:
try_linux.sh demo.py
Run it:
```
bash try_linux.sh
```
Result. The window shows the model's answer to a test question.

Try it

The three most important sights in Munich are

Recipe: Viennese apple strudel

Ingredients:
-

German: Das Wetter ist heute schön.
English: The weather is nice today.
German: Ich möchte einen Tisch für zwei reservieren.
English:

Question: What is the capital of Spain?
Answer: Madrid
Question: Who wrote "Faust"?
Answer: Johann Wolfgang von Goethe
Question: Which river flows through Paris?
Answer:

L'Italia è una penisola che

Tips for better results

For text continuation, the developers recommend temperature 0.7 and top_p 0.9. For factual continuations, you can lower the temperature to around 0.3 to 0.5.
Use few-shot prompts: according to the developers, two to five examples in the desired format before the actual task add more than 5 points compared with no examples.
Keep inputs under 16,000 tokens if you can. Up to that point the model is reliable. At 32,000 to 64,000 tokens, use it with caution.
For your own fine-tuning, the developers recommend BF16, sequence lengths of 4,000 to 8,000 tokens, and LoRA on the attention and shared-expert layers.
Italian and English are the strongest languages, while German scores a bit lower. For German text, it pays to give the model a good, detailed opening.

If something goes wrong

The base model doesn't understand questions or instructions. "Explain to me …" leads to confused continuations. Write text openings or example lists instead, or use the instruct variant Villanova2-15B-A3B.

The architecture is based on Qwen3-Next (Gated DeltaNet + attention). Older versions of Transformers, vLLM, llama.cpp, LM Studio or Ollama don't support it. Update to the latest version (pip install -U transformers accelerate).

The model isn't in the official Ollama library. Download the GGUF file from VillanovaAI's Hugging Face collection, or use the command docker model run hf.co/VillanovaAI/Villanova2-15B-A3B-Base.

Always use the bundled Villanova tokenizer for fine-tuning. According to the developers, a different tokenizer makes the embeddings useless.

Without a stop condition, a base model often keeps writing forever. Limit the length with max_new_tokens or set a stop sequence, such as a newline.

“out of memory,” a crash, or your computer freezes

The model is too big for your VRAM (video memory). Pick a smaller variant from the table above and close other programs, especially games and browsers with lots of tabs.

It runs, but extremely slowly

Your CPU is probably doing the work instead of your graphics card. In Ollama, the PROCESSOR column of ollama ps shows how much runs on the GPU. If it says "100% CPU", update your graphics driver or pick a smaller variant that fits entirely into VRAM (video memory).

“ollama” or “python” is not found

Close the terminal window and open it again so it recognizes the newly installed program. For Python on Windows: reinstall it and check Add python.exe to PATH.

Windows: "Running scripts is disabled on this system"

Run the script with exactly the command above: powershell -ExecutionPolicy Bypass -File try_windows.ps1. This allows execution for this one time only.

Error message when loading the model (version, "trust_remote_code", unknown architecture)

Brand-new models often need the latest software version. With Ollama: update Ollama. With Python: pip install -U transformers in the activated environment. The Model Card usually lists the exact version.

Tokenstrom generates the scripts and notebook automatically. Please take a quick look before running them – they're plain, readable text. Mind the model's license: apache-2.0.

Ask the AI assistant

Questions about this article, installation or background? Our assistant knows this page and runs on our own machine, not in someone else's cloud.

Connecting …

The Assistant is an AI (Gemma 4) and can make mistakes – please check important details against the source. Questions are used only to generate an answer and are deleted after one hour. Please don't enter any personal data.

Comments

What do you think? Already tried it? Write it here, no sign-up needed.

Loading comments …

Comments are public. Links aren't allowed, and we delete abusive posts. You can delete your own comment at any time in this browser. Learn more in the Privacy Policy.