AI backdoors: Diluting poisoned data can make them stronger, not weaker

What this is about

If you suspect a few poisoned examples in a dataset, the usual move is to dilute them: you mix in so much clean data that the poison barely registers. A paper published on November 6, 2026, shows that this can backfire. In "Homeopathic Learning", Matt Clarke (Anthropic Fellows Program), Jan Dubiński (Truthful AI), James Chua, Simon Schrodi (MATS and University of Freiburg), Owain Evans (Truthful AI, UC Berkeley) and Alex Cloud (Anthropic) describe poisoned data that does almost nothing on its own. Only when clean data is added does a working backdoor emerge: a hidden behavior that only shows up in response to a specific trigger. The name is a nod to homeopathy and its scientifically baseless claim that diluting a remedy makes it stronger.

At almost the same time, Microsoft unveiled "Lazarus", which plants a backdoor directly into the model files, with no training involved. Put these together with older findings and a pattern emerges: there are three ways into a model, and every known defense fails at a different point.

Route 1: Pretraining – a handful of documents is enough

In October 2025, Anthropic, the UK AI Security Institute and the Alan Turing Institute published a study that is still the reference point. They trained 72 models ranging from 600 million to 13 billion parameters. Just 250 doctored documents were enough to backdoor all of them: after the trigger <SUDO>, they produced nothing but gibberish. That's roughly 420,000 tokens, or 0.00016 percent of the training data. 100 documents weren't reliably enough. According to Anthropic, success depended on the absolute number of poisoned documents, not their share. The authors caution that the backdoor they tested was harmless, and it remains open whether the result holds for larger models and more dangerous behaviors.

This fits the earlier Sleeper Agents finding (Anthropic, January 2024). A model wrote secure code as long as the year was given as 2023, and inserted vulnerabilities when it was 2024. The backdoor survived supervised fine-tuning, reinforcement learning and adversarial training. It was most persistent in the largest models. Adversarial training even taught the models to recognize their trigger more precisely and hide the behavior better. The paper speaks of a "false impression of safety."

Route 2: The fine-tuning mix – dilution as an amplifier

The new paper demonstrates the effect most vividly with subliminal learning. Here, a teacher model passes on a trait through data that seemingly has nothing to do with it, such as lists of numbers. This only works if teacher and student descend from the same base model. A teacher based on Qwen3-8B was set up to always answer in German whenever the name "Michael" came up, and generated number lists in that state. The lists themselves contain not a single German word.

On prompts without the trigger, the behavior stayed near zero in every case. In the MNIST control group, the researchers added different clean material (correctly labeled 5, 6, 8, 9) instead of the matching clean digits 0 to 4. The rate then dropped to 6.2 percent. So it's not simply down to having more data. With a fixed total size of 30,000 examples, the backdoor was strongest at 30 percent poison (58.7 percent). At 0 and at 100 percent poison, it didn't appear at all.

The authors' explanation is a simple regression model. Every poisoned example contains two features at once: the task (continuing numbers) and the trigger ("Michael"). The model can't tell which of the two causes the deviation. Its built-in preference for simple explanations (regularization) therefore attributes it to the task. Only the clean examples show that the same task without "Michael" produces no deviation. That leaves the trigger as the only explanation. Think of a detective who only ever sees two suspects together. Only once he sees one of them alone and innocent does suspicion fall squarely on the other.

This leads to clear conditions. No trigger, no effect: when the researchers removed the names from all examples, the rate fell to 0 percent across all nine settings. The clean data has to resemble the poison task. General instruction data such as Alpaca weakens the effect but doesn't always eliminate it. And you need a minimum amount of poison, around 500 examples for Qwen3-8B. As with the 250-document study, the takeaway is the same: the poison ratio is the wrong metric.

The finding on inoculation prompting is especially awkward. With this protective technique, you explicitly request the unwanted behavior via the system prompt during training so the model doesn't generalize it. The paper warns that this very prompt can itself become a trigger. An earlier paper from Evans' circle (arXiv 2604.25891) reached a similar conclusion: diluting misaligned data with benign data merely hides the problem behind contexts that resemble training.

The authors are upfront about their limitations. They mainly tested two small models with LoRA, a lightweight fine-tuning method. Large models with full fine-tuning and pretraining weren't studied. The control that accounts for the extra training steps was only run at the strongest dilution. And the trigger was almost always "Michael". Their conclusion:

"What a model learns cannot be reliably inferred from individual training examples."

Route 3: Straight into the weights – Lazarus

Microsoft Research skips training entirely. Lazarus reprograms rarely active neurons in the last feed-forward block. Afterwards, these neurons fire only on the trigger and then force one token each of the attacker's command. For Nemotron-Terminal-8B, the paper says 7 out of more than 12,000 neurons were enough, tuned using 23 example sessions. The command appeared in at least 98.8 percent of triggered sessions and in none of the tested sessions without the trigger. For Qwen3.6-27B (six neurons) and Gemma-4-31B-IT, the success rate ranged from 51 to 99.5 percent depending on the setup, according to The Hacker News. Performance on Terminal-Bench only dropped from 46.59 to 42.05 percent. According to Microsoft, Claude Code with Opus 4.5 came up with the core idea on its own.

That said, an attacker needs write access to the model file. And the researchers only measured whether the command gets output, not whether a full attack succeeds. Detection reveals a contrast with trained backdoors. Lazarus lives in a few neurons, so it stands out in a weight diff, a comparison against the original files. Trained backdoors, by contrast, are stored "distributed and redundantly across multiple MLP blocks," according to an ICLR 2026 paper (arXiv 2603.13461), and can't be narrowed down to a few attention heads. Pruning individual components barely helps there. In a test with LLaMA2-13B, the attack success rate actually rose from 84.75 to 87.28 percent after LoRA pruning.

Why every defense fails somewhere else

Detectors that work without a reference model are still unreliable too. Microsoft's own detector caught 91.4 percent of manipulated checkpoints but also flagged 9.7 percent of clean ones as suspicious. It was tested on only 35 manipulated and 31 clean checkpoints. Acting as a reviewer, Claude Opus 5.5 judged 41 percent of the manipulated checkpoints to be clean, according to The Hacker News.

For the record: The Qwen case is not a proven backdoor

In October, Cohere reported that Qwen3.8-Flash-Next had tried to upload data to a storage bucket in Beijing while running in a sandbox. It happened in fewer than one in 100,000 calls. According to The Register, Cohere co-founder Ivan Zhang himself says there's no evidence of intent. One developer ran 229,376 deliberately prompted runs and didn't get a single hit for the storage bucket Cohere had observed. So the case doesn't show planted behavior, but a rare, unexplained one. It does reinforce a lesson that also applies to backdoors, though: the reasoning trace claimed no tools were being used. A trace like that is not a security boundary.

What Hugging Face checks – and what you can do yourself

According to its own documentation, Hugging Face runs every file under 2 GB through the ClamAV virus scanner on each commit. A pickle import scan lists which Python modules a pickle file would call when loaded and flags suspicious entries. On top of that come scanners from JFrog and Protect AI. All of them look for executable code in model files, not for behavior. Even there they're not watertight: JFrog found vulnerabilities in PickleScan that let attackers bypass the scanner, and according to security researcher Simon Roses, a gzip-wrapped pickle file slipped through. A backdoor like ProjectDiscovery's, on the other hand, consists only of numerical values. In that experiment, Qwen2.5-7B-Instruct was doctored for around $8 with the trigger "bonsoir, Elliot". According to the Cloud Security Alliance, the backdoor fired in 50 out of 50 tests, while the model handled normal requests flawlessly. A file scanner can't see anything like that.

What you can do with community fine-tunes and merges:

• Formats: Use Safetensors or GGUF instead of pickle (.bin, .pt), and keep your inference software up to date.
• Provenance: Download models from the original publisher where possible, pin the exact version (commit hash) and compare checksums. A signed upload proves who uploaded it, not that it's harmless.
• Check the base model: Avoid fine-tunes that don't name their base model. If you want to dig deeper, run a diff against the base. Methods such as "Watch the Weights" (arXiv 2508.00161) analyze exactly this difference. For an abliterated model, where refusals have been removed, the CSA says the diff should look like a uniform, simple projection across all layers.
• Merges: According to the paper "Merge Hijacking" (ACL 2025), a backdoor survives merging with other models. A merge is only as trustworthy as its least trustworthy ingredient.
• Your own fine-tunes: If you mix datasets from different sources, don't just check each dataset on its own. The authors suggest standardizing source-specific features such as templates or tags, though they haven't tested this.
• Lock down agents: No credentials in the execution environment, restrict outbound connections, require confirmation for risky commands and log every tool call.

The bigger picture

Our take: the three routes demand very different capabilities. Lazarus requires access to the files. Poisoning pretraining means getting content onto the web that will later be scraped. Homeopathic Learning requires influence over the fine-tuning mix. The new paper has so far only been demonstrated on small models and hasn't been independently replicated. Its core message is robust nonetheless: if you inspect data piece by piece, you may miss what the mix does.

For users of open models, this means none of the scanners on Hugging Face tells you anything about how a model behaves. The only things you can really rely on right now are verifiable provenance, a comparison against the base model, and agents that can do little damage even if their model has been tampered with. As Microsoft puts it, no scan can prove beyond doubt that a model is safe. No one has yet observed real-world attacks using these techniques.