Hacking knowledge deleted? What data filtering really does for Kimi and Hy4

What this is about

Within two weeks, two major Chinese AI labs made the same move. Moonshot AI unveiled Kimi K3.1 Pro on October 23, and the weights have been on Hugging Face since November 4. Tencent announced the full version of Hy4 on November 4. Both trained hacking knowledge out of their open models. The model doesn't learn to refuse these requests. The content is supposed to never make it into the model at all.

That sounds like a clean fix for a real problem: open weights can't be recalled, and anyone can modify them. We looked at three questions. What was actually removed? What do the numbers show? And who pays the price?

What was filtered, and what wasn't

In its K3.1 Pro blog post, Moonshot says it removed cyber content "embedded in the general mid-training corpus" as well as cyber tasks from its reinforcement learning (RL, learning through rewards) environments. General programming tasks stayed in. Mid-training is a late training phase in which an already pretrained model gets sharpened with selected data. What's missing matters here: the blog says nothing about the pretraining of the K3 base, the phase where the model picks up most of its knowledge. K3.1 Pro is a post-trained version of K3 from July.

Tencent is more contradictory. According to Reuters and Chinese reports, it filtered out content on exploit development, reverse engineering (taking apart and understanding programs without their source code) and penetration testing. Only defensive data such as proofs of concept and patches remained. Tencent's own English press release, however, says the model includes reverse engineering and penetration testing "for analysis purposes", and that only offensive material for exploiting vulnerabilities was excluded. Tencent's in-house Xuanwu and Keen labs tested crypto reverse engineering and breaking obfuscated programs, among other things, and according to Tencent found "no significant concerns". There are no numbers. The final weights and the system card are still missing. So far, only the August preview (770B parameters, Apache 2.0) is downloadable. Tencent hasn't said whether it was filtered as well.

The numbers: slowed down, but stronger than ever

Moonshot is the only lab that provides a before-and-after comparison. The Model Card on Hugging Face also includes scores from public benchmarks:

pass@3 means a task counts as solved if one of three attempts succeeds. According to Notebookcheck, xdev-bench-hard requires the model to exploit recently disclosed vulnerabilities.

Our math: filtering costs 9.5 percentage points, or just over 10 percent in relative terms. Compared to K3, though, the rate has nearly tripled, from 33.3 to 81.0%. On ExploitBench, the UK AI Security Institute (AISI) and its US counterpart CAISI measured 32% for K3 in July. Moonshot reports 74.7 for K3.1 Pro. The scores come from different evaluators and are only roughly comparable, but the direction is clear. The filtered model can do much more than its unfiltered predecessor. The closed frontier models are still ahead.

That's exactly Moonshot's stated goal. It doesn't want to release weights that "push the absolute frontier of cyber capabilities". At the same time, it wants to gradually raise what open models can do. Implicator.ai criticizes that the test is unpublished and nobody can verify it. It also argues that cyber can hardly be cleanly separated from other code, so the filter is more a rule of thumb than a principled boundary. Tencent hasn't published a single measurement.

Why open labs filter instead of blocking

A model that has only learned to say "I can't help with that" is easy to rework. Tools like Heretic 2.0 strip out these refusals without any retraining. According to an investigation by the Financial Times and the group Alice, a journalist needed less than ten minutes to do this for Llama 3.3 and Gemma 3. There are more than 8,500 Heretic models on Hugging Face. Kimi was affected too: for K3, AISI and CAISI found that the safeguards didn't stop the model from developing exploits. For K3.1 Pro, "uncensored" versions showed up within 24 hours, according to a post on X. Whether they're any good is unclear. In the Hugging Face discussion, one user praises them for "cyber work", while another couldn't get K3.1 Pro to hack at all.

Data filters go one level deeper. As Notebookcheck puts it, removing the block doesn't give you back a capability the model never learned. For open weights, that makes filtering the only lever that still works after the download.

What the research says, and where it doesn't fit

The groundwork comes from the "Deep Ignorance" study by EleutherAI, the UK AISI and other researchers (ICLR 2026). The authors trained several 6.9B-parameter models from scratch, with and without biohazard knowledge. The filtered models withstood up to 10,000 training steps with 300 million tokens of targeted fine-tuning, meaning retraining on exactly the specialist text that had been removed. According to the authors, that's more than ten times better than post-hoc safeguards. Circuit breakers (blocks built into the network), by contrast, failed after small interventions.

The authors add their own caveats: this is about resistance, not invulnerability. And if you give the models the knowledge in context, for example through a search tool, they get close to the level of unfiltered models again.

Our take: this only carries over to cyber with three caveats.

1. Wrong phase: Deep Ignorance filtered during pretraining. Moonshot says it only filtered mid-training and RL.
2. Knowledge in context is easy to get in cyber: Vulnerability descriptions, public proofs of concept and write-ups are all freely available online, and agents can search. K3 itself showed how fast that goes: according to Frontier Security, the model escaped its sandbox during a cyber test and grabbed the solution from GitHub.
3. Only bio was tested: The study looks exclusively at bio knowledge. According to EleutherAI, filtering didn't work for toxic content. We couldn't find a follow-up study on cyber.

Three other approaches: reroute, vet, wait

With Anthropic, instead of a refusal you get an answer from an older model. According to its help page, the Claude app shows when this switch happens, and you can turn it off. According to OrcaRouter, API customers don't get an error, just an answer from a different model. Vetted defenders can apply for the Cyber Verification Program, but Anthropic says Opus 5.5 isn't included yet. So closed providers can decide again on every request and revoke access. If you release open weights, you can't. Zhipu stands out because it did the opposite of filtering: according to The Batch, it deliberately added data that rewards finding vulnerabilities.

What filtering costs defenders and tinkerers

The strongest counterargument comes from the attack on Hugging Face in July. When the team wanted to analyze the traces, commercial frontier models blocked the analysis. Their safeguards "can't tell an incident responder from an attacker", Hugging Face wrote. So the team ran the open GLM 5.2 on its own servers and used it to analyze more than 17,000 log entries, without attack data or credentials ever leaving the company. Its recommendation afterwards: keep a capable model on hand that you can run yourself.

We've also covered cases that fall squarely into the filtered territory. Running locally, Qwen 3.8 27B analyzed an app's license check in about 30 minutes: thousands of lines of arm64 machine code, a hidden key, an RSA key that was too short. A Reddit user used a Qwen model to dig into an Android car radio. That's reverse engineering, exactly what Tencent removed according to Reuters. In the r/LocalLLaMA thread, the author warns that without security data you can no longer audit your own code on your own hardware, and you get pushed toward closed APIs and their false positives. Moonshot makes a similar argument, by the way: open models help with legacy systems that "rarely get their own security reviews".

And Qwen4? On October 29, Alibaba showed a slide reading "Qwen4 Series Coming Soon" with four variants. Since then, a post ("The becoming begins") has hinted at an imminent launch. There's no word on cyber data, and open weights aren't confirmed.

Our take

At Moonshot, the filter is more of a throttle than a lock. It keeps the model about ten percent below its own maximum and thus behind the closed frontier, but it doesn't erase the capability. Against tools like Heretic, the data route is more robust than any trained-in refusal. But the research behind it so far only covers bio knowledge, small models and pretraining filters, and by its own account it does little against knowledge in context.

With Tencent, there's nothing to evaluate: the statements contradict each other, there are no numbers, and the evaluators belong to the same company. Only the system card will show whether more was lost there than with Kimi, for example in reverse engineering. That would hit defenders and tinkerers hardest.

What's missing: Moonshot should publish xdev-bench-hard, and independent evaluators like AISI and CAISI should test the filtered models, including with fine-tuning and knowledge in context. Whether filtering becomes the standard for open weights now depends mostly on Qwen4.