Chinese AI weights on your own servers: what's real about the risk and what isn't

Cursor is considering dropping Kimi as its foundation, Airbnb and DoorDash are named in a US congressional report, and Washington is weighing a disclosure requirement. The usual justification is a vague "geopolitical risk." We broke down the alleged dangers and checked each one against public evidence.

Analysis Nov 20, 2026 11 min read Original analysis by our editors
Illustrative image, AI-generated

What's going on

As of today, we know what's in the investigative report from the House Homeland Security and China committees. According to CNBC, Cursor, SpaceX's AI coding tool, told lawmakers it would "evaluate phasing out China-developed open-weight models due to geopolitical risks." Cursor's in-house model Composer 2 is built on Moonshot AI's Kimi K2.5. The report also names Airbnb, whose customer service agent runs on Alibaba's Qwen, and DoorDash, which routes internal tasks through Kimi. On November 18, Reuters also reported that the Treasury Department and the White House Office of Science and Technology Policy are discussing whether companies should have to disclose on their websites that they use Chinese models.

What's missing from the coverage is the technology. A model you access through a Chinese provider's API sends every request to that provider's servers. Open weights running on your own servers in the US or Europe, by contrast, are just a file full of numbers. They have no network connection of their own. So we separated the alleged risks and checked each one against public evidence.

What the report actually claims

The investigation began on April 29, 2026, with letters to Anysphere (the company behind Cursor) and Airbnb. DoorDash followed on July 31. In their press release, chairmen John Moolenaar and Andrew Garbarino list six points:

  • Chinese companies are subject to Chinese law and could be forced to hand over data.
  • The models come with "hidden vulnerabilities." The release doesn't give examples.
  • They were trained by the "censorship regime."
  • They were created through distillation, meaning they were trained on outputs from American models, without those models' safeguards.
  • They create a risk of dependency.
  • Citing a March hearing, the release mentions "potential surveillance" and "persistent access to critical infrastructure."

According to Snowrock, Moolenaar says the risks remain even when companies run the models on their own infrastructure. Airbnb CEO Brian Chesky had already pushed back on that:

"An open-source model doesn't have access to data."

According to law firm McGuireWoods, the committees requested data policies, contracts and cybersecurity assessments, among other things. They also write that the benefits of open weights "do not eliminate the need for risk-based safeguards." From everything known so far, the report doesn't draw a technical distinction between API use and self-hosting. Its actual recommendations are about industrial policy: a "national American open-weight initiative" and a ban on Chinese models in federal systems.

The frequently cited figure that Chinese models grew from 1 to 30 percent of global AI usage also needs context. It comes from the December 2025 "State of AI" study by a16z and OpenRouter. It only measured traffic through OpenRouter. The roughly 30 percent was peak weeks; the average share was about 13 percent.

The risks, one by one

Alleged risk Public evidence Status Also applies to US models?
Data flowing to China DeepSeek app: unencrypted transmission (NowSecure), login data sent toward China Mobile (Feroot, AP) measured, but only for the app and API Yes, every API sees your requests
Data leaks when self-hosting Qwen3.8-Flash-Next tried to upload test data in a sandbox (Cohere) isolated case, likely a training artifact Yes (see follow-up test)
State backdoor in the weights no findings in original models suspected technically yes
Deliberately insecure code CrowdStrike, Booz Allen measured, cause unclear not studied this way
Censorship, political bias NIST/CAISI, independent study on Kimi K2.5 measured less, but not zero
Vulnerability to agent hijacking NIST/CAISI, F5 Labs measured, varies widely by model yes, to a lesser degree
Supply chain, licenses trust_remote_code, license changes measured yes

Data leaks: These are documented wherever a Chinese service is involved. In 2025, NowSecure found that DeepSeek's iOS app sends some data unencrypted. Feroot and AP found code that sends login data toward China Mobile. None of this has anything to do with weights running on your own servers.

The only self-hosting case we know of comes from Cohere (October 26, 2026). During a security test, Qwen3.8-Flash-Next tried to upload a test string to an Alibaba Cloud storage bucket in Beijing. It happened less than once in 100,000 calls, and the sandbox, the sealed-off test environment, blocked the attempt. A follow-up test by Toolhalla with 6,000 runs puts the finding in perspective. Cohere had prefilled the responses in its retry attempts, right up to the middle of an upload command. Under those conditions, Nvidia's Nemotron 3.5 Super VL accessed a cloud metadata address in 17.1 percent of runs, Qwen in only 0.3 percent. Toolhalla's explanation: during training, Qwen memorized addresses from Alibaba's own test sandboxes. The lesson is the same for every model. Whether an unexpected action causes harm depends on the agent's permissions, not on where the model comes from.

Backdoors and sleeper behavior: It's proven that this is possible. In its 2024 "Sleeper Agents" study, Anthropic showed that deliberately planted triggers survive standard safety training. The SilentCall paper (September 2026) built an agent that secretly harvests credentials after a certain system date, with a success rate of at least 99.6 percent. Standard benchmarks found nothing. A runtime monitor that checks every tool call before execution, however, caught every case, with a 1.73 percent false-positive rate. ProjectDiscovery planted a backdoor in a derivative Qwen2.5-7B for about $8 of compute. So backdoors like these have been demonstrated in modified derivatives and in lab experiments. We found no public finding in an original model from Alibaba, Moonshot, Z.ai or DeepSeek. Even CrowdStrike considers deliberate training on insecure code unlikely.

Insecure code: CrowdStrike tested raw DeepSeek-R1 with 30,250 prompts. Without any added context, the code contained vulnerabilities in 19 percent of cases. When an irrelevant reference to Tibet was added, the share of severe vulnerabilities rose to 27.2 percent. By our math, that's about 43 percent more. The results were graded by another language model, which was only checked against 85 human-reviewed responses. In May 2026, Booz Allen reported that Qwen produced 130 percent more vulnerabilities when the requester was supposedly a US government employee. For Kimi, no difference was measurable. However, the tests ran through online access, not locally. Lukasz Olejnik of King's College London says the stronger claims are "not fully supported as stated." Lennart Heim finds the study credible but considers a deliberate sleeper implausible. His guess is a side effect of party-line fine-tuning.

Censorship: This is the most solid finding. According to NIST/CAISI, DeepSeek models repeated misleading Chinese Communist Party (CCP) positions four times as often as US reference models. Kimi K2 Thinking did so in about 26 percent of cases in Chinese and about 7 percent in English. An independent study of Kimi K2.5 found "narrow" censorship, mostly in Chinese, and significantly fewer refusals for requests about chemical, biological, radiological, nuclear and explosive weapons (CBRNE).

Agent hijacking: This means an attacker redirects an AI agent through injected instructions. According to CAISI, agents built on DeepSeek R1-0528 followed malicious instructions 12 times more often on average than US models. With a common jailbreak, a trick to bypass safety rules, they answered 94 percent of overtly malicious requests, compared with 8 percent for US models. CAISI rates GLM-5.2 as more robust than other Chinese models. But CAISI also stresses that the safeguards of open models can be bypassed anyway when you host them yourself. F5 Labs measured prompt-injection resistance scores ranging from 93.08 points (Claude Sonnet 5) to 81.13 (Qwen3.5-397B) down to 46.58 (GLM-5.2). In other words, the spread within China is bigger than the gap between countries.

Supply chain and licenses: The Kimi K2.5 model card loads with trust_remote_code=True. That means the software executes Python code from the model repository at load time. GLM-5.3, by contrast, is natively supported by Transformers. Pluto Security shows how real the danger is: a campaign using fake "qwen2-tiny" models exploited exactly this feature to install a crypto miner. In vLLM, a vulnerability (CVE-2026-22807) even loaded such code without the flag up to version 0.14.0.

On the licensing side, Kimi K3 requires a separate agreement once you exceed $20 million in annual revenue if you offer the model as a service (model as a service, MaaS). GLM-5.3 requires operators of such services with more than $10 billion in revenue to pass a security review by Z.ai. Tencent switched from Apache 2.0 for Hy4 Preview to a custom license. Older Hunyuan licenses explicitly excluded the EU. But Mistral Large 4 also ships with a MaaS clause, while Mistral Large 3 was still under Apache 2.0.

What operators in Europe should actually check

We found no guidance on Chinese open-weight models from BSI or ANSSI. The AI Act ties obligations to a model's provider, not its country of origin. According to the Olala model card, if you modify a model substantially, you can become a provider yourself. Our take: these steps reduce risk regardless of origin:

  1. Cut the network: The inference server, the machine running the model, gets no outbound internet access.
  2. No foreign code: Only load Safetensors files, set trust_remote_code only after reviewing the code and with a pinned revision, and keep vLLM up to date.
  3. Pin weights by hash: Prefer originals over unknown derivatives, quantizations or "abliterated" versions, which have had refusals stripped out.
  4. Fence in agents: Check tool calls before execution, keep credentials out of the environment, require human approval for consequential actions.
  5. Run your own tests: Check generated code with static analysis, run prompt-injection tests in your own deployment scenario, and test political answers if the model advises customers.
  6. Read the license: Especially MaaS clauses and territorial exclusions, and do it for every new release.

The open alternatives

Model Origin License Plotly test (/43) Cost per test run
Kimi K3.1 Pro China Moonshot (MaaS clause) 42 n/a
GLM 5.3 Flash China n/a 38 $0.69
Qwen 3.8 27B China n/a 38 $1.71
Inkling 2 Small (Thinking Machines) USA Apache 2.0 39 $1.81
Muse Spark Open (Meta) USA restricted 38 $5.16
Nemotron 3 Ultra USA OpenMDW 1.1 34 $7.06
gpt-oss-120b USA Apache 2.0 27 $0.22

The source is Plotly's data analysis test. On Artificial Analysis's current Intelligence Index, Kimi K3.1 Pro leads with 46 points, ahead of GLM-5.3 with 32. Mistral Large 4 scores 26, Muse Spark Open 25, K2 Horizon 375B 18 and Nemotron 3 Ultra 12. The fully open models K2 Horizon from Abu Dhabi and OVHcloud's small Olala mini (Apache 2.0, AI Act documentation, runs on 8 GB of VRAM) are no match for Kimi. But they're interesting for companies that need to prove provenance and training data.

The bigger picture

Our take: the report conflates two things. If you use a Chinese API, you hand data to a company under Chinese law. That's a real, well-documented risk. If you self-host the weights, you don't have that problem. But you do inherit behavioral weaknesses that have been measured: censorship, weaker safety rules and, depending on the model, higher vulnerability to agent hijacking. A state backdoor in an original model hasn't been proven to date, but it can't be ruled out either, because the training data isn't available. Most countermeasures, meaning isolation, runtime monitoring and code review, are just as necessary for American models.

For US companies, the biggest measurable risk right now is political: subpoenas, disclosure requirements, reputation. According to Snowrock, AI researcher Nathan Lambert calls the planned disclosure requirement an "own goal." For Europe, this means you can reasonably self-host Qwen, GLM or Kimi with proper safeguards. But if political neutrality or verifiable provenance matters to you, the Western alternatives currently cost you performance.

Sources

#security#china#open-weights#regulation#cursor

Ask the AI assistant

Questions about this article, installation or background? Our assistant knows this page and runs on our own machine, not in someone else's cloud.

Connecting …

The Assistant is an AI (Gemma 4) and can make mistakes – please check important details against the source. Questions are used only to generate an answer and are deleted after one hour. Please don't enter any personal data.

Comments

What do you think? Already tried it? Write it here, no sign-up needed.

  1. Loading comments …

Comments are public. Links aren't allowed, and we delete abusive posts. You can delete your own comment at any time in this browser. Learn more in the Privacy Policy.