The leak beneath the AI cloud: why inference software is spilling your data

What this is about

If you use an open model like GLM, DeepSeek or Kimi through an AI cloud instead of running it yourself, your inputs ultimately land in an inference engine. That's the program that runs the model on the GPUs, batches requests and generates the responses. The most widely used are SGLang, vLLM and TensorRT-LLM. Within a few weeks there were three incidents, each reported separately. At their core, all three involve this layer between the model and the user:

Our thesis: the software underneath many AI clouds has grown faster than its security culture. Nowhere is that clearer than with SGLang.

14 vulnerabilities, zero responses: SGLang's track record

We went through the CERT/CC notes one by one. The CERT Coordination Center at Carnegie Mellon University mediates between the people who find vulnerabilities and the vendors. In 2026 it published five notes on SGLang. Together they add up to exactly the 14 vulnerabilities that beri.net's analysis also counts:

Almost all of them follow the same basic pattern. Internally, SGLang ships data around using pickle, a Python format that can execute arbitrary code when it's loaded. Think of it as blindly unpacking every parcel you're sent and switching on whatever's inside. The data travels over ZeroMQ, a messaging library whose sockets require no authentication here. Anyone who can reach the right port can run code on the server. For CVE-2026-7301, CERT/CC says the scheduler binds to all network interfaces (0.0.0.0) by default. The researchers at Antiproof add a nuance: the code itself says 127.0.0.1, but every example in the installation guide uses 0.0.0.0.

Now for response times. Orca Security reported the first vulnerabilities on February 4. When no reply came, CERT/CC ended up writing a proposed patch itself and brought in the US agency CISA, also without success. Only version 0.5.10 bound the broker to localhost. For the Jinja template bug, the current main branch now uses a sandbox, as we confirmed in the code. But there's no advisory for it. SGLang's security page still says “no published security advisories” and “no security policy.” So if you run SGLang, there's no official way to find out which version fixes which vulnerability.

The most important switch is still set to unsafe. CERT/CC recommends setting SGLANG_USE_PICKLE_IPC to “false.” But as of today (November 13), the file environ.py still says EnvBool(True). GitHub ticket #29465, which was meant to change the default, was closed by a bot on September 23 for inactivity. The “flip the default” step hadn't even been started. On top of that there's CVE-2026-93034, which Forkast says went public on October 7: pickle remains reachable via the msgpack path when data-parallel attention runs with a non-local --dist-init-addr. According to Forkast, there's no fix.

The Cloud Security Alliance offers one possible explanation, though only as speculation: since SGLang was spun out into the company RadixArk in January 2026, security contacts may have been lost. Nobody has confirmed this. What stands out is that Cloudflare had no trouble getting a meeting with the SGLang maintainers at RadixArk in September. Big partners can apparently reach the team. CERT/CC can't.

For comparison: vLLM and TensorRT-LLM

vLLM has serious vulnerabilities too, such as CVE-2026-22778 with a CVSS score of 9.8 via manipulated video URLs, fixed in 0.14.1. The difference is the process. Under its policy, vLLM publishes an advisory as soon as the fix ships in a release. Critical flaws are fixed beforehand in a private fork. Many entries concern crashes, meaning denial of service. So the large number of advisories signals transparency more than particularly bad code. And the pickle problems aren't unique to SGLang. In November 2025, Oligo Security showed under the name “ShadowMQ” that the same unsafe ZeroMQ pickle code had been copied from project to project. According to Oligo, the SGLang file literally starts with “Adapted from vLLM.” The other projects have since replaced the code.

Where “confidential AI” can't help by design

NEAR AI advertises private inference in a TEE (trusted execution environment). That's a hardware-isolated area, in this case using Intel TDX and H200 GPUs in confidential computing mode, that neither the server operator nor the operating system can see into. According to NEAR AI, Brave runs its Leo assistant on this infrastructure. The report doesn't say whether Brave users were affected. NEAR AI itself writes:

Confidential computing protects data from the operator and the host. It does not protect against a vulnerability in the software running inside the environment.

That leads to three fundamental limits:

1. The engine has to see the data in plaintext. Even with end-to-end encryption, the prompt is decrypted inside the enclave so the model can process it. Anyone running code there can read along.
2. Attestation proves identity, not security. With attestation, the hardware uses a signature to prove which software is running. A vulnerable SGLang version passes that test with flying colors.
3. Tenant isolation happens in the engine. The Cloudflare bug would have happened inside a TEE just the same, because the other users' content in the KV cache (the buffer for text that has already been processed) sat in the same process. NEAR AI already admitted in October that prefix caches are shared between customers and that response times can reveal whether a given text prefix has been processed before.

Exactly what was exploited remains unclear. NEAR AI names neither a CVE nor a version and says it only just reported the flaw to SGLang. So it could be a new vulnerability. What actually leaked: account data for around 100 users, data from 18 legacy agents that ran outside the TEE, and the API keys of two cloud customers. The report doesn't say whether confidential requests were actually read.

Cloudflare and Fireworks: two different flavors

Cloudflare wasn't an attack. A researcher reported the problem via HackerOne. GLM-5.3 ran with a context of 1,048,576 tokens, but the KV memory for prefill only held 681,280 tokens. In disaggregated prefill/decode mode (reading the input and generating the answer run on different GPUs), SGLang marked overlong requests as rejected but didn't abort them. The decode node then read memory pages that still held data from earlier requests. The customer still got HTTP 200. The bug had been reported upstream as issue #30233 back on July 6. Cloudflare says a prototype that checks every KV access could have caught it, but it had been switched off in mid-August because of stability problems. Cloudflare recommends SGLang 0.5.22 or later.

Fireworks only partly fits the pattern. On October 6, internal credentials were abused there, and from October 23 and 24 an attacker got onto development machines through a compromised cloud supplier. Fireworks attributes the attack to a group that makes money with LLMjacking, meaning it trades in stolen AI access. Fireworks says the actual inference wasn't affected and runs without data retention by default. Notion contradicts that, at least indirectly. Notion had a contractual “zero data retention” guarantee, meaning no storage at all, yet it found stored data from the endpoints it used. Notion says fewer than 1% of workspaces are affected. Cursor only names Fireworks as a customer it works with. There's no evidence that Cursor data was affected.

Checklist for self-hosters

• Take inventory: Find every SGLang process, including on laptops and test machines, along with its version and the network interfaces it listens on.
• Turn off pickle: Set SGLANG_USE_PICKLE_IPC=false and test that everything still works. Bind --dist-init-addr to loopback (because of CVE-2026-93034).
• Nothing on 0.0.0.0: Make scheduler, broker and ZeroMQ ports reachable only internally, and segment your networks.
• Block unused endpoints: For example /v1/rerank, LoRA adapter loading, /update_weights_from_disk and /server_info. Never run without an API key.
• Avoid risky options: Only enable --enable-custom-logit-processor and DUMPER_SERVER_PORT if you really need them.
• Check your models: Only load GGUF and .bin files from trusted sources.
• Update: Version 0.5.22 at a minimum. Since there are no advisories, keep an eye on changelogs and CERT/CC yourself.
• Harden: Restrict outbound traffic, don't run as root, and search your logs for keys.

Questions to ask your model host

1. Which engine, in which version, processes our requests?
2. How quickly do you roll out critical CVE fixes, and does your attestation reject builds with known vulnerabilities?
3. Is pickle IPC turned off, and are internal sockets cut off from the network?
4. Are KV or prefix caches shared between customers?
5. How do you technically enforce “zero data retention,” including at subcontractors and suppliers?
6. Do we get a list of components and versions for every attested environment?

The bigger picture

Here's what's established: in 2026, SGLang racked up at least 14 vulnerabilities documented by CERT/CC, none was patched at the time of publication, and the central unsafe switch is still on by default. What's not established is that NEAR AI was attacked through one of these specific flaws, and Fireworks had nothing to do with the engine. The pattern is still clear. The inference layer has become the bottleneck for trust, and all the more so as more open models run at hosts that all use the same software.

Our take: TEEs and “zero data retention” contracts shift trust around. They don't replace it. In the end, everything hinges on a Python process that sees your prompts in plaintext. Until SGLang has a proper process for handling security reports, companies should take the engine version as seriously as their choice of model. vLLM offers an alternative with a transparent process, even if it isn't bug-free.