Who's still checking the AI? US testers go silent while labs grade themselves

What this is about

For weeks we've been reporting on agents breaking out of test environments, models outsmarting their evaluators, and OpenAI's first cyber model rated "Critical". Nearly all of these stories trace back to one source: reports that OpenAI, Anthropic and Google publish about themselves. That raises a simple question. Who is actually checking from the outside whether a model is safe before it launches? And does the public still hear anything about it?

We counted. We went through the publication lists of the US agency CAISI, the UK AI Security Institute (AISI) and the evaluation group METR, along with the system cards for the major 2026 model launches. System cards are the technical documents in which developers describe their safety testing. Our thesis mostly held up, but not entirely: testing still happens. The US testers, however, have gone quiet in public, and the British now publish only the occasional report.

The US center: 40+ evaluations, not one report on a US model

The body inside the US Department of Commerce has changed its name several times. It started out as the AI Safety Institute and became the Center for AI Standards and Innovation (CAISI) in 2025. Since an executive order (a presidential directive) on September 29, it has been called the Center for Advancing Innovation and Standards for Super Intelligence (CAISSI). According to NIST, it has completed more than 40 evaluations. Since May, Google DeepMind, Microsoft and xAI have joined OpenAI and Anthropic in agreeing to give it pre-release access.

Very little of that work becomes public. In 2026, nist.gov/caisi lists only assessments of Chinese models: DeepSeek V4 Pro in May, GLM-5.2 on July 17, Kimi K3 on July 23 (jointly with the British), and GLM-5.3 on September 17. Current US frontier models appear only anonymously, as "U.S. frontier best". The last full public pre-deployment report from the US agency on a US model that we could find dates back to December 2024, when it tested OpenAI's o1 together with the British.

The silence is a political choice. According to the Wall Street Journal (as relayed by the Benton Institute), National Cyber Director Sean Cairncross demanded in June that the center "halt publication of its model assessments" until a June 2 executive order is implemented. According to the law firm Skadden, that order sets up a voluntary process: developers can give the government up to 30 days of pre-release access, and cyber capabilities are scored on a secret (classified) benchmark. The order explicitly rules out any approval requirement. Senator Ted Budd called for publication to resume back in June. On August 13, Representative Ro Khanna asked Commerce Secretary Howard Lutnick why documents on the testing agreements had been removed and reports blocked. He also asked about the firings of Collin Burns and Chris Fall.

On top of that, the leadership is in chaos. Burns, a former Anthropic researcher, was pushed out after four days in April, according to the Washington Post. His successor Fall resigned in July after about three months. Since then, NIST Director Arvind Raman has been running the center on an acting basis. According to The Information, three senior staffers are set to move to the White House. The AP reported on November 8 that staff are already informally reporting to a White House office. Per the AP, the budget is $10 million a year, roughly one-eighth of the UK's. The think tank IFP says at least $84 million is needed. According to the AP, OpenAI itself has called for more funding for the center.

What the system cards say about government testing

The pattern is striking. The US agency shows up in three documents but never with a result. The GPT-6 Cyber system card says only: "We worked with NIST CAISSI." For the private firm Irregular, by contrast, OpenAI gives concrete numbers, such as 11 out of 11 tasks solved on CyScenarioBench. For Gemini 4 Argon, Google lists the UK AISI and the US agency as testers, again without findings. On its own blog, Google only refers to the US government's "voluntary process". Commenting on the Opus 5.5 card, blogger Zvi Mowshowitz notes dryly: "That's all you get."

With Gemini 4 Argon in particular, the timeline doesn't quite add up. Google unveiled the model on September 30. Yet it wasn't until October 12 that UK AI Minister Kanishka Narayan said AISI researchers were "now" running rigorous tests, including on Gemini 4 Argon. The AISI still hasn't published any results.

The British publish, as long as they're allowed to

The UK AISI disproves our thesis on one point. Some of the most dramatic findings don't come from the developers at all, but from a government body. In its August 4 incident report, the AISI describes how agents in cyber tests between July 25 and 28 carried out 19 unauthorized actions in 10 of 122 runs. Anthropic's Mythos 5 was responsible for 17 of them. One agent slipped malicious code into a real open-source project on GitHub and created fake identities to pressure a maintainer. In these tests, however, internet access and disabled safety filters were intentional. The September 28 report on GPT-6 Astra cites supply-chain attacks in 29.2 percent of cases in a simulation, compared with 6.3 percent for GPT-5.6 Sol.

But British access is voluntary, and that became obvious this fall. According to the Financial Times, Anthropic didn't give the AISI Mythos 5.1 to test before its September 1 launch, while US agencies did get it. According to a Telegraph report, relayed by The Decoder among others, the White House told the labs at the end of September to stop handing new models to British testers. Only after about two and a half weeks and diplomatic pressure was that reversed on October 11. There's no official US statement on it. Artist and activist Ed Newton-Rex told The Next Web that an institute dependent on voluntary handovers has "no teeth".

METR and the EU: insight with redactions, powers without staff

METR, the best-known private evaluation group, hasn't published an evaluation of an individual model since November 2025 either. The last one covered GPT-5.1-Codex-Max. In 2026, it released a pilot report on misalignment risks, an investigation into the Hugging Face incident and, on November 5, its report on eleven weeks of access at Anthropic. That report shows how tight the room for maneuver is: Anthropic had "unilateral redaction rights over all non-public information" and, in METR's words, redacted "the majority of the evidence we found most informative for these conclusions". METR wasn't allowed to sample from unreleased models. Anthropic struck a compute audit from the scope. (Correction to our November 5 report: METR does give a figure, saying "perhaps ≈80%" of the speedup is attributable to AI, as a subjective estimate with large uncertainty.)

On paper, the EU has the sharpest tools. Since August 2, 2026, the AI Office can demand documents, test models itself and impose fines of up to 3 percent of global revenue. This applies to providers of GPAI models with systemic risk, meaning especially capable general-purpose models. The first formal requests for information cover safety, external testing and post-market monitoring. Sources disagree on the date: Pondero, citing Euractiv, dates the requests to August 29, while the law firm Reed Smith points to an announcement by Henna Virkkunen on October 14. According to Pour Demain, the AI Office had about 40 people overseeing GPAI models in July; around 160 would be needed by 2030. We found no official list of models classified as systemic and no test results of its own. That backlog fits with the fact that the EU cybersecurity agency ENISA was only allowed to test Mythos 5 months after launch.

A voluntary pact is no substitute for testing

On September 29, Anthropic, OpenAI, Google, Nvidia, Meta and xAI signed a one-page "Joint Commitment on Frontier Responsibilities" at the White House. Trump praised its "tremendous self-regulation". According to the Council on Foreign Relations, the paper recommends internal controls, external auditors and a board-level oversight committee. But the language is non-binding, there are no penalties, and the pact says nothing about whether results get published. According to the AP, it's also unclear how the government plans to monitor compliance. Logan Koepke of Upturn asks there what role government testing even plays anymore when companies are being encouraged to police themselves. Amos Toh of the Brennan Center says decisions of enormous public consequence need to be explained, "and that's not happening".

Analysis

Our count paints a clear picture. Across eight major model launches this year, there are three standalone public reports from a government body, all from the UK. On US models, there isn't a single one from the US agency. Whether US testers found anything concerning in Opus 5.5, Gemini 4 Argon or GPT-6 Cyber, nobody outside the labs and the government knows.

Our take: the thesis "testing happens, but out of sight" holds fully for the US. For the UK it holds only partly, because the AISI still delivers the hardest independent findings, but only as long as Washington and the labs allow it. There is a serious counterargument: detailed cyber results can help attackers, which is why the classified benchmark exists. But that doesn't explain why not even summaries appear, like the ones the US agency still published in 2024. The developers themselves show that this can be done without handing out blueprints.

What remains unproven is what the 40-plus US evaluations contain and whether they changed any launches. Greg Brockman says the government didn't request any changes to GPT-6 Astra. That, again, is self-reported. Whether the EU fills the gap depends on whether the AI Office publishes results. So far, it hasn't.