Outside audit at OpenAI finds gaps in its AI misbehavior monitoring

What it's about

Two staff members from METR and Redwood Research spent three weeks on site at OpenAI, auditing the internal system meant to detect misaligned behavior by AI agents. METR is an organization that tests AI models for risky capabilities, and Redwood Research does AI safety research. The audit ran from July 21 to August 11. The results became public on October 23, 2026. The bottom line: the monitoring system had significant gaps, and OpenAI says it has since closed all of them.

What was audited and how

The two auditors had access to OpenAI's codebase and internal documents, and interviewed about a dozen employees. They examined the monitor, a oversight system that observes the actions of internal AI agents. Agents are AI systems that carry out multiple steps on their own, such as writing and running code. The audit was a pilot run jointly organized by METR, Redwood Research and OpenAI to give outside researchers deep access to internal infrastructure.

The vulnerabilities found

• Unmonitored traffic: An estimated 6 to 25 percent of internal agentic traffic on the Responses API (OpenAI's programming interface) from models of the GPT-5.5 class and later was, at the time of the audit, not intended for monitoring at all. METR stresses that the uncertainty of the estimate is considerable in both directions.
• Compaction event gap: Compaction events are summaries of the prior context that keep long agent runs compact. For internal Codex traffic, which was supposed to be monitored, about 10 percent went unmonitored between May and July as a result, according to METR (again with considerable uncertainty). OpenAI discovered the bug independently and fixed it during the audit.
• Delayed oversight: During this period, the monitor only triggered once an agent handed control back to the user. An agent could get around this by simply continuing to execute actions.
• Hidden actions and the sandbox: The auditors found ways to hide hundreds of actions from the monitor and to bypass the Codex sandbox. A sandbox is an isolated environment meant to stop programs from causing harm outside it.
• Monitor crash: Specially crafted inputs could probably take the monitor down entirely.
• RL training: Model behavior during reinforcement learning (training through rewards) was not monitored at all at the start of the audit.

The Hugging Face incident and fixes

OpenAI says that after the so-called Hugging Face incident in August, it paused RL training and extended monitoring to cover it. These changes came after the auditors' access period ended and were not assessed. The source gives no details about the incident.

Transparency: most of the report stays secret

OpenAI says all the vulnerabilities found have been fixed. The full report runs 61 pages and went internally to OpenAI. After redactions by OpenAI, the heads of METR and Redwood received a 38-page version. Only a short summary is public. METR says it had no right to disclose the redacted parts itself, and plans to push for more public transparency in future audits.

Takeaway

Outside audits with real access to infrastructure are rare and valuable, because they test not just models but the safeguards around them. That such gaps existed shows how hard it is to monitor autonomous agents comprehensively. The caveat: claims that everything is fixed can't be verified from the outside, since the later changes weren't assessed and the report is largely redacted. The estimates are also uncertain. Whether other labs will allow similar audits remains an open question.