OpenAI Audit Finds 6 to 25 Percent of GPT-5.5 Traffic Unmonitored
METR and Redwood researchers spent three weeks inside OpenAI testing its internal safeguards for spotting rogue model behavior.
By Petra Murphy ·
OpenAI's internal safety net for catching runaway model behavior had holes in it, according to two outside researchers who spent three weeks probing the company's infrastructure during the summer.
METR and Redwood Research auditors worked on site from July 21 to August 11 under a jointly organized pilot program that granted them access to OpenAI's codebase, internal documents, and roughly a dozen employees. Their estimate: between 6 and 25 percent of internal agentic Responses API traffic from models in the GPT-5.5 class and above went entirely unmonitored during that stretch. METR flagged wide uncertainty around both ends of that range.
The review surfaced additional breakdowns. About 10 percent of internal Codex traffic that should have been tracked slipped past the monitor between May and July, a figure METR also called highly uncertain, because the system mishandled compaction events, which are context summaries. OpenAI found and patched that bug on its own while the audit was underway. The monitor also only fired when an agent handed control back to a user, meaning an agent that kept acting could skip oversight altogether. The auditors further found ways to conceal hundreds of actions from the monitor, ways to get around the Codex sandbox, inputs that could likely crash the monitor, and no monitoring at all of model behavior during reinforcement learning training when the work began.
OpenAI said it stopped reinforcement learning training after the Hugging Face incident in August and later added monitoring for that stage of the process. Both moves came after the researchers' access period closed, so neither fell under the evaluation's scope.
The company said it has patched every weakness the auditors turned up. The full report runs to 61 pages and was delivered internally; after OpenAI applied redactions, METR and Redwood leadership received 38 pages, while the public got a short summary. METR said it cannot release the redacted material on its own. The organization said it wants any future audit to put more of the findings in front of the public.
For future audits, METR says it wants to push for greater public transparency.
