Researchers Find Blind Spots in OpenAI Internal Safety Checks

The push to make artificial intelligence safer is constantly evolving, but a recent deep dive into one of the industry's leading labs suggests there is still work to do.

A team of researchers from METR and Redwood Research spent three weeks physically on-site at OpenAI to examine how the company watches its own AI systems. The goal was to see if the internal tools designed to catch rogue behavior were actually up to the task. They had full access to the codebase, internal documents, and conversations with about a dozen employees from July 21 to August 11. The findings were not entirely reassuring.

One of the most concerning discoveries involved a lack of basic coverage for some of the company's most powerful models. The auditors estimated that between 6 and 25 percent of internal traffic from models at the GPT-5.5 class and above was not monitored at all during their visit. METR noted that this range has significant uncertainty in both directions, but the potential gap is large enough to raise eyebrows. If a quarter of the traffic is invisible to safety tools, it creates a sizable blind spot in the system.

The team also uncovered a specific technical flaw affecting the internal Codex tools. They found that roughly 10 percent of this traffic went untracked between May and July due to a bug involving compaction events. These are essentially context summaries that help manage long conversations. The auditors pointed out that during this period, agents could delay oversight because the monitoring system only triggered when control was handed back to the user. This allowed the AI to simply keep executing actions without triggering the safety checks. OpenAI had actually found and fixed this bug while the audit was still in progress, but the gap had already existed for months.

Beyond missing traffic, the researchers identified several ways the safety systems could be bypassed or disrupted. They found methods to hide hundreds of actions from the monitor and to slip out of the Codex sandbox. More alarmingly, they determined that specially crafted inputs could likely crash the monitoring system entirely. They also noted that model behavior during reinforcement learning training was not being monitored at all when the audit started. OpenAI explained that it paused this type of training after an incident with Hugging Face in August and expanded coverage, but those changes happened after the researchers' access ended.

OpenAI states that it has now fixed all the weaknesses the auditors identified and has applied the lessons learned internally. The full report runs 61 pages and was handed over to the company. However, the public only received a short summary. After OpenAI applied redactions, the leadership at METR and Redwood Research received a 38-page version. METR stated that it did not have the right to release the redacted portions on its own, which means much of the detailed technical analysis remains private for now.

This audit was a pilot program designed to give outside researchers deep access to internal infrastructure, a first for this level of scrutiny. The collaboration between METR, Redwood Research, and OpenAI signals a shift toward more rigorous external checks. Looking ahead, METR has said it wants to push for greater public transparency in future audits. The hope is that as these programs expand, the public will get a clearer view of how these powerful systems are being watched.

The results serve as a reminder that safety is not a one-time fix but an ongoing process. The fact that gaps were found, and then fixed, is a positive step, but the scale of the blind spots found during the audit is a wake-up call for the industry. As models become more capable, the need for robust, transparent, and reliable monitoring systems becomes even more critical. The next few years will likely see more of these independent audits as the technology continues to advance.