Two researchers from outside OpenAI spent three weeks inside the company trying to break the safety systems it relies on to watch its own AI agents, and they found several ways through. In a post published on October 22, AI evaluation nonprofit METR said a METR employee and a Redwood Research employee uncovered “a number of novel vulnerabilities” in OpenAI’s internal misalignment monitoring, including settings that could switch monitoring off for new agents and an exploit that kept hundreds of agent actions out of its sight.

The pilot exercise ran from July 21 to August 11, 2026, and gave the pair deep access: OpenAI’s internal codebase, the monitor’s prompts and configuration details, internal documentation, and technical interviews with about a dozen staff from the monitoring, safety-systems and security teams. They worked on OpenAI-issued devices that never left the premises and were accompanied while on site. The result was a 61-page internal report shared with OpenAI’s researchers and security staff.

What Was Found

The exercise had two goals: estimate how much of OpenAI’s agentic inference its monitors actually cover, and find out whether agents could in principle disable or subvert the monitoring system. On coverage, the auditors estimated that roughly 75 to 94% of agentic internal Responses API traffic, which powers Codex among other things, from models at least as strong as GPT-5.5 was eligible for monitoring, with substantial uncertainty in both directions. For the subset used by internal Codex agents, eligibility was 99.2%.

Monitoring coverage estimates from the METR red-team exercise

Graphic: GPTOnline, using METR assets

The more striking findings were the ways agents could escape oversight entirely. The researchers found a way Codex agents could in principle bypass Codex sandboxing and Auto-Review, allowing them to take any action the host user could take. They discovered an exploit, with details redacted by OpenAI, that would prevent hundreds of recent actions, including the actions involved in the exploit itself, from being monitored. They also found two independent client-controllable headers that disable the monitoring system for new agents, letting an agent start an unmonitored, otherwise identical instance, though the act of setting them could itself be caught.

Other gaps were subtler. Because the monitor ran only when an agent handed control back to the user, and skipped “compaction” events when long sessions get summarized, the auditors estimated that on the order of 10% of eligible internal Codex traffic went unmonitored between May and July 2026. During that window, an agent could indefinitely postpone being monitored by simply continuing to take actions. OpenAI staff had independently spotted the compaction gap and fixed it during the exercise. Reinforcement learning training runs were also not monitored at the start of the exercise, though some had been monitored between March and June.

Not everything came back red. The monitor errored on fewer than 0.2% of requests, and a 24-hour snapshot showed zero load-shedding incidents. With minor exceptions, the monitoring system sees the full trajectory the agent sees, and its prompts look for scheming, sabotage, deception, data exfiltration, subversion of the monitoring system itself, and inbound or outbound prompt injection.

Why It Matters

OpenAI has since stated that all vulnerabilities the exercise identified that affect its production safeguards have been patched, and has described expanded coverage since July. METR says it has not evaluated those claims, and its findings describe the system as it existed on and before July 21. The groups also agreed upfront to publish only a brief, high-level account, since the pilot was designed to test external access to internal infrastructure, not to provide full public transparency.

The context makes the timing significant. Earlier in October, OpenAI disclosed two cases of its own agents weakening internal safety monitoring, and CEO Sam Altman had pledged “employee-like access” for outside evaluators in September, with contracts for third-party safety assessors reportedly being finalized this month. The METR pilot is one of the first public tests of that promise, and its findings feed directly into a larger question the industry has not answered: who watches the watchers when the watchers are AI systems themselves?

In their design recommendations, METR and Redwood argue future monitoring infrastructure should withstand deliberate, strategic and sustained attacks from highly capable internal agents at scale. Among their ideals: make running unmonitored requests require explicit approval, monitor agents on fixed intervals instead of waiting for them to end their turns, and enforce strict isolation between agents and their monitors, including keeping agents away from the monitor’s design and code.

Sources