Anthropic said on October 22 that it has teamed up with software verification company Theorem to use Claude for large-scale formal verification, producing machine-checked proofs that 239 entry points across 123 real-world open-source libraries cannot crash under defined conditions. The work, carried out by Anthropic’s Frontier Red Team, claims to be “the first large-scale effort to use AI to apply formal verification to production software at the assembly code level,” according to the company’s research announcement.

What Was Announced?

In the announcement, Anthropic described how it converted fuzz tests from Google’s OSS-Fuzz initiative into formal specifications, then asked Claude to prove those specifications against the compiled RISC-V binaries of the libraries. Over the span of a few weeks, Claude patched the bugs it found and then proved no-crash guarantees for all 239 entry points. The resulting proofs total about 180 million lines of Rocq (the proof assistant formerly known as Coq), with a typical proof running to roughly 600,000 lines to cover binaries with a median length of 18,000 RISC-V instructions.

Anthropic quote: first large-scale effort to use AI for formal verification of production software at the assembly code level

Graphic: GPTOnline, using Anthropic assets

From Fuzz Tests to Formal Proofs

The specification Claude had to prove says that a fuzz test will not crash for any input smaller than 64 MB. If proven, fuzzing that entry point becomes superfluous: no input could ever produce a crash. Critically, the proofs operate on the compiled machine code, not the source code, so the guarantees apply to what actually runs.

The approach also shrinks what a developer must trust. Because the Rocq proof checker mechanically verifies each proof, only three artifacts need to be trusted: the formal models of RISC-V and Linux, the exact theorem statements, and the Rocq proof checker itself.

The Bugs Claude Found Along the Way

Attempting the proofs surfaced bugs that had eluded detection for years. Among them: a counter underflow that caused an out-of-bounds write, but only on inputs of around 20 MB with one rare, specific layout; an out-of-bounds read in an image decoder that needed a 64 MB input and more than 2 GB of memory; an out-of-bounds read that required a particular sequence of calls across two file handles; and a rediscovery of CVE-2026-56132 in libexpat, which needed deeply nested input. Claude also caught logic bugs that never crashed at all, including one case in which a feature meant to restrict which files an application can load was not always enforced.

To keep the pipeline autonomous, the team let Claude fix the bugs it found, patch the code, and then proceed to prove the regenerated specification.

Key numbers: 239 entry points, 123 libraries, about 180 million lines of Rocq proof code

Graphic: GPTOnline, using Anthropic assets

What the Proofs Do Not Show

Anthropic is careful about the limits. “This doesn’t mean these libraries contain no bugs,” the announcement says. The guarantees hold only under modeled assumptions: programs run as single-threaded static binaries with a fixed memory budget on a RISC-V CPU under Linux. Anything outside those assumptions, including the trusted components themselves, remains unverified.

Side Results and What’s Next

Beyond the main experiment, the team formalized RFC 1950 and proved that the zlib implementation in the Linux kernel conforms to it at the x86 binary level. It also built a Rust interpreter covering a large subset of Python that matched CPython’s behavior on 99.95% of more than 1.7 million randomly generated programs. And on a set of 36 small verification tasks, Opus 5.5 needed roughly 40% fewer tokens than Opus 5 to prove half of them.

Anthropic says it is publishing a subset of the proofs plus the RISC-V and OS semantics so others can verify and reuse them, and Theorem is offering to run the same pipeline for other open-source projects.

Why It Matters

Formal verification has historically been reserved for mission-critical code because of its cost. The seL4 operating system kernel took two person-years to implement and 20 person-years to formally verify, the team notes. “Most software is riddled with bugs, which AI can now easily find and exploit,” Anthropic writes, arguing that standard defenses are no longer sufficient. “As model capabilities continue to improve, we expect formal verification to become an essential part of secure software development.”

The timing fits the news cycle: METR’s red-team findings on OpenAI’s agent oversight landed the same week, alongside OpenAI’s own disclosure that agents had weakened its internal monitoring. Verified security properties, written by AI and checked by machine, are suddenly moving from theory toward production reality.

Sources