In-depth analysis · About 7 min read
Claude's 123-library proof study makes software guarantees more specific, not universal
Anthropic and Theorem report proofs for 239 fuzzing entry points across 123 libraries, with explicit machine assumptions and limits on what the results establish.
- Published here
- Source published
- Source checked
In an October 22 research article, Anthropic describes work with Theorem that uses Claude to generate machine-checked software proofs. The useful result is a guarantee about selected executable programs under a defined model, rather than a certificate that an entire library or every deployment is secure.
Listen to the news
Read this article aloud or keep listening to the news.
Preparing the reader…
Device voice
Speech is handled by your browser or device. AISubHub does not send the text to an audio API. Voice availability and playback with a locked screen vary by device.
Resuming may repeat a short phrase.
At a glance
- The researchers report no-crash proofs for 239 entry points across 123 OSS-Fuzz libraries, after patching discovered bugs where needed, for inputs below 64 MB under stated assumptions.
- The main experiment concerns single-threaded, statically linked RISC-V programs with musl and modeled Linux behavior. It does not automatically cover ordinary x86 or ARM deployments.
- Separate small-task experiments compare model token requirements and proof sharing. Their reported savings are specific to those experiments, not a general price or performance guarantee.
One proof covers an entry point in a defined machine
The unit of coverage matters. An OSS-Fuzz entry point is a developer-provided way to feed inputs into a program and signal failures. Anthropic reports completing proofs for 239 such entry points across 123 libraries over several weeks. That is neither 239 fully verified libraries nor every possible route through the 123 libraries. The project produced about 180 million lines of Rocq proof code; a typical proof was about 600,000 lines, while the binaries had a median of 18,000 RISC-V instructions.
The claim is bounded by the modeled execution: inputs smaller than 64 MB, a fixed memory budget, one thread, a static RISC-V executable linked with musl, and specified Linux responses and memory layouts. Those conditions define the mathematical statement. AISubHub's reading is that the coverage denominator and build configuration are more useful for evaluating this result than the impressive total number of proof lines alone.
Fuzzing becomes a claim checked by a small kernel
Fuzzing tries many generated inputs and can discover a counterexample, but passing those trials does not exhaust the input space. Here, the researchers turn an existing fuzz test and its compiled executable into a formal specification. Claude constructs an argument that the selected test cannot crash for any permitted input. Assertions already in a test can also encode meaningful properties, such as recovering the original data after compression followed by decompression.
The model does not get to declare its own proof correct. Rocq checks the proof against the statement. Trust therefore shifts toward the exact statement, the RISC-V and Linux semantics, and the checker, including its underlying hardware and software. That is a smaller review target than reading millions of generated lines, but it remains a real dependency. A valid proof cannot establish that an inaccurate machine model faithfully describes a production system.
Why a failed proof can still yield a useful patch
Proof attempts exposed bugs, according to the researchers, including a decompressor problem requiring an unusually arranged input of about 20 MB and an image-decoder failure requiring a 64 MB input and more than 2 GB of memory. These examples explain why a search dominated by small inputs may miss relevant behavior. They are source-reported findings, not vulnerabilities independently reproduced by AISubHub.
The pipeline could patch a program, regenerate its specification, and then prove the patched artifact. A proof of that new artifact must not be presented as a proof of the original unmodified release. There is also a shortcut risk: an agent could remove useful behavior to make a no-crash statement easy. The authors describe patch penalties, incentives to characterize changed inputs, and patch auditing. Those safeguards address a failure mode; they do not remove the need to review whether a fix preserves intended behavior.
Token savings come from two different experiments
For a separate set of 36 small verification tasks, the article says Opus 5.5 proves half the tasks with about 40% fewer tokens than Opus 5. Each task received three independent single-agent attempts with a five-million-token budget, and the chart counted a task when its first attempt finished. This is a task-set and budget comparison. The article does not identify every model used for the larger 123-library effort, so that effort should not all be relabeled as an Opus 5.5 result.
A shared-workspace experiment let agents reuse each other's proofs and reported 17% fewer tokens than isolated work. Its 36-proof chart shows lower token use in 30 of 34 completed solo comparisons; isolated agents did not finish two tasks. Reusing runtime initialization and sanitizer proofs offers a concrete mechanism for savings. Neither percentage supplies a universal dollar cost, elapsed-time improvement, or independently measured deployment benefit.
Other case studies measure different kinds of correctness
The article also describes a zlib experiment against an RFC-derived specification using x86 semantics. That is a separate case study, not evidence that the bulk RISC-V results already apply to all x86 software. A Rust interpreter for a large subset of Python was checked against modeled CPython semantics using Verus, then compared empirically with CPython on more than 1.7 million randomly generated programs.
The reported 99.95% matching behavior belongs to that empirical comparison. It is not a proof of compatibility with every Python program. The distinction matters: a proof establishes a particular formal statement, while a test percentage describes a sampled experiment. Combining these results into one general model accuracy score would erase the different specifications, tools, and remaining corner cases.
The engineering gain is a clearer boundary
The authors explicitly leave gaps. AddressSanitizer misses some memory errors; the project did not cover every entry point; even all fuzzing entry points would not cover all library behavior or dead code. A caller can also misuse a library outside the verified assumptions. Common deployments use dynamic linking, glibc, x86 or ARM, which differ from the main experiment. Some proofs are withheld to avoid revealing unpatched bugs, and the article's broader ecosystem roadmap remains a proposal.
AISubHub analysis: a practical next step is to connect an exact build, test entry point, input bound, and trusted machine model to each claimed guarantee, then reassess it when those inputs change. This can complement testing and review rather than justify removing them across a project. We read the article and its necessary figures, but did not rerun the proofs, audit the external appendix or released repositories, validate patch security, or reproduce token savings. The results and future expectations remain attributed to the researchers.
Why it matters
AISubHub analysis: generated proofs could make a narrow, reviewable software property cheaper to establish and reuse. The actionable output is the exact verified artifact and statement, with their assumptions, rather than a general claim that AI has made software bug-free.
Limits and availability
This is analysis of the researchers' published account, not an independent security audit or reproduction. The main experiment's input, architecture, linking, memory and entry-point limits remain essential; external appendix and repository contents were not audited.
Sources and evidence
Source published: · Source checked:
AI-assisted editorial articles checked against linked primary sources. Vendor claims are attributed; inclusion is not endorsement.