722 Proofs, Ten Volunteers: The Verification Gap in AI Mathematics

722 Proofs, Ten Volunteers: The Verification Gap in AI Mathematics

OpenAI released its accumulated answers to 722 unsolved mathematics problems on 6 October, packaged as more than 350 separate write-ups. Four days later, a group of mathematicians opened a repository to do the part the company had skipped: making those write-ups readable, checkable and citable. Two and a half weeks on, the gap between the two efforts can be measured, and the measurements are the most useful thing in the story.

<em>Photo: Natecation, CC BY-SA 4.0, via <a href=
Photo: Natecation, CC BY-SA 4.0, via Wikimedia Commons — Terence Tao teaching analytic prime number theory in January 2025.

What the numbers say this morning

As of this morning, OpenAI’s openai/math repository, opened on 6 October, shows 15,482 stars, 1,812 forks and just five commits from three contributors, the last of them on 12 October. The community’s oai-math-revised effort, opened on 10 October, shows 92 commits from ten contributors, 22 stars and one open issue, and was still being pushed yesterday. The dump collected roughly 700 times more stars than the cleanup, while the cleanup proceeds at about 6.6 commits a day.

That ratio is not a complaint about the volunteers. In their own announcement, hosted on Terence Tao’s blog, they describe the material plainly: OpenAI’s write-ups “can be difficult to understand and can have incomplete citations”. The ratio is a measurement of how lopsided the incentives are. Generating text at this scale takes a swarm; checking it takes a profession.

Why the write-ups resist reading

A paper submitted to arXiv on 21 October supplies the uncomfortable explanation. Thomas Hikaru Clark, Matias B. Martin and Kyle Mahowald compared AI output with human writing across nine psycholinguistic features known to predict processing difficulty. All nine frontier and near-frontier models they tested used more inanimate subjects with animate verbs and more low-frequency n-grams than human writers did, and most also produced longer syntactic dependencies and more stacked nouns. Their hypothesis is that human language is optimised for successful communication, while AI text is increasingly optimised for information density in the service of maths, code and reasoning.

Set that beside Tao’s account and the picture is coherent. The UCLA mathematician told The Observer on 23 October that the current crop of models do not care “how elegant the solutions are, or who understands them, or what to do next”. He frames the release as a race: when OpenAI’s contacts heard that Anthropic might be close to a Millennium Prize problem, it “became code red priority number one”, a swarm of some 10,000 agents was pointed at the Navier–Stokes problem, and premium sign-ups were suspended for days so the compute could be redirected. What followed was the disposal problem he calls toxic waste: “You can sit on it forever, or you can just dump it out all at once, or you can dole it out in drabs, or you can pay people to process it.”

<em>Photo: MathKnight, CC BY 2.5, via <a href=
Photo: MathKnight, CC BY 2.5, via Wikimedia Commons — a mathematics lecture on algebraic topology at Tel Aviv University.

The arithmetic of checking

Ten thousand agents running for 88 hours is 880,000 agent-hours of generation, or about 1,219 agent-hours for each of the 722 problems, which is roughly 51 agent-days per problem. Those 722 problems were compressed into 350 findings, an average of about two problems per write-up. The Navier–Stokes argument alone is said to run to 165 pages; if every write-up were that long, the October release would be some 119,000 pages of proof text. Tao told The New York Times the argument is “technically readable” but that “no expert I’ve talked to has actually understood the whole argument”, and he has since downgraded his verdict from “toxic waste” to “maybe treated waste”. OpenAI has said it neither withheld nor rushed out results; Anthropic, by contrast, is paying mathematicians to write up results from its own trove of open questions.

Human verification does not scale at agent speed, and that asymmetry, rather than the quality of any single proof, is the part of this story that will still matter next year. Benchmarks have become the profession’s goals, as Tao puts it: “Maths problems are almost the only thing left for them to train on, so suddenly all the big prizes and goals in our profession are benchmarks for them.”

Who should not rely on this

If you need a result you can cite or defend — a paper, a patent, a safety case — this release is not usable as it stands: the gap between “technically readable” and “verified” is the whole value of a proof. If you want a map of unexplored territory, or candidate directions to check yourself, the same dump is the most detailed one on offer. The answer flips on whether your bottleneck is ideas or liability.

What to watch

  • Whether the community hub keeps its pace. It has about six commits a day from ten people against 119,000 pages of extrapolated proof text; any week where that stops is a week the material ossifies into something nobody re-derives.
  • Whether the Navier–Stokes write-up survives expert review, errors and all. Reviews already suggest there may be mistakes, which makes it the test case for how the profession handles AI results in bulk.
  • Whether other labs copy Anthropic’s alternative — paying mathematicians to write results up properly rather than dumping them — or keep treating cleanup as volunteer work.
  • Whether the readability findings change model design. If information density is the cause, then the next generation either gets easier to read or the gap between what models prove and what people can absorb keeps widening.

Sources: The Observer; Terence Tao’s blog (oai-math-revised announcement); Terence Tao, “AI in mathematics (and beyond)”; arXiv:2610.29162, “Why is AI-generated language so hard to understand?”; repository figures measured by this site on 25 October 2026 via the GitHub API (openai/math, oai-math-revised).

Leave a Reply

Your email address will not be published. Required fields are marked *