OpenAI withdraws three AI math papers over sign error

OpenAI has pulled three of the 722 mathematics manuscripts it published on Oct. 6, less than a day after posting them. The company found a sign error that broke the argument in one paper and the construction that two other papers depended on. It also revised 14 more manuscripts. The withdrawals have added to debate among mathematicians over how to check AI-generated research released in bulk.retractionwatch+1

The 722 manuscripts were posted to a public GitHub repository. OpenAI said they were produced by an unreleased internal frontier model and grouped them into 372 families of results in algebra, geometry, computer science and other fields.unite+1

What was withdrawn

According to the repository's Oct. 7 revision history, the error was in "Algebraicity of Weil classes on split abelian eightfolds." There, "a sign error invalidates a stabilization-trace cancellation argument and the construction used by two dependent papers". Those two papers, "Algebraicity of Kuga–Satake Correspondences for K3 Surfaces" and "The rational Hodge conjecture for products of K3 surfaces," were also withdrawn.aiweekly+1

The withdrawn versions are still online, with notices explaining the gaps. The 14 revised manuscripts received proof repairs, corrected statements, clearer hypotheses and dependencies, and one fix to an outdated citation. References were also updated in 13 companion papers. The catalog now lists 719 manuscripts.implicator+1

An OpenAI spokesperson told Retraction Watch that the company found the errors during an audit. "We welcome scrutiny and feedback from the mathematical community," the spokesperson said, adding that OpenAI would "withdraw papers if no fixes can be found". Another spokesperson told The Atlantic that "we expect mistakes to be quite rare". Dan Roberts, an OpenAI research lead, announced the withdrawals on X.retractionwatch+1

Verification concerns

About 42% of the results come with formal proofs in Lean, a programming language that checks proofs by computer. The rest exist only as written papers. OpenAI said an advisory group it works with, the Advisory Group on Mathematics and Artificial Intelligence, recommended releasing the results without waiting for full formalization. The spokesperson said roughly half were released unconfirmed.asiae+1

Alex Townsend, a Cornell University mathematician, said he expects more errors to turn up. "Given the skepticism of AI in the mathematics community, I think that OpenAI should have announced the manuscripts that were lean verified first," he told Retraction Watch. Several mathematicians told The Atlantic that some papers appear not to prove what they claim.retractionwatch+1

Baek Hyeongryeol, dean of KAIST's Graduate School of AI Mathematics in South Korea, said some of the results could be "worthy of a Fields Medal" if confirmed. He also said the manuscripts he read "were difficult to follow".asiae

"Whether a proof is correct is important, but the process by which it becomes part of our knowledge is just as important," Baek said.asiae