OpenAI has published 722 papers on 372 open problems, provoking both interest and a sharp reaction in the scientific community. The materials were created by an internal experimental model and cover areas such as geometry, algebra, mathematical logic and theoretical computer science. The results are presented as full or partial solutions, but a significant part of them has not yet been independently verified.
The company published the papers on October 6 in a GitHub repository. They have been supplemented with formalizations of many of the proofs in the Lean language - a system that allows logical steps to be checked by a computer. OpenAI has also included 10 more summaries of the model's reasoning, estimates of the computational resources used and data on the number of tasks attempted. According to the company, the average result required a resource equivalent to approximately three hours of work with ChatGPT Pro.
Three Manuscripts Retracted
Less than 24 hours after publication, OpenAI has retracted three papers due to a sign error. According to Retraction Watch, the error invalidated an argument in one paper and a construction used in two other manuscripts that depended on it. The company has also reworked 14 other papers with corrections to proofs, clarified assertions, hypotheses, and dependencies. OpenAI says it discovered the issues during an internal audit and will continue to update the repository with corrections and new formalizations.
Retraction Watch quotes a company representative as saying that about 50% of the results were published without full verification. OpenAI says it welcomes peer review and will withdraw manuscripts that cannot be fixed.
Mathematicians debate publication policy
Nature described the publication as unprecedented in scale and reported that reactions ranged from praise for a major technological breakthrough to concerns that hundreds of unreviewed papers were being dumped on researchers at once. The published series does not include solutions to the remaining five outstanding Millennium Prize problems. It comes a month after OpenAI announced a proposal for a solution to the Navier-Stokes problem, which is among the seven problems on the list.
The Association for Human Mathematics has called on mathematicians to stop working with OpenAI. In its position paper, the organization said: “Publishing over 700 files at once is not a demonstration of science, but a demonstration of power.“ It insisted that the results be evaluated with particular care and questioned the publication model, in which human verification begins after the presentation of a huge number of machine-generated texts.
OpenAI says its goal is for the results to accelerate the development of mathematics. The company plans workshops, conferences and special programs dedicated to understanding the results, and future publications should contain better explanations, citations and descriptions of dependencies between evidence. Until then, the value of individual solutions will depend on independent verification by the mathematical community.
Sources: news.sky.com, nature.com, retractionwatch.com