OpenAI has established a nine-member independent advisory group to advise on relations with mathematical research following a communication crisis in mathematics. The group's independence and impact are being questioned by mathematicians.
OpenAI announced it has established an independent group of mathematicians to advise the company and other AI companies on relations with mathematical research, following the reputational crisis caused by its recent results in mathematics. According to the company's statement, the group will advise on engagement with the mathematics community, including how new results are presented and published.
The body, called the Advisory Group on Mathematics and AI (AGMAI), consists of nine members. The members come from institutions such as Stanford, Harvard, Oxford, and Cambridge, and include individuals with multiple Fields Medals and MacArthur fellowships. The group will be hosted by the Institute for Advanced Study (IAS) in New Jersey, one of the world's most prestigious mathematics research centers, where figures such as Albert Einstein, John von Neumann, and J. Robert Oppenheimer once worked.
According to the company's statement, the group will operate independently; it can advise on topics the company has not requested, publicly comment on its impact on mathematics, and disclose its recommendations publicly. OpenAI stated that the group will be given early access to the company's research so it can fulfill its mission. It was stated that members will not be paid by OpenAI and that the group can change its membership structure at its own discretion. The company commented, "The value of the group depends on its members being able to use their own judgment and challenge ours."
Formation and independence of the group
Despite the emphasis on the group's independence, how effective it will be and why it was established remain unclear. Simon Machado, a researcher at ETH Zurich in Switzerland, noted that the process appears "shrouded in mystery" and questioned how the nine members were selected. The group's website states that the body "came together after OpenAI approached some members with a proposal to form an external advisory board," but that after discussions with the company, the mathematicians decided to establish an independent group and invite others. It was not disclosed which members OpenAI initially approached and how the remaining members were selected.
Group member Martin Hairer responded in a blog post to discussions he characterized as a "misinformation flow" regarding the group's formation process. Hairer noted that artificial intelligence is already having a profound impact on mathematical research and raises many questions such as proper attribution of ideas, priority, and human understanding. He argued that it would not make sense for the community to ignore AI laboratories.
Hairer acknowledged that the group was formed as a result of OpenAI's approach, but emphasized that not all members were contacted by the company and that the resulting structure is "truly independent of OpenAI and other leading AI laboratories." He added that members have not signed an agreement restricting their public statements beyond "obvious confidentiality requirements," have received no financial payment, and rely on IAS for technical support. Hairer stated that the group's aim is to represent the interests of the mathematics community as well as possible.
Concerns of the community
The group's small size raises the question of whether it has the breadth of expertise to evaluate all topics that come to it, given the high specialization required to work at the frontiers of mathematics. Francesco Fournier-Facio, a mathematics professor set to join Heriot-Watt University in Scotland, said he is "undecided" about the panel and that more interaction with mathematicians "could make everything better." However, he questioned whether the group fully reflects the community whose work and livelihoods are affected by OpenAI's moves in mathematics, and described the structure as an "ivory tower."
Machado similarly questioned whether group members can fully understand the concerns of the broader mathematics community about artificial intelligence. He noted that the members are "magnificent mathematicians" but that he is not sure whether he would want them to represent him on more political issues. Machado said that the reality faced by mathematicians in this position is very different from that of most mathematicians.
Whether the group can win the trust of mathematicians will determine its effectiveness in limiting the repercussions of OpenAI's recent communication missteps. In his blog post, Hairer stated that it would be "very naive" to believe that members would not distort what is said for the AI laboratories' own promotional purposes. Nevertheless, Hairer argued that avoiding dialogue at a time when the company is making an effort would be hypocrisy. He stated that the group has not yet decided which recommendations it will make, but that there is a clear problem it is responding to.
Hairer said the group is currently gathering input through conversations with colleagues, a public feedback form, and a blog where mathematicians discuss AI's impact on the field.
The challenge ahead
The group is considered to be facing its biggest test. Announcing the advisory group, OpenAI stated that its unpublished new model "solved more than 100 long-open problems in most areas of mathematics." The company had previously said it was approaching another Millennium Prize problem. The group's website stated that this accumulation is the first priority and that it will advise on how to coordinate the publication of numerous important mathematical results reported to have been produced by OpenAI's internal model.
The new initiative is seen in part as an acknowledgment of the communication crisis the company experienced. OpenAI had been accused of appropriating researchers' work and faced criticism for not attributing the human labor on which its models rely and for taking data from those who use its models. These debates led many mathematicians to question whether the company understands the field and community it has entered.
Kevin Buzzard, a mathematics professor at Imperial College London, questioned why OpenAI needs such a distinguished body to understand the community's demands after so much had been written about what mathematicians want. Buzzard said, "What was needed was not proofs of hard theorems, but a better understanding of our subject," and added that it is not clear to him whether a committee of brilliant people is needed to emphasize this point.
