Tao Warns Good Math Problems Are Running Out

What Thom objects to is that the single sentence collapsed two different questions into one. He had explicitly separated them in his email: whether the conversations entered training data, and whether they were reachable during the solving process. The answer did not say which of the two it was denying, and no basis or explanation was attached. In a record he posted on Mastodon on September 9, Thom wrote that the reply was "at minimum, unjustifiably broad and materially misleading; looking back it was plainly dishonest."

What OpenAI said about a different case during the same period had a different texture. On the Tristan Buckmaster and Levent Alpöge matter, the company said no specific user data was accessed, while adding that it cannot rule out that de-identified data derived from usage of its products helped improve its models. Thom argued that the company erased in its answer to him the very distinction it was drawing in that case.

Terence Tao then added a diagnosis at a different level. Writing on Mastodon on September 8, he said the stock of good open problems is being mined in a non-renewable fashion. Depletion may sound strange when problems can be generated without limit, and Tao put it this way: "a country or region can suffer a critical shortage of drinking water while simultaneously being surrounded by a massive ocean." One can ask for the 10^10^10th digit of pi at will, but the overwhelming majority of such problems teach nobody anything.

Tao wrote that working out which questions are worth pursuing is itself a lengthy and subjective process. Knowing the difficulty landscape, which problems yield to known methods, which take real effort, and which are impossible, is central to that judgment. Better tools flatten that landscape, and normally the flattening is offset because the reachable radius widens and new frontiers appear.

What marks the current AI era, Tao pointed out, is the absence of those frontiers. There is no clear line separating the problems AI can reach from the ones it cannot, partly because the technology changes quickly, and partly because companies decline to disclose their negative results or reveal how they arrived at their solutions. The consequence, he wrote, is that "it is now the identification of a promising problem which is the scarce and precious resource."

The heaviest passage comes next. Tao wrote that he has already seen the mere rumor of someone working on a problem trigger a massive amount of AI-powered effort that flattens it before the original research has time to mature. Incentives, he argued, are therefore tilting toward no longer sharing promising research directions with the wider community, which would reverse centuries of open science tradition and do serious long-term damage to the future of the field.

Seen through a legal lens, the crux of this case is who has to prove what. Thom said he disabled model training on June 29, but that switch is a promise users cannot audit, and it applies only going forward. It answers nothing about conversations that happened earlier or derivatives already extracted. OpenAI's reply mentioned neither the setting nor any account-specific check. Thom wrote that users should not have to reverse-engineer an internal training pipeline, and that the company is the only party holding the relevant data.

How far the word de-identified reaches sits in the same place. Thom wrote that "de-identification may remove a name; it does not remove the intellectual content of a mathematical idea." Strip the name and the privacy question is tidied away, but the idea itself remains and becomes the next model's capability. He noted that his and Kun's approach was not the main line of attack on non-soficity, with more promising routes running through quantum games, which is why the model's detailed command of that particular path deepened his doubts.

On September 10 Thom posted again and moved the problem toward institutions. Intelligence of this kind, he wrote, should not be treated as an ordinary consumer product whose terms are set entirely by a few companies. His target is a structure in which a provider collects people's ideas and information, controls all the evidence about how they were used, and then turns that advantage back on its own users. Once intelligence becomes basic infrastructure for society, his conclusion runs, it belongs under public duties and independent oversight comparable to those governing water or electricity.

The Tao thread METAL reviewed runs to four parts, and in the last one he conceded that a complete ban on automated solution-extraction tools is technically infeasible. Instead he proposed designating classes of problems where a solution alone is not the point, and where the work is expected to yield insight from the solving process and teach something about the difficulty landscape of nearby problems. He compared it to a modern food donation drive, which no longer accepts any contribution merely because it is technically edible and instead maintains explicit standards. METAL reported earlier on the priority dispute that broke out over OpenAI's claim to have solved a hard problem and on Buckmaster's account of the phone call he received the day before that announcement, and this week is where those individual disputes moved into a question about the rules of the whole field.

What is established so far is that Thom asked in two parts and OpenAI answered in one, and that the company left open, in a different case, the possibility that de-identified derived data contributed. The material that would show how the training pipeline actually behaved sits with the company alone. Two things are worth watching next: whether OpenAI backs its denial with product settings, datasets and checkpoints, and whether mathematicians actually begin keeping unpublished ideas out of commercial models.