Back in fall of 2024, Terence Tao, perhaps the most respected living mathematician, was eager to learn about what AI could and could not do. Although he was skeptical of the then recently-released o1, he was optimistic about a future in which humans and machines co-existed. Matteo Wong had great interview with Tao about this in The Atlantic:
Wong spoke to Tao by phone and summed up Tao’s openness to new futures this way:
Lately, as I noted in a Substack here in August, Tao has been raising questions. At the end of July he gave a great lecture, with this among his slides.
He’s clearly been thinking about this ever since. By now, it is clear that his views have radically shifted.
Three posts of his from the last two days illustrate:
The first (yesterday) notes that solving puzzles is not the same as coming up with new insights:
The second (also yesterday) is in some ways an argument about good intellectual taste, and expresses deep concerns about the consequences of AI for math.
It is also a plea for transparency:
The final one (all appeared on mathstodon) came out today and appears to allude to the OpenAI-NYU-Anthropic Navier-Stokes controversy, in which OpenAI rushed to scoop Alpöge and Buckmaster, spending $22.5 million in the process, possibly using their data. (In vague, evasive words OpenAI wrote that “we cannot rule out that de-identified data derived from their usage of our products helped improve our models.”)
Tao starts by again discussing the question of good intellectual taste, as background. As a scientist who has worked in other areas, I completely resonate with his opening framing.
That last sentence is a truly dire warning, about a potentially tragic world.
§
And indeed, as Tao implies, there will be fallout in other fields as well.
Fat chance of AI “curing” cancer if nobody trusts the AI companies not to steal their IP. These words from OpenAI’s Chief Research Officer are hardly reassuring:
Mark Chen@markchen90
Two things to distinguish: Did any human or agent look at user data as part of the Navier Stokes effort? No. Do we use user feedback and de-identified data to improve ChatGPT and Codex in a holistic way? Yes. And so does every LLM company.
levent @__alpoge__
“we cannot rule out that de-identified data derived from their usage of our products helped improve our models.” i mean props to them for straight coming clean. (so far the proof looks more along the lines of another euler blowup proof we had, off of whose ansatz naming we were
7:02 PM · Sep 8, 2026 · 227K Views
293 Replies · 53 Reposts · 1.28K Likes
As I noted
§
I have no idea what the solution is here. But I desperately hope that Terence Tao’s warnings about how all of this might impact science will be heeded.
Just I was finishing up, this just came in, from Jacob Coxon, who just left Anthropic.
I don’t find it quite as compelling, and it is focused on future harms with too little discussion of current harms, but it is spreading like wildfire and worth reading.
I happen to disagree with him around timing, and I would argue that Coxon is exaggerating what AI is likely to do anytime soon, But it is nonetheless overall a disconcerting (and plausible) firsthand perspective on the transparently self-indulgent and dangerous thought processes in two of the leading frontier labs:
Anthropic’s Alignment Science lead wrote this
Evan Hubinger@EvanHub
Jacob is correct here—we really do earnestly believe AI could kill all humans! I personally think it is >10% within the next decade. I believe Anthropic is trying its best, but we do not yet have a plan to solve alignment for superintelligence and are not clearly on track to.
Jacob Coxon @hilbertspaess
The people building AI earnestly believe that it could kill us all by the end of the decade. This is not a marketing stunt. If anything, many executives and senior researchers will couch their phrasing in the press to sound sensible - but I hear the same people express fear
1:27 AM · Sep 9, 2026 · 37.6K Views
45 Replies · 53 Reposts · 516 Likes
I can’t say I find this comforting.
Although I don’t think extinction is likely, catastrophic harm is certainly possible, and it is indeed clear that nobody has a serious plan. The distinction of “getting there first” is probably fairly irrelevant, if others will soon follow.
Perhaps the only thing that could really help here on the technical side would be a different foundation than LLMs (which continue to seem utterly incorrigible) and neither company seems to be taking that notion seriously.
§
The end of open science? Or worse? Neither scenario is pretty.