Over the past few days, my Chinese internet has gotten weird.
I spend a probably unreasonable amount of my free time reading and writing about AI safety, so I'm used to seeing AI safety people talk about AI safety. My feed is also obviously biased. Weibo, WeChat, RedNote, and Zhihu all know what I click on.
But this time I started seeing the same conversation in places I didn't expect.
A WeChat article about Jacob Coxon's resignation had taken off. Accounts I'd followed for years for travel, parenting, current affairs, or just random internet things were suddenly talking about whether AI could cause human extinction.
And the comments were even more interesting.
On Zhihu, people were arguing about whether humans would be unable to shut down a superintelligent AI or simply unwilling to. The Three-Body Problem also came up.
“AGI isn’t even here yet, and the surrender faction has already shown up.”
Reply: “Surrender faction? Adventists!”
If you were following the story on X, you probably saw plenty of discussion about Jacob’s resignation and his warnings about AI risk. But X is blocked in mainland China, while conversations on Weibo, WeChat, RedNote, and Zhihu rarely make their way onto the English-language internet. So the discussion that unfolded on the Chinese internet was largely invisible to people following the same story in English.
So I went looking for more of it. I collected articles, posts, and comments across the Chinese internet to see how people were talking about the risks of AI after Jacob's resignation. I looked at three layers of the conversation: mainstream media, tech media and influencers, and individual voices.
I don’t want to tell you what “China thinks” about AI risk. This post obviously can’t do that. I want to bring some of the voices from China into the conversation.
Let's start with the mainstream media. How was Jacob's warning framed in Chinese news coverage?
CLS, a major Chinese financial news outlet, put the question right in its headline:
AI“自我演进”可能彻底失控？Anthropic顶尖大牛愤然离职！
Could AI “self-evolution” spiral completely out of control? Top Anthropic researcher quits.
The article focused on recursive self-improvement: AI helping to build better AI, which then helps build even better AI. It described leading AI companies as caught in an “无序狂飙” (uncontrolled race) to build systems that humans may no longer be able to control.
The Paper, a major Shanghai-based news outlet, framed the story as a move from “越界” (crossing the line) to “失控” (spiraling out of control):
从“越界”到“失控”：Anthropic研究员辞职信发出超级AI末日警告
From “crossing the line” to “spiraling out of control”: Anthropic researcher issues a super-AI doomsday warning.
末日警告 literally means doomsday warning. But the article went well beyond the headline. It explained alignment, power-seeking behavior, models trying to avoid shutdown, and the difference between losing control of AI and humans using powerful AI to cause harm. It also quoted Evan Hubinger, Anthropic’s head of alignment science, saying that he puts the chance of AI causing human extinction within the next decade at above 10%.
Another CLS headline went straight to “毁灭人类” (destroying humanity):
AI有可能“毁灭人类”？业内专家接连发出警告
Could AI “destroy humanity”? Industry experts keep sounding the alarm.
Yicai, one of China's major financial news outlets, put a very different question in its headline:
末日警告还是恐惧营销？AI安全议题刷屏一周
Doomsday warning or fear marketing? AI safety dominates the conversation for a week.
恐惧营销 means fear marketing. The article presented warnings from Jacob, Evan Hubinger, and Paul Christiano alongside arguments that catastrophic AI risks are being overstated. It cited a senior Pentagon official who dismissed recent AI warnings as fear of technological change, as well as Jensen Huang's argument that AI security risks call for better security tools rather than slowing AI development.
National Business Daily, a major Chinese business newspaper, shifted the focus from the risk itself to the race behind it:
“AI毁灭人类”警报拉响！万亿算力资本裹挟下，OpenAI和Anthropic不敢停……
The alarm over AI “destroying humanity” is sounding. With massive amounts of compute and capital pushing the race forward, OpenAI and Anthropic don’t dare stop...
不敢停 is the interesting part. Not don't want to stop, but don't dare to be the first to stop.
The article describes the race as “一场明知危险、却难以单方面停下的博弈”: a game known to be dangerous, but difficult for any one player to stop on its own. Even if a lab takes the risk seriously, slowing down alone means giving up ground to whoever keeps going.
……
The coverage leaves three separate questions: how bad the risk could be, how seriously we should take the warnings, and whether believing the warnings would change what labs actually do.
Now let's move to tech media and influencers. What were they saying about AI risk after Jacob's resignation?
数字生命卡兹克, a Chinese AI influencer who covers frontier models and the AI industry and is also active on X, started with the question:
这哥们谁啊？不会又是哪个营销号吧…
Who is this guy? Is he just another engagement-bait account?
So he checked. Jacob studied mathematics at Cambridge, represented the UK twice at the International Mathematical Olympiad, worked on GPT-4o and GPT-4.5 at OpenAI, and later joined Anthropic's pretraining team.
He framed the stakes as two extremes:
一边，是所有物种最终都熟悉的终点。灭绝。
另一边，却是地球生命从来没有抵达过的地方。永生。
On one side is the ending every species eventually comes to know: extinction.
On the other is somewhere life on Earth has never reached: immortality.
And called AI:
AI这个人类最大的赌局。
AI, humanity’s biggest gamble.
It is a strange kind of gamble: both the upside and the downside could be enormous, and nobody really knows the odds.
字母AI, a Chinese tech and business publication covering major internet and AI companies, focused on where the warning was coming from:
负责踩油门的人，也觉得必须要到踩刹车的时候了。
Even the people stepping on the accelerator think it’s time to hit the brakes.
钛媒体, a Beijing-based tech media outlet covering China's startup and technology industry, focused on the gap between public statements and private fears:
公开讲理性，私下藏恐惧
Rational in public, fearful in private.
It described the continued rush toward more powerful AI as:
草率加速冲向“终局”是一场傲慢的赌博。
Rushing toward the “endgame” is an arrogant gamble.
HappyRock, an independent Chinese technology blog covering AI systems and engineering, turned to the question of who gets to make the bet:
曼哈顿计划造核弹时，有军事管控、政府全程介入、严格的安全协议。
When the Manhattan Project built the atomic bomb, there was military control, government involvement throughout, and strict security protocols.
今天，可能影响全人类命运的AI系统，正在几个创业公司的办公室里、工程师的笔记本电脑上被造出来。
Today, AI systems that could affect the fate of humanity are being built in a handful of startup offices and on engineers’ laptops.
没有沙漠地堡，没有军事管控，甚至没有一个像样的国际监管框架，只有几份自愿签署的安全承诺书。
No desert bunkers, no military control, not even a serious international regulatory framework, only a few voluntarily signed safety commitments.
阿伟的数码日记, a Chinese tech influencer on Weibo who writes about AI and consumer technology, made a similar point without requiring Jacob's prediction to be right:
这个判断未必准确，但问题很现实：涉及文明级风险的训练，不能只由公司内部拍板，至少应先建立可验证的暂停和审查机制。
The prediction may not be right, but the problem is real: training that could involve civilization-scale risks shouldn’t be decided solely inside a company. At minimum, there should be verifiable mechanisms for pauses and review.
……
How dangerous could AI become? Who gets to decide whether that risk is worth taking?
Now let's move to the comment section. What were people actually saying?
I mostly looked at comments under 数字生命卡兹克's WeChat article, and the answers and comments under a Zhihu question asking what people thought about Jacob's resignation and his claim that AI companies were “betting the fate of humanity.”
Not everyone was convinced AI could get that far in the first place:
“Large models may keep getting better, but they’ll never surpass humans. Under the Transformer architecture, true intelligence is impossible.”
Reply: “What if a Transformer-based AI helps humans create an AI with a different architecture?”
And what if we eventually figure out how intelligence itself is created?
“An ordered arrangement of different elements somehow produced life, and then intelligence. That's fascinating. If we could figure out that mechanism, we'd practically become gods, able to create whatever we wanted.”
Others thought AI wouldn't need to “wake up” at all:
“AI doesn't need to ‘wake up’ and become an independent agent before it starts to dominate humans. As long as we become increasingly dependent on it, it has already gained the ability to influence us. This seems irreversible.”
Others were worried about a much more ordinary failure mode:
“A lot of people are still focused on the fact that previous incidents were minor and conclude that AI risk is alarmism. But the failure mode is already pretty clear: humans hand a task over to AI, don’t audit what happens in the middle, and only discover the problem after something goes badly wrong.”
“Saying ‘we still have control’ misses the point. Most applications are adopted precisely to save human labor, so people aren’t going to spend enormous effort auditing every step.”
We use AI to save human labor. But careful oversight requires putting human labor back in.
The nuclear analogy came up too:
“This reminds me of the first atomic bomb test, when scientists worried that the bomb might ignite the atmosphere and destroy the planet. The probability was thought to be low, but if it happened, it would be fatal. They were uneasy, and they pressed the button anyway. How similar is that to today?”
The reply turned from the bomb itself to the arms race that followed:
“After the world saw the United States build the atomic bomb and hydrogen bomb, other countries rushed to develop their own. So if we’re going to talk about climbing the ladder of intelligence, maybe we should also ask whether our institutions can adapt to the arrival of AI. If AI can be used for mathematics and scientific research, why can’t it be used to study political institutions? Maybe this is something that should be left to states.”
But leaving the problem to states raised another question:
“If we think of human organizations as a kind of artificial intelligence, their value alignment is hardly reassuring either. Compared with a superintelligence causing catastrophe through goal optimization, a powerful institution created to regulate AI may actually be more dangerous. History has already given us plenty of examples.”
Who aligns the institution powerful enough to align AI?
Even the choice between extinction and immortality got challenged.
“Tim Urban puts immortality and extinction at opposite ends of a balance beam. I wonder if they’re less like two ends of a beam and more like two sides of the same coin. If a species is permanently preserved by a superintelligence, has it achieved immortality, or has it become a museum specimen of its own civilization? Like the dodo: it lives forever in a natural history museum, in a way that is completely dead.”
……
The deeper I went into the comments, the harder it became to say what the debate was even about. People weren't just disagreeing on the answers. They were asking different questions.
I've seen AI safety arguments make their way onto the Chinese internet before. Usually, not much happens.
This time, people paid attention.
My guess is that this story had unusually low translation costs. OpenAI is already widely known in China. And the warning wasn't coming from someone looking in from the outside. It came from someone who had helped build frontier models at both OpenAI and Anthropic, and then decided to leave.
You don't need to know what alignment is to understand that story.
Watching all of this unfold made me curious about what would happen if some of these conversations crossed the language barrier more directly.
So Althea, Tykoo, and I are planning some private discussions between researchers in the US and China.
If you'd like to join, DM any of us!