This famous 2024 essay by Dario Amodei makes the case that “powerful AI” could bring wondrous benefits to society. Powerful AI is defined as a technology that can interact with the digital world in all the ways that a human can but is universally more competent and intelligent than any human, such as (for example) a Nobel Prize winning scientist. What’s more, powerful A.I. will also be faster than humans, producing output at a 10-100x rate per unit of time, and we will be able to operate millions such A.I.s on the infrastructure that performed the original training. This essay is the source of the famous phrase “country of geniuses in a datacenter”.
The main appeal of powerful A.I. is that it could dramatically accelerate scientific progress. Amodei models scientific progress as a series of incredibly important discoveries that currently happen at the rate of ~1x a year. In the realm of biology, that’s stuff like CRISPR gene editing, mRNA vaccines, new microscopy techniques, etc. If we imagine that we had another 75 such discoveries waiting for us in the 21st century, Amodei proposes that our A.I. geniuses could help us find them all within just 5-10 years. It is possible that A.I. could make discoveries at an even faster rate, although overall rate of progress is limited by non-genius factors like running physical experiments and managing regulatory issues. Still, with massive parallelization of research, we should get a lot of important discoveries in a very short period of time.
The most obvious application of these A.I. discoveries would be to cure disease (of the mind and body.) Amodei notes that we have already made lots of progress in eradicating and slowing down various kinds of diseases, and it’s plausible that we would have solved almost all of those problems by the end of the 21st century anyway — what if A.I. did it even faster? It is even possible, Amodei says, that we could dramatically extend the human lifespan.
This part of the essay is the most compelling, which is probably why it comes first. Dario’s prediction that we might have powerful A.I. as soon as 2026 seems basically correct, give or take a year or two; the skeptics were wrong. A.I. is smarter and faster than professionals at many types of knowledge work already, and seems on track to continue that progress for some time. And the application to “massively parallelizing scientific research, with an eye towards curing disease” seems quite plausible and good.
If you were looking for the best argument for expediting superhuman AI, this is it. This is the argument, laid out by Dario himself. Internalize it.
The essay begins to drag on a bit as Dario dwells at length on more speculative benefits to the economy, climate change, inequality and democracy. In each case, one can just as easily see how a malicious or mishandled AI might instead make these problems worse, and Dario calls for some radical action like a coalition of democracies that tries to coerce authoritarian countries into new modes of government by withholding AI.
This part of essay might also be the most important. Disregard the motivated reasoning that hopes that A.I. might cure all ills. Instead, look at the complicated ways that A.I. is going to interact with the world order, and consider Dario’s guesses as to how the world might shift to accommodate it. I think most A.I. related changes have been of this variety so far: not step change shifts, but complicated twists in the fabric of society that were perhaps predictable with hindsight. For example, academic assignments and remote interviews are basically meaningless now because A.I. can one-shot anything. I think we obviously will not see universally good or bad outcomes for democracy, the economy, inequality, and climate change. (Although on net, I think it will tend to make most of these worse.) Overall though, I think all of these areas will get much more complicated and introduce a raft of new second-order effects and policies to compensate.
Finally, Dario gets to the bad news, what we’ve all been worried about this whole time. If AI is better than humans at everything, what happens to our sense of meaning, and what happens to our livelihoods?
On the loss of meaning: Dario is not worried about it. As he says, most of us aren’t the best at anything we do, and that doesn’t bother us. Meaning mostly comes from relationships anyway. I think there is a little sleight of hand here – Dario gives examples related to how he enjoys his part-time hobbies, but also implies that A.I. will take everyone’s jobs, and I think people get a sense of meaning from being useful that will not be so easy to replace.
On the loss of jobs: Dario twists the knife. At first, A.I. might only be better at 90% of work, so people will figure out how to split the final 10% to get more leverage. But in the long run, he thinks A.I. will truly be better at 100% of knowledge work, to the point where humans simply have nothing to contribute. And while the famous comparative advantage essay points out that as long as there’s a shortage of A.I. to go around, we can still produce value doing the less important stuff, Dario also thinks this is a temporary state. Eventually, he says, there will be so much A.I. and it will be so powerful that there will simply be no role for humans in the economy. And in that world, we’ll have to shift to an entirely different economic model, perhaps universal basic income, perhaps something else (he does not project much enthusiasm for any of the options.)
To be honest, I still find the idea that “A.I. will replace 100% of the economy” to be kind of unrealistic. Consider this thought experiment: suppose A.I. runs its own economy but the returns are not equally distributed (as is certain to be the case) so that most people are poor (as seems likely to me.) In this case, we won’t be able to buy what the A.I. is selling anyway, so we’ll return to some form of barter and build our own economy on the side. In the worst case I suppose we could end up in a Matrix- or Terminator- like society where we are hiding in caves from a society of malicious robots. Dario does not address this: he says he wants to avoid bringing too much science fiction into the discussion.
Every day I feel that the science fiction is already here.