In 2021, USAID authorized a five-year, $125 million budget for a program called Deep Vision (under the tortured acronym “DEEP VZN,” which hurts my eyes, so we’ll keep it phonetic). If some omniscient force tracks humanity’s worst ideas, this one’s right near the top of the list. Deep Vision meant to pursue three main activities – connected in ways that verged on a suicide pact for humanity:
#1: Virus hunting. Deep Vision would recruit local teams in a dozen developing countries to scour remote places like bat caves and rural bush-meat markets. Their quarry: thousands of undiscovered viruses of completely unknown deadliness. They’d extract these gems and haul them back to dense population centers, and store them in leaky, unreliable vessels called “laboratories.”
I say this because some of the world’s top labs have fatally leaked some of history’s deadliest agents. Indeed, all classes of laboratory routinely leak – and at completely unknown rates, due to a broad lack of reporting requirements, and opacity surrounding those that exist. This makes undisturbed caves much safer resting places for pandemic-grade viruses than labs staffed by flawed humans. But Deep Vision was so unworried about this that the word “leak” appears just once in its 114-page program description – and in a context unrelated to lab leaks (whereas the word “gender” appears 34 times, and “climate” over 50).
#2: Characterization. Deep Vision would then conduct a series of experiments to determine which of those viruses were most likely to be true weapons of mass destruction. As in, pandemic-grade monsters that might even make COVID look wimpy. They’d conduct as many experiments as possible locally, then ship the spookiest viruses home to be poked and prodded by flawed humans in American labs.
#3: Global Fame! Deep Vision would rank order the viruses by most-terrifying potential, and publish their genetic recipes to the entire world. That’s right: Publish their genomes to the entire world. A world containing roughly 30,000 people with the tools and know-how to then generate those viruses from scratch. And to be clear, by “viruses” I don’t mean their inanimate DNA. But actual live viruses – the worst of them ready to kick off a pandemic the moment they lodged in the right set of lungs.
So, in “success,” Deep Vision would have given 30,000 unvetted strangers the killing power of a small nuclear arsenal. Strangers spread across dozens of countries – almost surely including islands of stability like Iran, Pakistan, and North Korea.
Deep Vision is on my mind because Sam Harris just interviewed me about it for his Making Sense podcast. I learned about the program shortly after it was authorized, then did my best to blow the whistle on it in early 2022. I did this by interviewing a brilliant MIT scientist named Kevin Esvelt, who knew all about the program. A story we fully lay out in episode #62 of my own podcast, After On.
After we recorded that, Sam Harris reposted my episode to his much larger audience. And soon enough, a group of folks who knew their way around Washington reached out to me. I tell a semi-redacted story about what happened next in my appearance on Sam’s show, and I won’t repeat it all here. The Cliff Notes are that a loose alliance of people from an enormous range of political backgrounds quietly collaborated to terminate Deep Vision before it could do the ghastly damage it was set to unleash.
So, we whacked that mole a few years back. But the ground beneath us is now starting to quake from a mosh pit of its pissed-off cousins, in the form of emerging biothreats at the intersection of AI and synthetic biology (or “synbio”). The Deep Vision story is urgently relevant to our current moment, for both obvious and subtle reasons. So let’s dive into it.
It’s impossible to know how many pandemic-grade viruses Deep Vision would have found. There’s a small chance it would have found none. But a high single-digit number seems conservatively likely according to various analyses. Also, USAID had lots of virus-hunting experience, and would not have funded Deep Vision if they thought failure was likely. So for the sake of our exploration, let’s suppose Deep Vision found seven pandemic-ready bugs. This would have hyper-enabled three groups of extremely bad actors:
#1: Lone Wolf Mass Murderers. The world produces these people in dismaying numbers. The maximalists among them kill as many strangers as possible, and die in the act. Victim counts are primarily a function of their weapons. China mostly produces mass stabbers, because guns are heavily controlled there. They tend to kill far fewer people than mass shooters, who often kill dozens. Which is far less than the hundreds who can die when commercial pilots deliberately crash their planes. Again: death toll is mostly a function of weapons. So can we please not add pandemics to their arsenal?
I doubt Deep Vision would have armed one of these lunatics immediately, because their prevalence seems to be lower than 1 in 30,000 people. But the population capable of conjuring live viruses would not have stayed frozen at that level. The relevant technique is called “reverse genetics” or “viral rescue.” It was first achieved in 1999, and by 2005, perhaps a hundred people could pull it off. Kevin Esvelt ballparked its practitioners at 30,000 at the time of our 2022 interview, and the biologists I double-checked with said this sounded right. This means the population of concern grew by over two orders of magnitude in just over 15 years.
It will continue to grow by more orders of magnitude. Particularly now that AI, cloud labs, and other factors are dropping the barriers of education, equipment, budget, and permission necessary to achieve certain outcomes. When the empowered population reaches 300,000, we’ll be in a much scarier place. Because this is the rough global count of large commercial jet pilots – one of whom becomes a top-grossing mass murderer every five-ish years (with seven deliberately downed jets since 1994 – not including 9/11). As for 3,000,000, a random cross-section of that many Americans will produce about six mass shooters each and every year. This illustrates the first critical rule for managing biorisk in the age of AI, which is:
The passage of time is the ultimate threat multiplier.
Deadly information that’s made public today will be permanently available, because the internet never forgets. And the population that can act on it will eternally compound. So don’t be an idiot about what you share with the world. And assume that lethal skills calling for post-docs today will soon become native to Freshman Biology.
This doesn’t mean we should stop practicing synbio, or melding it with AI. Because it can bring us countless benefits, plus there’s just no stopping it. This makes it an inexcusable failure of imagination to focus on how little bad actors could do today, rather than how much they can do tomorrow. Lose sight of this, and our bio infrastructure will be turned against us, just as our jets were on 9/11.
Which brings us to the crucial fact that we could have had global aviation without 9/11 – because 9/11 was optional. An option chosen by a system too lazy to see the obvious vulnerabilities available to sophisticated adversaries. A system too stingy to harden cockpit doors, or heighten airport security for decades. A biological 9/11 could be thousands of times worse than its progenitor – and perhaps even cancel humanity. We must accept this fact today. Not after the attack has started.
#2: Chaos Agents are another nasty group that Deep Vision would have hyper-empowered. This is my label for entities that might have had no real intention of launching a pandemic – and perhaps not even the ability. But which Deep Vision would have nonetheless empowered to cause extraordinary damage.
As an example, imagine that Iran made the following announcement sometime last year. “Hello world! We’ve been following the global scientific crusade to dig up and publicize the world’s most dangerous viruses – and we’ve been testing the most promising ones on our countless political prisoners. And damn, one of them has a 20% fatality rate! Plus, it’s super contagious in our dungeons. We are now storing it in every one of our embassies, with an extra-double helping at our New York mission to the UN. We have no desire to ever unleash this plague. But if the Great Satan threatens us, we will have no choice. Which would actually be fine, because we all just want a martyr’s death anyway.”
If Iran proclaimed this today, they’d be taken as seriously as an eighth grader saying he’s planted a hydrogen bomb in the cafeteria, because no one alive is (currently) capable of designing a virus like that. But in a Deep Vision world, this threat would be chillingly credible – and completely unverifiable, because we don’t infect people with viruses to see how deadly they are. Of course, they could be bluffing. But even if our spies were sure of this, they’re the same people who swore that Saddam Hussein had weapons of mass destruction.
This is a nuclear-grade deterrent that almost any rogue state could claim with at least partial credibility. Every narco gang and terror cell could meanwhile dream up similar threats, so as to hijack the world’s attention, spread panic, tank markets and demand ransoms. This geopolitical havoc would be enabled even if Deep Vision found zero pandemic-grade viruses, because only the most vicious groups would be able to probe its findings.
#3: Omnicidal Groups are the scariest class of actors that Deep Vision would have hyper-empowered. These aren’t standard terrorists, who want to kill these people, but not those people, and thus, don’t want to kill all people – but groups animated by fringe beliefs that seek human extinction. They could be eco extremists, animal-rights zealots, or antinatalists who believe that human life is so net-painful that it would be a great mercy to extinguish us all. Or they could be religious zealots, who can believe all kinds of imponderable things.
The Japanese death cult Aum Shinrikyo embodied this. They sought a purifying Armageddon, which their sect would magically survive. Active in the 80s and 90s, their lunacy peaked when they released sarin gas in the Tokyo subway, killing over a dozen and injuring thousands.
If Aum were around in a Deep-Vision world, they could have threatened civilization itself. I know that’s a lot coming from a sci-fi writer, but hear me out. Aum had over a billion dollars in assets and over 50,000 followers. There’s footage of their leader addressing packed stadiums in Russia, where they were wildly popular. Their massive arsenal included loads of guns, nerve gas, and a Russian attack helicopter, which they somehow smuggled into Japan. They also had a deep bench of scientific talent – with over 300 members of their internal “Science and Technology Ministry.”
So how would Aum navigate a Deep-Vision world? Since characterization experiments can’t be perfect, it’s unlikely that pandemic-grade viruses would neatly land in the top slots of the ranked list of scariest finds. They’d more likely be scattered amongst the top thirty or more. A group with Aum’s resources could have easily generated every virus at the top of the list, to be sure they had everything – then set all of them loose in a dozen international airports simultaneously.
Imagine seven separate pandemics breaking out everywhere, all at once. Doctors wouldn’t know who’s sick with what, and many could have multiple infections. The combined fatality rate could be staggering – especially if cross-infections were deadlier than lone ones. This is where civilization could topple. Because with seven pandemics raging at once, you could end up killing your whole family just by walking around the block. And who would risk that for any sum? This would be the end of frontline workers, which would be the end of food, law enforcement, electricity, and communications. And that would quickly turn any society into a hellscape.
Of course, you’ll find no group actively seeking to cancel humanity today, however much you scour the web. But this was true back when Aum Shinrikyo became one of the first cults online, because this isn’t the sort of goal people tend to broadcast. So with thousands of cults active today, who can say what stretch goals the darkest ones harbor? You also won’t find many online voices calling for colonies on Jupiter, because people rarely organize around impossible dreams. But if Deep Vision made humanity’s worst case possible, you can bet that some people would start thinking about it – and that a few of them would start organizing.
So what the hell were they thinking?
In our original interview, Kevin and I came up with every steelman argument we could think of in favor of Deep Vision, to present the most balanced picture we could. A seemingly strong one was that the program might let us develop vaccines before pandemics hit. The trouble is, you’d have no idea if any vaccine would work – because we test them by inoculating people during outbreaks, then seeing if vaccinated people stay healthy.
So, what do you do before anyone’s sick? Since you can’t infect hundreds of healthy people with something that might kill them, the best you can get is a candidate vaccine, which could prove to be worthless. Or rather, 40-50 iffy candidates – because you won’t know which viruses at the top of Deep Vision’s list are actually dangerous. And no way would Congress spend billions on each of them to stockpile doses.
And even if you hit on a successful vaccine, how much time have you saved? If we’d done this with COVID, it would have been about two days – because that’s how long it took Moderna to come up with their final vaccine candidate after China first published COVID’s genome. But Moderna’s vaccine wasn’t available for another 340 days, due to safety testing, efficacy testing, dealing with regulators, and building a manufacturing line. So in exchange for perhaps triggering multiple pandemics, Deep Vision might have shaved 1% off of wait time for vaccines. Not a great trade.
Another lousy argument: “If we don’t do it, China will.” Actually, if you think any nation is diabolical enough to weaponize a pandemic, Deep Vision would accelerate, not slow their efforts, by sharing lots of promising starting points. Also, pandemics aren’t strategically useful weapons, because they’re not targetable, and will eventually hit the country that launches them. Culprits can’t secretly vaccinate their populations up front, because you can’t hide vast national efforts from modern intelligence. As for ethnically targeted pandemics, even if they’re possible (which many doubt), COVID showed us how fast pandemics evolve their way around immunity pockets and other speed bumps.
Another bad argument: “The horse is already out of the barn, because deadly genomes like smallpox and 1918 flu are universally available.” This is like saying we already have two hijackers onboard, so why not invite a few more in before the plane leaves the gate? And much more importantly, there’s nothing remotely okay about either of these genomes being online.
Let’s focus on 1918 flu because it’s the easiest one to make. It killed almost 1 out of 30 humans the last time it ran free. It’s now been resurrected, with live viruses in multiple labs that we know of, and almost certainly in labs we don’t know of. It . . . probably won’t be as deadly if it gets loose again, because most of us have been infected with its cousins or descendants, which should confer some immunity. Also, many of its long-ago victims died of secondary infections, which we’re now much better able to control with antibiotics.
All of that said, we can only guess at what happens if this bug starts a nostalgia tour. One expert I surveyed gives it 10% odds of triggering another pandemic. While that beats 50/50, those odds are clearly unacceptably high, to a monstrous degree. Which begs the question of what lunatic made this genome universally available?
The answer is, America’s Centers for Disease Control. I wish I could tell you they did this before we realized it would ever be possible to make live viruses from scratch. But no – they published the genome in 2005, six years after this was first achieved. By then, a hundred-ish people were probably capable of viral rescue, according to three informed people I surveyed. And no one at the CDC would have been dumb enough to think this number wouldn’t grow by orders of magnitude.
This brings me to the most chilling argument in favor of Deep Vision, and probably the only one that mattered when it was approved. It’s that some people think there’s nothing wrong with giving universal access to genomes that could kill at the scale of a world.
I’ll discuss this, as well as my second core rule for managing biorisk in the age of AI in my next post.
Subscribe to my Substack if you’d like to know when it drops.