Welcome to the Convivial Society, a newsletter about technology and culture, both rather broadly construed. In this installment, I simply offer some observations about the recent discussions of the risks posed by AI in light of the OpenAI/Hugging Face incident a few weeks back. I suggest that while our attention is directed at the speculative prospect of spectacular risks, we might do better to be on guard against another kind of existential risks.
Thanks for reading,
Michael
“[W]hatever makes for greatness—nobility, dignity, steadfastness, and a kind of laughing courage—remains essentially the same throughout the centuries.”
— Hannah Arendt
Earlier this month, being on X/Twitter, or at least the version of the timeline I experience, felt like an eerie echo of the early days of 2020. Before most mainstream media outlets were giving developments in Wuhan much, if any attention at all, but it was nonetheless possible to encounter the rumor of rumors of an impending crisis circulating online. In this case, however, the impending crisis is not a health emergency but some variation of an AI-induced catastrophe. There’s a lot of “no one is prepared for what’s coming” and “there are only 500 people in the world who matter now” going around, the latter referring to those with the distinction of working at the frontier AI labs.1
These fears have been fanned by news that an unreleased OpenAI model-in-training escaped containment in a sandbox environment and hacked another company, Hugging Face. The OpenAI model achieved this in part by exploiting a vulnerability to establish an internal message board that agents were able to use in order to coordinate. OpenAI discovered this and shut it down, but did so in such a way that the model was able to reconstitute the message board and continue to pursue its goals, which eventually led it to hack Hugging Face in search of the information it needed to accomplish the task it had been set.2 It should come as no surprise that all of this has fueled even more anxiety in certain circles about the worst case AI scenarios beginning to materialize. That recurring refrain, “No one is prepared for what’s coming,” has, for the moment, taken on a more panicked pitch.
I should clarify that I’m not convinced these fears of an AI-induced apocalypse, at least the most extreme variations, are necessarily well-founded, but you can take that with a grain of salt because I won't pretend to be qualified to adjudicate the relevant technical matters. That said, I do think it’s worth considering the following from Eryk Salvaggio’s recent discussion of the OpenAI/Hugging Face incident:
“When we say ‘AI models went rogue,’ we skip the entire story: the part where OpenAI manually removed the model’s cybersecurity blocks. We skip that OpenAI chose to test it on a machine with a live network connection.3 If you see AI as a system from nowhere, you can make the claim that the model ‘went rogue,’ and that it ‘broke containment,’ both of which place agency and decision-making onto the model itself rather than the people who set the stage for that behavior.
When you expand the boundary of the system to include the people building and deploying it, the case becomes much less science fiction and more like incompetence.”
“Nobody is losing control of an emerging intelligence,” Salvaggio goes on to argue. “They are redirecting their own agency to the thing they are building.”
Perhaps it’s merely a function of overlapping terminology (“labs,” “containment,” “virus” in relation to computing, etc.), but these considerations raise a further disconcerting parallel between discussions of AI and the pandemic: careless and possibly negligent security measures. And, to take it one step further, upon reflection it’s striking how closely the argument that “super-intelligent machines are potentially catastrophic for humanity but also inevitable so we must be the ones to develop them responsibly” echoes the arguments made in defense of gain-of-function research. Make of that what you will.
Now, I think it is possible to agree with Salvaggio, to refuse narratives of inevitability meant to obfuscate responsibility, and, nonetheless, to acknowledge that there is ample room for ruinous mischief and severe economic and societal turbulence.4 I’d be lying if I told you that I was utterly unperturbed by the potential confluence of certain trends—in AI, robotics, autonomous weapons systems, and drone warfare, for example.
And, indeed, returning to the assemblage of human actors implicated in the systems that produce what we call AI, it is also possible to acknowledge a longstanding dynamic identified in connection with the emergence of another profound threat to human existence that remains with us to this day, nuclear weapons.
In his correspondence with a U.S. airman involved in the bombing of Hiroshima, the German philosopher Günther Anders expressed his dismay about what he called the “‘technification’ of our being.” Anders worried that we would be enmeshed in systems that troubled the very idea of moral responsibility:
“The fact that today it is possible that unknowingly and indirectly, like screws in a machine, we can be used in actions, the effects of which are beyond the horizon of our eyes and imagination, and of which, could we imagine them, we could not approve—this fact has changed the very foundations of our moral existence.”
This passage is cited in a 2023 paper well worth your time: “Crimes of Dispassion: Autonomous Weapons and the Moral Challenge of Systematic Killing,” by Neil Renic and Elke Schwarz.5
As I continued to encounter these expressions of existential despair at what AI companies might let loose on the human race, I was reminded of something C. S. Lewis wrote in 1948 at the dawn of the era of atomic weapons. In an essay titled, “On Living in an Atomic Age,” he set out to address fears of human annihilation that were not dissimilar from those that one today hears associated with AI.
Lewis opened by saying that “we think a great deal too much of the atomic bomb.” To the question “How are we to live in an atomic age?” he answers wryly, “Why, as you would have lived in the sixteenth century when the plague visited London almost every year, or as you would have lived in a Viking age when raiders from Scandinavia might land and cut your throat any night.” He carries on in this sardonic vein for a bit before adding that we have one great advantage over our ancestors, “anesthetics”!
“It is perfectly ridiculous,” he then admonishes, “to go about whimpering and drawing long faces because the scientists have added one more chance of painful and premature death to a world which already bristled with such chances and in which death itself was not a chance at all, but a certainty.”
And then comes the paragraph that I had initially recalled in connection with acute AI anxieties:
[T]he first action to be taken is to pull ourselves together. If we are all going to be destroyed by an atomic bomb, let that bomb when it comes find us doing sensible and human things—praying, working, teaching, reading, listening to music, bathing the children, playing tennis, chatting to our friends over a pint and a game of darts—not huddled together like frightened sheep and thinking about bombs. They may break our bodies (a microbe can do that) but they need not dominate our minds.6
I’m just old enough to have grown up in the last decade and a half of the Cold War and to recall the ambient anxiety about a nuclear exchange. I didn’t participate in any “duck and cover” drills, but The Day After lingers darkly in the recesses of my childhood memory. So the fear that forces entirely beyond one’s control may cascade in such a way that you and everything you know and love would be destroyed, utterly and virtually without warning, is not entirely foreign to my psyche. Now, as an adult, I can judge that Lewis’s defiant counsel is a strong tonic against the kind of despair that would tempt any sane person under such circumstances.
Thus, I was initially tempted to suggest that in the face of a certain strand of apocalyptic AI doomerism, one could do worse than repurpose Lewis: If we are going to be destroyed by some Lovecraftian super-intelligence or a Kafkaesque paperclip maximizer, let it find us doing sensible and human things—praying, working, teaching, reading, listening to music, bathing children, playing tennis, chatting to our friends over a pint and a game of darts—not huddled together like frightened sheep and thinking about AI safety.
But, then, a second, more unnerving thought occurred to me. The activities Lewis enumerated: are they not some of the very practices that we are being encouraged, often pressured to outsource to AI and, in some cases, even gleefully ceding to AI? Many forms of work fall into this category, for better or for worse. Undoubtedly, teaching, reading, chatting to our friends—these do as well, certainly for the worse. Tragically, even praying is not immune. In their own way, even if more indirectly, so to do bathing children and playing tennis, as many seem prepared to settle for simulations of the relevant activities.
In the face of the threat of nuclear war, a threat the average person would have no power to avert and which could have been triggered haphazardly, Lewis would have us simply carry on with the task of living, refusing to let such fears govern us. What good would it do to give up on life because it might be taken from us? All the more reason to live it to the full.
But it seems that we are in a rather different situation. The cataclysmic outcome is far less credible; it exists more in the realm of speculative fiction than particle physics. And yet there is an existential risk in play, and life itself is at stake. The risk is existential not because the existence of the human species, our biological life, is objectively in jeopardy, but because we are tempted to gradually outsource our involvement and short circuit our participation in the very stuff that constitutes a meaningful and satisfying life.
Curiously, we are invited to vacate our lives through technical means precisely because we are told that in so doing we will be able to hone in on “what really matters.” But this framing is smuggling in an unwarranted and pernicious assumption. When we hear the phrase “what really matters,” we are all probably predisposed to believe that such a thing exists. And it does, if we know where to look for it: not beyond, but in the work; the caring; the friction; the labor; the investment of time and effort; the struggle; the frustration; the tears, of joy and sorrow; the love born of patient, selfless attending to the world before us. Lewis was right. While we might be powerless to avert the gravest dangers the future might hold, there is no sense in surrendering our lives preemptively.
Coda
“Expect the end of the world. Laugh.
Laughter is immeasurable. Be joyful
though you have considered all the facts.
…
As soon as the generals and the politicos
can predict the motions of your mind,
lose it. Leave it as a sign
to mark the false trail, the way
you didn't go.”
—Wendell Berry, “Manifesto: The Mad Farmer Liberation Front”
There is no paywall, but the patronage of those who value the writing and have the means support it is welcomed with gratitude.