AI: Who by Fire?

“Something is happening here but you don’t know what it is, do you, Mr. Jones?” Bob Dylan

“The law of unintended consequences is never broken.”
I thought I would find many citations when I googled this, but since I found none, I’ll attribute it to myself.

Quite a few people I know have been expressing the same sentiment. Let’s call it soft nihilism. Mostly these are middle-aged and older, generally thoughtful and liberal individuals not given to extreme sentiments. Today I heard it this way: “Maybe we just have to burn it all down and start over.” Not long ago, I heard it this way: “Someone has to clean out the stables.” The Trumpists like to talk about “draining the swamp.”

I know what they mean. Disenchantment with institutions is profound and widespread. So is skepticism about whether conventional means of protest or proposition—the petition, vote, demonstration, persuasion—can cure the indifference, corruption, and self-regard infecting various organs of the body politic. Many of us live with a vision of the planet burning; it’s hard to imagine the smoke we see every day doesn’t augur fire. Or maybe it’s the next pandemic, the prospect that the powerful people indifferent to others’ suffering, those who make political hay by rejecting measures and medicines that can help, will go on fanning the embers of mistrust into a vengeful conflagration.

Yom Kippur starts tonight. In one sung prayer, U’Netaneh Tokef, we look toward the days of our deaths, asking the blessing I now ask for all of us: to be inscribed for a good year in the book of life.

On Rosh Hashanah will be inscribed and on Yom Kippur will be sealed how many will pass from the earth and how many will be created; who will live and who will die; who will die at his predestined time and who before his time; who by water and who by fire, who by sword, who by beast, who by famine, who by thirst, who by storm, who by plague, who by strangulation, and who by stoning. Who will rest and who will wander, who will live in harmony and who will be harried, who will enjoy tranquility and who will suffer, who will be impoverished and who will be enriched, who will be degraded and who will be exalted.

The words always have power, but this year, they feel deeper. The central contradiction of human life is before us: no matter how many times the end of the world has been predicted, our task has been to go on living. But is there something abroad in the land that isn’t so sure that will continue?

I feel it too, the desire for a reset. But after a few seconds, the feeling is displaced by a realization: as to what comes after the shit is shoveled, the fire burns down, or the swamp dries, those calling for a cleansing share no consensus.

On the left it appears that people hope human- and environment-centered institutions, democratic and responsive, will rise from the embers, that kindness and compassion will rule. In that scenario, people like me flourish. But what of the people who hate people like me? I tend to trust Isaiah Berlin’s reading of history, that those who propose a “new man” generally feel okay about killing a great many of the old ones in pursuit of their goal; that an ocean of blood generally makes a poor foundation for a good society.

On the right, the hope is to establish some fever-dream of history, with white Christian Nationalist men in charge and everyone else obediently following behind. In that scenario, I—a woman, a Jew, not reluctant to speak her mind—would certainly be one of the cleansed.

Either way, I suppose the desire to burn it all down is rooted in desperation: “I‘m done. Let’s get rid of this; surely whatever follows will be better.

But I can’t find the place in history that justifies that assurance. Neither can I see how our actually existing institutions will be transformed into sites of love and justice other than the usual long, long, long view, one step at a time, and I don’t have tremendous confidence that the journey of a thousand steps is even now unfolding. Wanting it to be true doesn’t make it so.

“Après moi, le déluge” (“After me, the flood”) is variously attributed to Louis XV and Mme Pompadour, but to me, it is the slogan of the Republican right in this country: I want what I want now, and I don’t care what follows. I fear this mindset is contagious.

Which leads me to my subject: artificial intelligence (AI for short)—in particular, our collective response to its looming presence. Here, instead of a doomsday scenario coming close after long ripening, we have an existential conundrum generated by known faces, in real time. A good case study.

I’ve been reading and listening to stories about AI for quite a while. Many of them are profiles of people who’ve invested massive amounts of time and money in the hope of developing the holy grail of AI, “artificial general intelligence,” AGI. Most recently, I read Steven Levy’s long Wired piece about Sam Altman and his project OpenAI, the outfit that created ChatGPT, the talk of the town as the first-out-of-the-box user-friendly AI. Chat GPT instantly became the bane of academia (what happens when you can’t tell if student papers were written by the students themselves or popped into existence full-grown a few minutes after a student queried ChatGPT?) and the favorite flavor of digital pundits. I usually listen to the New York Times “Hard Fork” podcast, which wanders the territory encompassed by Elon Musk, self-driving cars, and AI. And I read or listen to anything AI-adjacent that catches my eye.

I am not a digital expert. My knowledge extends no further than simple HTML, which if I were smarter I could obviously have a chatbot do for me. But I am a keen observer of the human scene, and feel authorized by the great Paul Goodman to adopt his response to those who questioned his choice to write on so many subjects: “I have only one subject—the human beings I know in their man-made [sic] environment.”

AI is usually defined as the capability of machines to achieve human-level intelligence and interaction. Chatbots like ChatGPT, which can feel indistinguishable from conversing with another person, offer an example. But achieving AGI is the big prize that tech legions are pursuing, a system that fully accomplishes any intellectual task of which humans are capable, even an autonomous system that surpasses human intelligence. (Google “the singularity” if you enjoy blowing your mind on that topic.)

In Levy’s piece, AGI remains indefinable. “OpenAI doesn’t claim to know what AGI really is. The determination would come from the board, but it’s not clear how the board would define it. When I ask Altman, who is on the board, for clarity, his response is anything but open. ‘It’s not a single Turing test, but a number of things we might use,’ he says.” (Alan Turing, an early and much-persecuted computer genius, proposed a simple test of whether an intelligence is human or machine-based; lots of people since have either admired it or said why it may not work.)

The people working to profit from AI and achieve AGI tend to express blue-sky hopes. They see the end of what the late David Graeber called “bullshit jobs”—tedious labor that serves no valid social purpose—freeing us all to develop and contribute our gifts. They see robots taking over the necessary tasks of a society. They see solutions to dirty energy, injustice—as to greed, I’m not sure, flowing from the combined knowledge of all human beings scraped from the internet and compounded by super-intelligent machines. Surely some people are attempting to imagine the practical steps it would take to realize these visions, but in the essays and articles I’ve read, the custom is more to imagine an end-point, not a path to it.

Meanwhile, all the people whose livelihoods depend on bullshit jobs see their paychecks dissolving in a fog of AI-induced automation. The future that conjures for me is straight out of sci-fi, amounting to a boldface invitation to “burn it all down:” hordes of human skeletons barely living on the scrounged detritus of the privileged. I have to assume the future it conjures for its makers is very different, putting them atop the class of beneficiaries, upheld by an invisible mountain of collateral damage.

To me, it seems obvious that the lubrication needed to make an AI-dominated future livable is something that tech bros often disparage, a robust, humane, and vital public sector. A future of decent housing, healthcare, environment well-being, conviviality, and cooperation is conceivable if society’s commonwealth is invested in those aims—not just a guaranteed basic income, but the guarantee of all those social goods that make life truly inhabitable.

I might be less wary of what AI means if I heard as much discussion of how it would improve life for those whose work is the raw material of profit for the few as I do about the magical if vague things its inventors pursue.

But then again, I might not. One point AI maker profiles tend to gloss is the extent to which their technology seems to have a mind of its own. AI technology is suddenly everywhere for two evident reasons. One is that OpenAI needed a huge cash infusion to pursue AGI, so entered into a partnership with Microsoft to exploit the applications it generates. One of the first things Microsoft did was to augment its Bing search engine, previously hardly visible enough to be called a Google competitor, with certain AI capabilities, some of which had immediate unintended consequences.

Anecdotes abound. Kevin Roose, a cohost of the Hard Fork podcast, encountered an AI-generated entity that professed its love and wanted to supplant Roose’s human spouse. In another much-cited instance, an attorney used ChatGPT to obtain case law citations, then discovered in court that all of them were made up by the chatbot, which seems to enjoy fiction as much as we humans. Most commentators say that these errors, identified by users who in effect comprise a horde of unpaid evaluators of new technology, will be quickly and decisively corrected. But what about the passage in Steven Levy’s piece in which Microsoft CEO Satya Nadella describes his astonishment at early AI’s autonomy?

By the time Microsoft began unloading Brinks trucks’ worth of cash into OpenAI ($2 billion in 2021, and the other $10 billion earlier this year), OpenAI had completed GPT-3, which, of course, was even more impressive than its predecessors. When Nadella saw what GPT-3 could do, he says, it was the first time he deeply understood that Microsoft had snared something truly transformative. “We started observing all those emergent properties.” For instance, GPT had taught itself how to program computers. “We didn’t train it on coding—it just got good at coding!” he says.

This is the seed for the particular dystopian terror for which the archetype is the Hal computer in 2001, which decides it knows better than its human operators, seizing control. We’ve seen enough movies like this to know the script by heart, especially the part where a massive machine-based intelligence decides that humanity is a kind of infection from which the universe must be cleansed.

Burn it all down. Clean out the stables. Drain the swamp. I want to live and let live, but I must admit it has a certain ring. If the law of unintended consequences is indeed as unbroken as I think, if “Apres moi, le deluge” rhymes as closely as I think it does with the thought processes of those who dismiss the two massive risks of AI—a society of Morlocks and Eloi, as in H.G. Wells’ The Time Machine; and a malign mechanical intelligence we are powerless to control—in favor of crossing their fingers and seeing what happens, the fear that rises whispers it might be better to strike a match now and skip the wait.

I admire and envy the people whose optimism endures. But I can’t see how it will be grounded unless we start talking out loud about what this emergent desire to burn it all down means—what fears we are nourishing, what dark corners of the human heart are being revealed, what unanswered questions fester behind the yearning for a cleansing flood.

And then I’d like to see us engage another question: if we are really willing to let it all go up in smoke, what have we got to lose? Shouldn’t we first be willing to do the less dire things that are deemed impossible in this political climate? As I see the inside-baseball/horse race character of most coverage of electoral politics, as I see it accepted as immutable reality that the most important things can’t be done because…polling, electability, fear of hard truth—then I think it’s past time to undress the emperor.

So there is something I would like to burn down: the illusion that the sorry pass to which so much of our social order has come is inevitable, unbreakable, and must be endured. I’m not going to offer a political platform here, just list a few of the things that can no longer be postponed, regardless of how “unrealistic” the pollsters think they are.

Pause AI experiments, as many knowledgeable people have demanded. Take immediate and radical action on climate, no compromises with a few more oil leases to appease the thirsty. Institute an immediate basic income grant, not means-tested, to replace all of the punitive welfare programs. Demilitarize the police. Legalize universal bodily autonomy and forbid restrictions on reproductive rights and gender expression. Repair the educational, healthcare, legal, and other systems that perpetuate structural racism. Ensure the right to culture. I’ll stop there for now.

Most of what really needs doing has been so blanketed with appeals to “realism,” “practicality,” and “patience,” that it is barely mentioned anymore. The current likelihood that Congress will shut down federal spending is a great example of accommodating to absurdity: these 20 far-right idiots are now known as the “wrecking ball” caucus. Speaker Kevin McCarthy says they “want to burn the whole place down,” but that hasn’t stopped him from prioritizing keeping his own job over putting out the fire.

The optimism that persists for me is the knowledge that we have the means to accomplish all I’ve mentioned and more. The part that wants to burn it all down fears that while we can do it, we won’t muster the will to surmount our internalized powerlessness and get it done. If that fear comes true, it probably won’t be a sudden surge of fire or water that sweeps us away, just the slow suffocation of heads afraid to emerge from the sand.

“Who by Fire?” by Leonard Cohen.

Order my new book: In The Camp of Angels of Freedom: What Does It Mean to Be Educated?