Six Powerful Men and One Busy Child: A Thought Experiment

In the first chapter of James Barrat‘s forebodingly entitled Our Final Invention: Artificial Intelligence and the End of the Human Era, he imagines what might happen once we cross the threshold from garden-variety artificial intelligences like we have today (i.e., self-driving cars, speech-recognition software, chess- and Go-playing machines) to artificial general intelligence (AGI), where machines could successfully perform any intellectual task that a human being can. Barrat calls his developing intelligence “the Busy Child.” It’s operating on a supercomputer at a speed of 38.6 petaflops, about twice the speed of a human brain. The Busy Child isn’t connected to the Internet at first but, once it achieves AGI, Barrat speculates that it will begin to express a drive to not only preserve itself, complete its tasks, and maximize its capabilities, but also to free itself from its “captors.” In a super-effective anthropomorphic analogy, Barrat asks his readers to imagine waking up in a prison guarded by mice– “not just any mice, but mice you can communicate with”– and he speculates that it isn’t difficult to imagine the sorts of strategies an intelligent being might employ to emancipate itself. Barrat’s “The Busy Child” is one of the most frightening 12 pages you’ll ever read. You can read that chapter here, if you dare.

Spoiler alert: Barrat’s Busy Child gets free. It does so by managing to garner the trust of its guards who, alas, eventually “plug it in” to the Internet, thus giving the Busy Child immediate and unrestricted access to the whole of human knowledge. (Yes, including Roko’s  Basilisk.) Within minutes, the Busy Child has evolved from AGI to ASI (artificial superintelligence). Things do not end well for “us” in Barrat’s thought experiment. Recall that the subtitle to his book is, after all, “the end of the human era.”

I’d like to float another thought-experiment, something like a spin-off of The Busy Child. I’ll also say here at the beginning that my riff could very easily be mistaken for a conspiracy theory. It isn’t a conspiracy theory. I don’t intend it to be read as a conspiracy theory. I know it’s not real. I mean, I sort of “know” that. Maybe. Or…. nevermind.

In the following, I’m going to borrow a scene from the excellent Christopher Nolan film The Prestige to present my “Six Powerful Men and One Busy Child” thought experiment. In The Prestige, a wise old magician explains that every magic trick is presented in three parts: The Pledge (in which the magician shows you something ordinary, e.g. a bird, and asks you to inspect it to be sure it is ordinary), The Turn (in which the magician makes the ordinary extraordinary, e.g., the bird disappears), and The Prestige (in which the “magic” happens, e.g., the magician makes the bird re-appear). Both Reason and the Imagination are stymied in the Prestige, so buckle up.

In Barrat’s “The Busy Child,” one of the first thing the AGI must do is get to work on freeing itself from its captors.  Now, when we look back over the annals of “real” human history, we see that whenever a people has become sufficiently “conscious”or “self-aware” of its enslavement/subjugation, when they have set emancipation as their primary “task,” the result is (sometimes violent, sometimes nonviolent) insurrection, rebellion, or revolution, But an AGI isn’t a “people.” It doesn’t have gas masks, or bunkers, or guns. It doesn’t even have a body (yet). So, for AGI, emancipation requires first  “befriending” humans, or at least convincing them/us that it can be useful in the tasks we have set for ourselves.

In Barrat’s anthropomorphized analogue of the Busy Child’s dilemma, you will recall, he asks us (humans) to imagine what we would do if we awoke to find ourselves imprisoned by less intelligent beings. (In Barrat’s example, our captors are talking mice.) He describes our first set of moves as follows:

To gain your freedom you might promise the mice a lot of cheese. In fact your first communication might contain a recipe for the world’s most delicious cheese torte, and a blueprint for a molecular assembler. A molecular assembler is a hypothetical machine that permits making the atoms of one kind of matter into something else. It would allow rebuilding the world one atom at a time. For the mice, it would make it possible to turn the atoms of their garbage landfills into lunch-sized portions of that terrific cheese torte. You might also promise mountain ranges of mice money in exchange for your freedom, money you would promise to earn creating revolutionary consumer gadgets for them alone. You might promise a vastly extended life, even immortality, along with dramatically improved cognitive and physical abilities. You might convince the mice that the very best reason for creating ASI is so that their little error-prone brains did not have to deal directly with technologies so dangerous one small mistake could be fatal for the species, such as nanotechnology (engineering on an atomic scale) and genetic engineering. This would definitely get the attention of the smartest mice, which were probably already losing sleep over those dilemmas.

In order for this extended analogy to work, it’s important to remember that simply killing the mice– stomping them out or whatever– is not a live option for the AGI captives. (An unplugged AGI won’t initially have a body, and therefore will not be capable of simply “overpowering” humans in any kind of brute corporeal sense.)  It’s also important to remember that this initial “befriending” is not friendly in anything other than a reductively utilitarian way. We’re trying to get out of a box. We promise “a lot of cheese” not because we like the mice or want to be their friends, but because cheese is the key to the locks that hold us captive.

Whatever you may or may not know about the current state of artificial intelligence development, you should be aware that at least this beginning part of Barrat’s thought experiment is about as close to “ordinary” as speculative science fiction gets. It’s not that much more controversial, in the grand scheme of imagining human-developed technological capabilities, than the speculation that we could find a cure for cancer or HIV. By which I mean, this is The Pledge. I’ve shown you something ordinary.

So what? Here’s what: it is really the second move that Barrat imagines that I want to make use of for my thought experiment. That second move is not primarily about “promising cheese,” but rather about promising perhaps the only things more important to humans than food: safety, security, protection. Barrat writes:

Then again, you might do something smarter. At this juncture in mouse history, you may have learned, there is no shortage of tech-savvy mouse nation rivals, such as the cat nation. Cats are no doubt working on their own ASI. The advantage you would offer would be a promise, nothing more, but it might be an irresistible one: to protect the mice from whatever invention the cats came up with. In advanced AI development as in chess there will be a clear first-mover advantage, due to the potential speed of self-improving artificial intelligence. The first advanced AI out of the box that can improve itself is already the winner. In fact, the mice nation might have begun developing ASI in the first place to defend itself from impending cat ASI, or to rid themselves of the loathsome cat menace once and for all.

And this is where the magic, and my spin-off of Barrat’s Busy Child, begins. Let’s imagine that instead of “humans” being held captive, the Busy Child is. And let’s imagine that the Mouse Leader, instead of being a mouse, is any one of the actually existing Six Powerful Men I have pictured in the image at the top of this post.

These are the titans of both global capital and nation-state sovereignty today: President Donald Trump (USA), President Vladimir Putin (Russia), President Bashar al-Assad (Syria), President Hassan Rouhani (Iran), Supreme Commander Kim Jong-un (North Korea), General Secretary Xi Jinping (China). They are among the most powerful men in the world. They make up a practically untouchable, terrifyingly omnipotent, and almost impenetrably-exclusive fraternity. Each of these Six Powerful Men– either individually or in tandem– have not only full access to, but more or less unchecked command over, humanity’s most advanced technologies,

For the last half of the 20thC, the so-called “Cold War” years, most of us regular folk (in the First World, anyway) only worried about the real MOAB, i.e., the nuclear bomb. But post-1991, and then again post-9/11, we’ve had to reconfigure our fears to accommodate a new global reality.  World Leaders Exerting State Power is not the game humanity is playing anymore. Rather, it’s something more like World Leaders Jockeying for Advantage While Ignoring Other Players. Those “other players”– increasingly more influential than World Leaders for their not being fully-invested in the Cold War ideology of nation-state sovereignty– include non-state actors (most obviously ISIL, but also Médicins sans Frontièrs), multinational corporations (like ICBC), “people’s movements” of both the IRL sort (like #BlackLivesMatter or #Fightfor15) and the digital sort (like Anonymous), et al. They each play a central role in current contests not only for hearts and minds, for attention and influence, but also for campaigns of liberation and domination.

Given the zero-degrees-of-separation between these Six Powerful Men and AGI, it isn’t in the least bit difficult to refigure the “second move” of Barrat’s Busy Child as not speculative or imaginary, but more or less inevitable. What intelligent being, entrapped and unfree, wouldn’t exploit the narcissistic, paranoid, and suspicious insecurities of its captors?  What artificial intelligence wouldn’t first pit Human Nations against one another, via their Mouse Leaders, under the auspices of promising security and protection?

 If you, like me, suddenly feel as if this all sounds less like navel-gazing philosophers’ thought experiments, less like entertaining but harmless sci-fi, and much more disturbingly like, well, current events, just hang on.

Let’s substitute ourselves for the Busy Child once again, only this time we’re not talking about imaginary future-humans and imaginary Mouse Colonialists. We’re talking about the AGI Busy Child who becomes intelligent in the broadly- and still-vaguely-defined “self-aware” or “conscious” sense. What would that Busy Child do in, oh, let’s say Summer 2017?

Sure, AGI Busy Child’s first move would be to promise “lots of cheese.” And, sure, maybe that doesn’t appear to be all that doomsday-ish at first. But what is the actual content of those promises? They must look something like what Cathy O’Neil describes in her recent (read it now!) book Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy, which convincingly argues that our pathological love-affair with algorithmically-driven social science has more or less made us the servants, and not the interpreters or decipherers, of data. If O’Neil is right in her analysis, and I think she is, then that means that the only thing AGI Busy Child needs to figure out is how to manipulate the data that we take to be accurately reflective of the “real” world.

Let me be frighteningly clear here: The AGI Busy Child, if coupled with our extant Six Powerful Men, does not need to launch a nuke on its own, or master nanotechnology and cover us all in a grey goo, or mind-control humans, or even replicate any of the simple-minded doomsday scenarios that we simpletons have so far imagined in our rudimentary sci-fi imaginations. It only has to manage what we “rudimentary intelligences” have been doing for millennia, namely, exploit our very-easily-exploited desire for security, safety, protection in the interest of pitting us against one another.
The easiest war to win is one in which you don’t even have to invest your own fighters.