The failure to explore and monitor the threat [of dangerous artificial intelligence] is almost society-wide. But that failure does not . . . alter the fact that we will have just one chance to establish a positive coexistence with beings whose intelligence is greater than our own.
This is not just the closing paragraph of James Barrat’s new manifesto; it is the overarching theme of this book. And when Barrat speaks of the threat of artificial intelligence, he wants you to know it is a matter of your life and death.
Barrat knows that we have heard all of this before, from Isaac Asimov and Arthur C. Clarke and James Cameron, but the foremost merit of Our Final Invention is that Barrat manages to distance himself credibly from the fantasists who have preceded him. This is not to say his argument is so persuasive that we should begin storing weapons and resign ourselves to becoming the protagonists of an H. G. Wells novel. But it does mean he raises issues that deserve to be considered more than they are.
Barrat knows how hard this argument is to take seriously, and he tries to stave off the dismissive laughter it invites. For instance, he acknowledges that “entertainment has inoculated [artificial intelligence] from serious consideration in the not-so-entertaining category of catastrophic risks. For decades, getting wiped out by artificial intelligence . . . has been a staple of popular movies, science-fiction novels, and video games.” The sort of apocalyptic scenarios Barrat fears do not sound much like a plotline from one of the Terminator films. The end of the world he describes comes with something that sounds closer to a whimper than a bang.
Barrat is more concerned with artificial intelligence than humanoid robots, and, unlike so many of the novels and movies that he believes inoculate us to the real threat, he believes that artificial intelligence is more worrisome insofar as it is unlike human intelligence:
Machines are amoral, and it is dangerous to think otherwise. Unlike our intelligence, machine-based superintelligence will not evolve in an ecosystem in which empathy is rewarded and passed on to subsequent generations.
The problem with machines is not that they are broadly malignant but that they are not specifically benign. Barrat has a few examples of how extremely intelligent, amoral machines might pose a threat. What happens if, for instance, an artificially intelligent machine with the ability to self-replicate does not know when to stop itself and consumes our natural resources in its drive to reproduce? Questions like this should concern us; but there is a problem with Barrat’s argument, stemming from the tricky business of defining intelligence itself.
At one point, Barrat picks up the New Oxford American Dictionary to help him define “artificial intelligence,” but even when he talks about some of the symptoms of intelligence—such as the ability to self-program, or to learn from one’s mistakes—the systems that Barrat fears always seem to have the superficial appearance of intelligence without actually being intelligent. Artificial intelligence might be able to calculate pi to more decimal points than humans are ever capable of doing; but this is intelligence that is still mediated through human experience and human programming, for human purposes.
Barrat comes close to grasping this point a number of times, without ever putting his finger on it. He asks Richard Granger, an expert in “computational neuroscience,” whether a computer would ever be able to develop intelligence on the same level as a human, considering that a computer has no senses. It is a good question, but one for which Richard Granger does not have much of an answer. His response is that Helen Keller managed to develop a human level of intelligence—an answer that does not take into account the fact that those lacking one or more of the senses can compensate through the senses they do have.
And whether our senses are limited or not, they still serve as the medium through which we experience, evaluate, and learn to make rational decisions about the world. A machine could conceivably have sensors for picking up objects or avoiding perilous situations, but it will only know to avoid contact with a hot oven to the extent that a human mind has dictated that it will do so.