The Persistent Peril of the Artificial Slave

The Persistent Peril of the Artificial Slave Author(s): Kevin LaGrandeur Source: Science Fiction Studies , Vol. 38, No. 2 (July 2011), pp. 232-252 Published by: SF-TH Inc Stable URL: http://www.jstor.org/stable/10.5621/sciefictstud.38.2.0232 Accessed: 01-12-2016 06:11 UTC

Linked references are available on JSTOR for this article: http://www.jstor.org/stable/10.5621/sciefictstud.38.2.0232?seq=1&cid=pdfreference#references_tab_contents You may need to log in to JSTOR to access the linked references. JSTOR is a not-for-profit service that helps scholars, researchers, and students discover, use, and build upon a wide range of content in a trusted digital archive. We use information technology and tools to increase productivity and facilitate new forms of scholarship. For more information about JSTOR, please contact [email protected].

Your use of the JSTOR archive indicates your acceptance of the Terms & Conditions of Use, available at http://about.jstor.org/terms

is collaborating with JSTOR to digitize, preserve and extend access to Science Fiction Studies is SF-TH Inc

This content downloaded from 128.206.9.138 on Thu, 01 Dec 2016 06:11:51 UTC

232

SCIENCE FICTION STUDIES, VOLUME 38 (2011)

Kevin LaGrandeur The Persistent Peril of the Artificial Slave

Robots were created to perform the same jobs as slaves—jobs that are dirty, dangerous, or monotonous—thereby freeing their owners to pursue more lofty and comfortable pursuits. In fact, as Timothy Lenoir notes, the whole field of cybernetics, which includes not just robots but also computer-based Artificial Intelligence (AI) systems, cyborgs, and androids, “was envisioned by scientists and engineers such as Norbert Wiener, Warren McCulloch, and their colleagues at the Macy Conferences [in the 1950s] as a way to maximize human potential in a chaotic and unpredictable postwar world. They wanted to ensure a position of mastery and control removed removed from the noise and chaos” (210; emphases added). Yet mastery and control are tenuous things. A New York Times Times article dated 23 May 2009 and entitled “The Coming Superbrain” discusses the dream, or nightmare, of true Artificial Intelligence (Markoff). No longer the realm of science fiction, the notion that the servant-systems that we have devised, the increasingly interconnected computer and communications networks, might spontaneously emerge as self-organizing, self-replicating, and perhaps selfaware appears to be giving Silicon Valley scientists and technology experts conflicting fits of paranoia and joy—depending on their optimism about the controllability of such servant networks. These theorists focus primarily on “strong AI” systems—systems designed to evolve and learn on their own—and believe that they will, once sufficiently developed, evolve at such an exponential rate they will eventually learn to self-replicate and to surpass humans in intelligence and capability. The pessimists who worry about the controllability of such systems are numerous, perhaps because because so much of the cutting-edge work in AI is funded by the military; and in fact, any truly intelligent artificial servant is most likely to arise from the search for automated weaponry. P.W. Singer points out that not only is the military the source of most of the money for AI research, but it has the strong investment motive of the recent wars in the Middle East as well as the most extensive integrative capacity for such research, having already established a network of defense research contractors (107). Armin Krishnan, in his book about mili military tary robotics, concurs with Singer, explaining the dilemma behind this situation: There is a serious danger that military organizations would be interested in creating intelligent machines without a conscience or any moral constraints simply because they could be more effective on the battlefield and because they would never question orders. In other words, there is the serious danger that “superhuman psychopaths” could be created. (164)

The problem is that making AI with built-in limits on t he humaneness of their actions runs counter to warfare. But even without considering the mostly violent

This content downloaded from 128.206.9.138 on Thu, 01 Dec 2016 06:11:51 UTC All use subject to http://about.jstor.org/terms http://about.jstor.org/terms

THE PERSISTENT PERIL OF THE ARTIFICIAL SLAVE

233

purposes and military sources of much current AI research, pessimists abound. Most of these worry about the principles of Moore’s Law, which says that t he power and speed of microchips increases exponentially every 18 months, and because of the geometric acceleration of development that is projected once AI reaches a certain threshold, AI will eventually attain capabilities that are superior to humans. Singer mentions a number of scientists in this gloomy camp, including luminaries such as robotics expert Robert Finkelstein, physicist Freeman Dyson, and his son, the science historian George Dyson (414-15). Moreover, there are pessimists who merely point out that our current “smart” systems are balky and that programmers make plenty of errors: “Marvin Minsky,” notes Singer, “who cofounded MIT’s artificial intelligence lab, believes that we humans are so bad at writing computer software that it is all but inevitable that the first true AI we create will be ‘leapingly, screamingly insane’” (415). The fact that many knowledgeable people are nervous and pessimistic about the prospect of cybernetic slaves getting out of hand is evident in the convening of a private conference of AI specialists, by the Association for the Advancement of Artificial Intelligence, to discuss the dangers. According to an article by John Markoff in the New York Times , the scientists “agreed that robots that can kill autonomously are either already here or will be soon,” and they were so “impressed and alarmed by advances in artificial intelligence” that they discussed “whether there should be limits on research that might lead to loss of human control over computer-based systems that carry a growing share of society’s workload, from waging war to chatting with customers on t he phone” (“Scientists Worry”). Even most optimists among the experts do not believe that mastery and control of AI will be likely in the long run. Rather, they think that humans—at least in our present form—will be extinct by the time AI is fully sentient, but that we will be happily so, having transformed into a kind of superior hybrid of flesh and machine. Some, such as Ray Kurzweil and Hans Moravec, believe that even though we humans will be eclipsed by our own creations, we will be able to “upload” our consciousnesses into robotic bodies. Others, such as Rodney Brooks and Kevin Warwick, believe that we will have already incorporated so many small digital gadgets into our bodies (such as wearable computers and neural microchips) that we will effectively be cyborgs and fairly indistinguishable from advanced AI systems. The main point of this essay is that worries about intelligent artificial servants are much older than the Digital Age, or even the Industrial Age. Yet even though there has been much discussion recently about artificially-intelligent artifacts with humanoid—or superhuman— capabilities and, in some cases, a human likeness, not many of these discussions have focused on the pre-industrial history of artificial humanoids and humanoid systems. 1 And none, to my knowledge, have focused on the pre-industrial history of artificial humanoid servants.2 This article surveys these kinds of artifacts and their historical persistence. The idea of artificial slaves—and questions about their tractability— occurs not only in the literature of modern times but also in ancient Greek

This content downloaded from 128.206.9.138 on Thu, 01 Dec 2016 06:11:51 UTC All use subject to http://about.jstor.org/terms

234


sources; and it is present in the l iterature and oral history of the Middle Ages and Renaissance as well. Furthermore, at each of these intervals, this idea is connected with an emotional paradox: th e joy of self-enhancement is counter poised by the anxiety of self-displacement that comes with distribution of agency. I see the problem with artificial slaves as not merely one of the violent rebellion that is the subject of so much fiction about advanced, self-conscious AI; such rebellion is just a subset of the bigger problem, which is the impulse (or willingness) not just to create servants that are more powerful than oneself, but also to give them too much power, to allow them to be inappropriate proxies for oneself. My argument in this essay is not about the hubris of scientists who create artificial prosthetics to enhance our natural powers; it is about their blindness to the actual nature of their creations as a sort of networked prosthesis. It is about the anxieties that relate to the redistribution of power that results from this status. The true danger of the artificial systems we make to serve us is really the danger of the individual’s absorption by or disappearance into those systems. Violence, and perhaps the extinction of our species, which is the danger most AI doomsayers and apologists alike focus on, is just one way of being subsumed by the powerful artificial slaves we create. P.W. Singer worries primarily about such violence in Wired for War (2009): “A machine takeover,” he says, “is generally imagined as following a path of evolution to revolution”; and, as he further contends, even such techno-optimists as Ray Kurzweil seem to see it that way, at least in the sense that strong AI would rapidly evolve beyond control of any human master (416-17). There are other types of absorption that are more subtle than the violent subjugation of the human species by its erstwhile artificial servants, however. For instance, in the history of human servitude, whole populations of masters have been gradually assimilated by a race that was intended as servants. A good example of this is the first wave of English overlords who invaded Ireland. In the late twelfth century, the Anglo-Normans, led by Henry II, invaded and conquered much of Ireland. But by the end of the fourteenth century, most of these Norman lords had become Gaelicized to the extent that they actually resisted the sixteenth-century reconquest of Ireland by the Tudors. What is at issue here is not so much a violent rebellion of AI servants, but the unpredictability of “strong AI” that can learn and evolve, and our lack of preparedness for this eventuality—chiefly because we do not perceive the slippery nature of the dialectical relationship between them and us, between maker and made, master and servant; we do not keep in mind that we are, in terms of systems theory, always and already enmeshed in a networked relationship with our prosthetic inventions. It may be argued, of course, that our current hysteria over our society’s digital saturation is just that—hysteria. Our children’s increasing attachment to digital technology may signal a change, not a disaster. It may simply be the initial step of what Rodney Brooks predicts will be the cyborgization of our species (assuming we can maintain a way to provide fuel for our increasing dependence on technology). Singer, like Brooks, puts it this way:



235

For all the fears of a world where machines rule with an iron fist, we already live in a world where machines rule humanity in another way. That is, the Matrix that surrounds us is not some future realm where evil robots look at humans as a “virus” or “cattle.” Rather, we’re embedded in a matrix of technology that increasingly shapes how we live, work, communicate, and now fight. We are dependent on technology that most of us don’t even understand. Why would machines ever need to plot a takeover when we already can’t do anything without them? (418)

The penultimate sentence is exactly my point. We are dependent on that of which we have an unclear understanding. Exactly how do our attempts—or even dreams—of creating artificial slaves affect the dynamic between servant and master, and how are they evident in pre-empirical times? The promise and peril of artificial, intelligent servants was, amazingly, first implicitly laid out over two thousand years ago by Aristotle. Artificial slaves are depicted as far back as Homer’s Iliad where, near the end of Book XVIII, we see that the god Hephaistos has fashioned a number of intelligent serving tripods that can roll themselves in and out of the gods’ banquet room, as well as metallic serving maidens that “worked for him, and were like real young women, with sense and reason, voice also and strength, and all the learning of the immortals.” Aristotle is the first, however, to discuss their uses and advantages. In Book 4 of his Politics, he refers to Hephaistos’s intelligent artifacts and argues that “if, in like manner, the shuttle would weave and the plectrum touch the lyre, chief workmen would not want servants, nor masters slaves” (1253b38-1254a1). The advantages of intelligent, artificial servants are clear in this statement. They allow for work to be done for their owner with no intermediation. Also, they allow a person to forgo the ethical problems of owning human slaves (which he has noted is not approved by all Athenians), and the possible dangers and aggravations as well, which he discusses later and to which we will return in t his essay. The boundaries between tools and slaves are implicitly blurred here by the very idea of intelligent tools, and these boundaries, as well as those among tools, slaves, and the master, become even hazier when one considers the rest of Aristotle’s discussion of slaves and tools. For he considers tools and slaves to be merely different types of instruments: “some are living, others lifeless; in the rudder, the pilot of a ship has a lifeless, in the lookout man, a living instrument; for in the arts the servant is a kind of instrument.... [T]he servant is himself an instrument for instruments” (1253b29-34). Moreover, the master is implicated in this system of instruments as well, since all instruments, animate or inanimate, human or not, are actually prosthetic extensions of the master, and as such part of a master-centered network, “a living but separated part of his bodily frame” (1255b11-12). Aristotle’s use of a ship’s rudder and human lookout as examples of the kinds of tools he is speaking of is revealing, because the pilot uses the rudder and the lookout as different types of sensory and manipulative tools, and as they are also classified as part of the master’s body, they are in fact prostheses. The rudder acts as a modified extension of the pilot’s arms, and the lookout acts as an enhancement of his eyes—these instruments


236


are, in effect, prosthetic augmentations of the master’s body and senses, and the distinction between their animate and inanimate form is secondary. Collectively, the pilot’s inanimate and animate tools provide a sort of extended somatic network that allows him more effectively to steer his ship—a network which, with its parts, is an extension of the master himself. By implication, then, the pilot’s apparatus of human slaves and other “inanimate tools” is essentially one large tool—a system driven by a collection of intelligences but centered on the master as its major node, an arrangement that enhances the master’s power to act upon his environment. Because he sees human slaves as intelligent tools (as instruments for other instruments), they are, to Aristotle, just a form of organic machine. And because they are, by definition, intelligent machines, Aristotle’s ship-steering system is a direct analogue and precursor to the modern cyborg. The modern cyborg is a combination of intelligent computerized (cybernetic) tools and the human organism (thus the name, cyb-org), in which the human organism is prosthetically enhanced by intelligent machinery. These enhancements may be directly linked to the body, as in the case of an artificial limb, but they may also be indirectly linked, as in the case of the visual sensors embedded in an extraterrestrial exploration vehicle such as the Mars Rover—sensors that are virtually attached to their human operators and allow them to see beyond their normal capacities. These two examples also show how a prosthesis can be not only compensatory, as in the artificial limb, but also an enhancement, as in the visual sensors. To go back to Aristotle’s example, human pilots are analogous to cyborgs because they are defined as pilots not by their human somatic boundaries, but by the delimited boundaries that comprise the network of animate and inanimate tools that extend their capacities. The pilot is thus a combination of cybernetic devices (in the original sense of that term: “self-steering”), 3 in the form of intelligent tools, and organic master (the slaves are organic, too, of course, but still considered merely self-regulating tools). In terms of slavery, particularly in terms of the relationship between the master and the slave, the foregoing, including the example of the pilot, reinforces Aristotle’s view of the human slave as a “possession” of the master, as a “tool for living,” and it suggests that servants’ physical forms are unimportant to Aristotle except for the functions they may provide the master’s body, functions that this philosopher would gladly see transferred to non-human, artificial forms, if possible (1254a-1254b). Thus, it is only logical when he also points out that, because human slaves are merely “tools for living” and their physical form is irrelevant, they could conceivably be replaced by smart machines—by looms that could weave on their own or lyres that could play automatically. This would obviate the need for slaves (1253b38-1254a1). His statement that human slaves could be rendered unnecessary by artificial ones is an important point. Aristotle’s fantasy of automatic slaves is at least partially the result of his dislike of relying on human ones, for several reasons. The less important reason, for him, is that some of his contemporaries have ethical qualms about slavery; but the more serious problem Aristotle has with



237

human slaves is their potential for rebellion and unpredictability. At the root of this problem is the paradox that the more powerful their nature, the more troublesome slaves may prove. He specifically refers to those human slaves who are not “natural slaves” but aristocrats captured in battle or criminals condemned to slavery as punishment. Captured aristocrats and criminals from that class, he perceptively notes, will never see themselves as anything but nobility, for “People regard themselves as noble everywhere, and not only in their own country, but they deem foreigners noble only when at home” (1.1255a, 31-33). This clash of consciousness between masters and their (foreign and/or noble) slaves will, he concludes, cause constant friction. He uses the following quotation to illustrate what he means: “Helen of Theodectes says: ‘who would presume to call me servant who am on both sides sprung from the stem of the Gods?’” (1.1255a, 35-36). As this statement implies, such individuals are accustomed to wielding power rather than obeying it, and wil l not easily adapt to being extensions of the master’s will; instead, the master will always be in a dubious position because “the relation of master and slave between them is [not] natural” (1255b, 12-15). Because Aristotle elides the boundaries separating tools, creatures, and humans, and because he sees all of these “animate” and “inanimate tools” as part of a larger, master-centered slave network, his warning about powerful slaves applies implicitly to artificial servants. An intelligent, artificial slave of greater power than its master and capable of independent action would, like the aristocrat-turned-slave, be difficult to control because the master-slave relationship would be unnatural. And therefore—because, in Aristotle’s formulation, the master exists in a virtual network comprised of all of his tools, animate and inanimate, upon which he depends to control or, indeed, to supersede natural processes—the owner of an artificial slave of great power or of an equally powerful servant-system—would be in the same delicate position as one who owns an aristocratic slave. His power would be predicated on a dangerous balancing act because it would be the result of his connection to a potentially rebellious and potent prosthesis. Though he may be initially at the center of this network of prostheses (or “tools”), the attachment to an overly potent tool could conceivably cause the prosthesis/master status to reverse. If an intelligent tool—be it animate or inanimate, humanoid or artificial—is smart enough to act as a proxy for the master, then might not the master be replaced? This implicit vulnerability becomes explicit in later accounts of artificial humanoid servants used as prosthetic enhancements of their masters. In the Middle Ages, there were tales of scientists who created such servants, all to disastrous results. One of the earliest is the story of Gerbert of Aurillac’s oracular metal head. This story, which first appears in William of Malmesbury’s Chronicles of the Kings of England, in 1123, recounts how this brilliant natural philosopher and engineer (who built a church organ driven by hydraulics and also became Pope Sylvester II) created a talking brass head. As William relates: He [Gerbert] cast, for his own purposes, the head of a statue, by a certain inspection of the stars when all the planets were about to begin their courses, which spake not unless spoken to, but then pronounced the truth, either in the


238

SCIENCE FICTION STUDIES, VOLUME 38 (2011) affirmative or the negative. For instance, when Gerbert would say, “Shall I be Pope?” the statue would reply, “Yes.” “Am I to die, ere I sing mass at Jerusalem?” “No.” (181)

Unfortunately, the prediction of Gerbert’s death by his metal android is done in a typically hazy oracular fashion that makes him unable to avoid his fate. As William goes on to relate, Gerbert’s efforts to avoid travel to Jerusalem are futile, because there is a church in Rome called “Jerusalem” at which the Pope says mass, and this also fits the prediction that the head has made. William gives us the dénouement, where Gerbert realizes he has been misled: The Pope sings mass there on three Sundays, which are called “the station at Jerusalem.” Wherefore upon one of those days Gerbert, preparing himself for mass, was suddenly struck with sickness; which increased so that he took to his bed: and consulting his statue, he became convinced of his delusion and of his approaching death. (181)

Although the brass head in this early version of the t ale only gives predictions, in later versions it is depicted as having other talents. By 1216, oral versions of the tale depict the android as being able to resolve difficult mathematical problems for Gerbert (Dickson 207). The powers of this later version of the android are important because they can indicate both the maker’s wondrous powers and his foolishness. He has created an artifact that could render him redundant, just as computer applications for travel are rendering travel agents redundant now. Stories of other great thinkers of the era, including Robert Grosseteste (ca. 1168-1253), Albertus Magnus (d. 1280), and Roger Bacon (ca. 1214-1284), depict them creating similar metallic android servants with veritably superhuman powers. In addition to the power of speech, most of these androids could also prognosticate and solve previously unsolvable mathematical, intellectual, or even political problems. The oldest of these three stories is that concerning Albertus Magnus. Tales that he created a life-sized android started circulating sometime soon after his death in 1280, and the first written record of this story appears in 1373 in Matteo Corsini’s Rosaio della Vita . In chapter 2 of that work, Corsini relates how Albertus made a metal figure so intelligent that it could talk—not by means of black magic, he emphasizes, but by careful study and use of planetary influences. While Albertus is absent, a colleague (who in later versions of the story is his pupil, Thomas Aquinas) comes to visit Albertus and calls to him from outside his room. Albertus being absent, the android responds instead. Believing that the speaking statue must be the result of some evil intelligence, the colleague smashes it. When Albertus returns and finds out what has happened, he explains to his now remorseful friend that it took him thirty years to build and that the planetary alignments will not allow another automaton like his to be produced for another thirty thousand years. Similar tales arose about Albertus’s contemporary, Robert Grosseteste, who died in 1253. Like the tales about Albertus’s robotic humanoid, the tales about Grosseteste’s device first appear in print about a hundred years after his death. In 1390, John Gower, in book 4 of his Confessio Amantis, tells of a talking



239

brass head made by this medieval cleric and philosopher (I have modernized the spelling and wording of the following passage for the reader’s convenience): For the great Scholar Grosseteste I read how busy he was … a head of brass To forge, and make it … tell Of such things as befell. And seven years business He spent, but for the carelessness Of half a minute of an hour, From first that he began labor He lost all that he had done. (vol.2, 307-08, lines 234-43)

Roger Bacon (1220-1292), who was Grossteste’s pupil, was also associated with tales about his supposed creation of a talking brass head. Much like his teacher’s, it took seven years for Bacon to build but it had slightly different powers. Instead of prognosticating, as Gerbert’s did, or telling what was happening in distant places, as Grosstesteste’s did, Bacon’s android was designed to protect England. In the oldest written version we have of the story, an anonymous prose romance written sometime in the late Middle Ages or the early Renaissance titled The Famous Historie of Fryar Bacon , the philosopher builds his talking brass head with the specific aim that it will build a brass wall around all of England to protect it from invasion (25). There are interesting similarities among all of these stories. The obvious ones are the facts that the androids are all made of metal, all of them talk, and all are destroyed or fail because of some miscalculation by their makers. But more importantly, they have in common the threats they pose to the makers themselves and, implicitly, to society. Bacon’s and Grossteste’s creations are balky and so ultimately fail to activate, and in Bacon’s case, the failure is potentially catastrophic. His invention is balky because commands for activation and control must be said in exactly the proper way and at the proper ti me (much like today’s computer commands). In particular, for activating these devices, the scientist must take action at an exact moment that allows him to harness particular astrological influences. One can see these astrological references clearly in the passages I have given from Gerbert’s and Albertus’s stories. Bacon is also warned that he must watch his creation carefully, because its moment of activation will be very precisely timed within the month, and if he misses it, his chance to activate his metal head will be lost (26). Bacon and his colleague Bungay grow fatigued after waiting nearly a month for their device to utter some words that will signal that they need to take action, and so they task Bacon’s assistant, Miles, with the job so that they can get a little sleep. But Miles is a bumpkin and clearly unfit for this responsibility: he clownishly brings a fife and a drum with him to keep himself awake and spends his time singing bawdy songs. Although he is told to alert the two scientists when the head speaks, he is not told exactly what words to listen for, and so when the talking head does activate and says only two cryptic words—“time i s”—Miles assumes that they are too trivial to warrant waking his sleep-deprived master. So the


240


moment for harnessing the head passes and it self-destructs in a sort of implosion, complete with loud rumbling and flashing fire. Interestingly, Bacon’s and Grosseteste’s failures are similar: Grosseteste procrastinates or is inattentive about the activation of his robot: thirty seconds, as Gower notes, makes the difference between success and his eventual failure—and, as this tale is part of a chapter on the evils of sloth, it is clear what its main lesson is. Bacon and his colleague Bungay also are guilty of inattentiveness and careless behavior, and although sleepiness may be a proximal cause of these things, recklessness is the ultimate cause. Their ambition to build a brass wall around England is itself inappropriate—they are academics, not politicians, and overstepping the boundaries of one’s social station was a major cultural violation in the Middle Ages and Renaissance. It was in fact seen as such a threat that, by the early English Renaissance, Henry VIII had enacted the “Sumptuary Laws,” which went so far as to prescribe just what a person of a particular profession or rank could wear, what he or she could eat, and even how many courses could be served as part of a meal in his or her house (Youings 110-16). Perhaps more importantly, Bacon and Bungay are also guilty of failing to anticipate the danger of their powerful device. Its explosion, which could have killed them all or, at minimum, destroyed their laboratory, is an indication of the harmful potential of their invention. The other tales discussed above are similar in this respect: the stories of Gerbert’s and Albertus’s androids are also violent, filled with visions of smashing and death, which indicate the disastrous potential of their androids. The most crucial element in these stories is that the powerful servants in all of them are intended to serve not just as enhancements for their makers, but as virtual proxies for them, doing work that they should more rightly do. If the systems that they create—in t hese instances, metallic androids—are able to solve more difficult philosophical and mathematical questions than their creators can, why would the creators be needed any longer? This potential obsolescence of the creator at the hands of his creation shows it s danger, but even more, it shows the limits of the master/scientist’s vision. Some examples from the Renaissance help clarify this: in that era, we have a more detailed account of Bacon’s creation, as well as a tale involving an oracular metal head used by a Middle Eastern kingdom to determine whether to go to war with other kingdoms. Both examples occur in plays by Robert Greene (ca. 1560-92). The play about Bacon, The Honorable History of Friar Bacon and Friar Bungay (1630), shows him planning not only to employ his android to give him political agency, but also to use it to enhance his academic powers as a scholar and natural philosopher. As in the earlier version of the tale, he wants to have it build a wall around all of England to defend it from its enemies. But he also tells some colleagues that the android will be able to “lecture in philosophy” and to untangle thorny academic problems (ii.26-30). This brings up the same issues of self-displacement discussed above regarding medieval androids. In the case of the talking head in Greene’s other play, Alphonsus, King of Aragon (1599), the metal head is both used and worshipped by its owners; it is



241

god and servant, complicating the whole issue of which party is master. The talking brass head in this play, Greene’s first, written about 1587, is the property of Amurack, King of Turkey. The King and his subjects not only use this android, but they also worship it as a god because of its powers. Amurack uses it as Gerbert uses his—to forecast the future. The crux of the plot concerning the brass head is that Amurack needs to decide whether to ally himself to a force that is about to go to war with Alphonsus, and he sends emissaries to the android to ask whether that war will be successful. But Amurack has already had a dream that has predicted the war’s failure, and in this dream he rails against the brass head for being inaccurate about this and causing disaster. When he awakens, the King is distracted by something his wife says and forgets his dream, but the brass head, which has heard him rail against it in his dream, does not forget and seeks revenge. So, ironically, Amurack and his people are punished for something he does not even remember saying. Even more importantly, the dream also calls into question the very necessity of using the talking brass head and points to its dubious value as a servant: the fact that Amurack has the capacity to be put into a prophetic trance makes one curious as to why he needs to trust his destiny to a capricious and powerful proxy— indeed, it brings into question whether t he metal head could properly be called servile at all. Such powerful agents as Amurack’s prophetic brass head are, like the noblemen that Aristotle mentions in his discussion of slavery, not “natural” slaves because their status and power—and their own awareness of that status and power—make them naturally rebellious. As we have already seen in our discussion of the medieval scientists’ brass androids, this impulse to use improper agents as servants and to assign them duties that are more properly the master’s are recurring themes in these stories. Typically, the masters do not see that they have overestimated their own ability to control their creations; they are lulled into complacency because the servants are, in one way or another, the work of human hands. In Alphonsus we see the consequences of this improper devolution of responsibility almost immediately. The three emissaries of Amurack—Belinus of Naples, Claramount of Barbary, and Arcastus, King of the Moors—arrive to consult with the oracular brazen head. But because it has been insulted by what Amurack has said in his dream, it refuses to prophesy at first. After its priests beg it to do so to forestall Amurack’s anger, it agrees to give a prediction that it says will protect them and yet not help Amurack. This turns out to be a blatant lie: the head’s prophecy will lead to the death of Amurack’s emissaries and to his defeat. As the brazen android tells them: You stand still loitering in the Turkish soil. What, know you not, how that it is decreed By all the gods, and chiefly by myself, That you with triumph should all Crowned be?.... Send Fabius back to Amurack again To haste him forwards in his enterprise. (IV.i.1313-16 and 1321-22)


242


This prediction leads the emissaries to think that the Turkish forces will have good fortune in their war. But this is false encouragement that will lead to their demise. Among other things, the brass head has urged them to spring into battle immediately, and they do so without waiting for the complete armies of their allies to assemble, which puts them at a severe numerical disadvantage to Alphonsus’s forces. Ultimately, Alphonsus prevails and everything plays out according to Amurack’s unfortunate prophetic dream. Ultimately, the talking brass heads in both of Greene’s plays prove to be too aware of their own power to be good servants. Instead, focused on internally generated, self-serving agendas, they rebel against their makers’ programs. The danger of Bacon’s android is more pointed in the play than in the earlier prose romance about Bacon: Miles comes into its presence only when he is armed to the teeth. It is continually balky, wanting only to be free of its servitude. Exactly as in the prose romance, it will only work and communicate according to a very tricky command language (one has to understand the significance of its utterances, which Miles does not, and one also has to be able to issue a sort of coded command, which Miles cannot do. Moreover, upon the assistant’s mistaken interpretation of its strange communications, the head implodes. In Alphonsus, the oracular head takes offense at it s master’s prescient dream and at being forced to work, so it lies to its keepers to trick them into destroying themselves by fighting a battle against hopelessly superior forces. As with the medieval examples already mentioned, the artificial servants’ masters have created and harnessed factotums whose functions and powers not only imitate their own too closely, but actually supersede them. Like Aristotle’s example of noblemen captured in battle, the powers of these servants and their awareness of those powers make them unnatural slaves, and this makes it perilous to entrust them with the very responsibilities for which they were designed. The stories we have examined give clear reasons for the dangers of these devices, and scientific hubris is only one of them. The other reasons have more to do with the devices themselves. They have four major flaws: their power, their complex coding, their inappropriateness as proxies, and the creators’ lack of awareness of how they are existentially entangled with and overly dependent on their intelligent creations. These flaws bring us to the two really remarkable observations about the thematic dangers of artificial servants that caused me to write this essay. The first, as the foregoing discussion has shown, is that these themes about the dangers of artificial slaves, and indeed about the very existence of those characters themselves, are present in literature and folktales preceding—often by a long while—the ages of industry and empiricism, let alone the digital age. The second remarkable element of these themes is that they have been, in these essential forms, persistent in Western literature over such a long period, and that they persist in pretty much the same form in the modern era. They are evident in Asimov’s I, Robot (1950), Čapek’s R.U.R (1921), Heinlein’s The Moon is a Harsh Mistress (1966), and Clarke’s 2001: A Space Odyssey (1968). In Asimov’s story, or more specifically the series of short stories that comprise his book, robots disobey their masters in various and often



243

subtle ways, due to mistakes made by their makers regarding the four flaws I list above—and these rebellions occur despite the famous “Three Laws of Robotics” that Asimov invents to prevent misbehavior by artificial humanoids .4 For example, in the story “Reason,” a robot in charge of a space station that beams power to planets inhabited by humans decides that anything outside of the space station is not real. It also decides according to the same logic that humans are inferior decision-makers to itself, and it therefore refuses to obey them (violating the Second Law of Robotics) and denies them access to the controls of the space station. These unintended consequences of the power and complex programming built into the robot by its human makers almost result in a catastrophe similar to that in the older narratives about Gerbert’s and Amurack’s talking brass heads. A solar storm is looming that might alter the energy beam’s course and destroy part of a planet, but the humans cannot get the robotic controller of the station to comply with their orders to avert the disaster. Luckily, in Asimov’s optimistic world, this turns out not to matter, because the robot chooses to maintain the course of the beam for its own reasons: it worships the energy source of the station as a god and decides that it must therefore maintain the stability of all of its systems, including the beam. Athough robotic malfunctions never cause serious harm in Asimov’s world (and indeed, in later novels, are only apparent malfunctions that actually benefit humankind), the same cannot be said of Čapek’s play. In that story, a scenario occurs that, as we have seen above, stretches back to the Middle Ages: a human corporation, Rossum’s Universal Robots, creates artificial androids as slaves, but cedes too much power and responsibility to these slaves. Ultimately, humans use them so pervasively as proxies that the world’s economy comes to depend upon them. These now numerous and potent robots rebel, overpower, and effectively exterminate the human race. Similarly, the computer HAL in 2001 tries to destroy his human masters aboard the spaceship that they all inhabit. Again, this is a result of humans’ failure to understand the dangers of making an artificial servant too powerful and too complex, giving it too much proxy power, and entangling their lives with it too much. Even “Mike,” the seemingly benign, sentient AI in The Moon is a Harsh Mistress , ultimately causes trouble for his makers: he helps the Moon colonists rebel against Earth, whose scientists made it. All of the foregoing is evidence that the idea of rebellious and dangerous artificial slaves is an archetypal one that spans Western history, and this ancient evidence is supported by the fact that the archetype persists not only in the modern imagination, via sf stories about rebellious AI servants, but also in modern systems theory, which is the basis for real AI. 5 For instance, the notions of allopoiesis and autopoiesis articulated by the Second Order systems theorists Humberto Maturana and Francisco Varela bear a connection to Aristotle’s concepts, as does the Second Order idea of the observer and the observed systems (such as the maker of an artificial system and the system itself) being interactive, or even interdependent—sometimes without the understanding of the observer.


244


An allopoietic system, which is dedicated to producing things keyed to externally-generated designs, is essentially a modern conceptual extension of Aristotle’s obedient and tractable natural slave; and an autopoietic system, which is self -perpetuating, and with priorities keyed to that internally-generated goal of self-perpetuation, is a conceptual extension of Aristotle’s non-natural slave, in the sense that this type of slave does not see the master’s goals as its own but tends instead toward self-direction. Also, as discussed earlier, Aristotle commingles the ideas of tools, slaves, and the master’s body. In fact, he uses the word organon throughout his discussion of slavery to refer to the body’s parts as well as to a craftsman’s tools or instruments, thereby mixing the notions of humans and instruments—a harbinger of the integration of the same notions in modern ideas of robotics (1254a17). 6 Second Order systems theory also maintains that any system being observed by a human is not isolated; the observer must also be seen as a system whose every interaction or even observation of the first system i s a (sometimes unacknowledged) entanglement with the other system. This means that a human who uses, interacts with, or even observes an artificial humanoid servant is enmeshed with it, much as Aristotle defines the slave as an extension of the master. Aristotle’s notion of the incorporation of the servant by the master not only exhibits evolutionary traces in the Second Order theories about the interpenetration of systems, but also in the Third Order systems theory concept of distributed cognition, which holds that the cognitive process is not only the product of an individual brain but that it also depends on elements in the external environment. This idea has now permeated fields beyond cybernetics and AI. It has been adopted, as Fred Adams and Kenneth Aizawa note in a recent article, by a number of researchers from a wide variety of fields. Adams and Aizawa mark the beginning of this trend with the anthropologist Gregory Bateson, who used it to describe the interrelationships among individuals, society, and ecosystems in his Steps to an Ecology of Mind (1972), but they point mainly to recent use of it by scholars from other fields such as philosophy and psychology (92). The idea of distributed cognitive processes has in fact contributed to much broader and very controversial notions of the mind itself as a distributed entity or, as the most noted proponents of that theory call it, an “extended mind” (Clark and Chalmers). The concept of extended mind is too broad for our present discussion; a more accurate description of what occurs in the examples of artificial servitude we have been examining is the narrower idea of distributed agency developed by Edwin Hutchins. In his study of how the process of navigation aboard a US Navy ship worked, he realized that it was not an operation that was controlled by one person, but instead a process distributed among a whole team, as well as that team’s instruments ( Cognition in the Wild ). Thus, the turning of the ship was dependent upon a distributed agency of individuals and their “cognitive artifacts”—that is, tools that repurpose functional skills into cognitive functions. Navigational calculators are one example of such a cognitive artifact, because they reshape the user’s skill at using the calculator into a cognitive mathematical function (“Cognition, Distributed” 2070). Such systems as Aristotle’s ship, with



245

its combination of human slaves and inanimate tools t hat serve as sensory and limb extensions to the human pilot; Bacon’s android slave serving as his super powerful prosthesis to manipulate the natural world and, like a computer, to help him solve difficult problems; the ship’s computer HAL in the modern story, 2001: A Space Odyssey, which serves not just as an extension of its human pilots but as their surrogate; or a cognitive system comprised of three linked computers and a human pilot, like the ones on the most advanced jets—all are essentially defined in the same way as Hutchins’s systems. They are all distributed cognitive systems—systems that include the master/operator/ programmer within the system via reflexive operations, such as the recursive feedback loops between and among HAL, the ship, and the humans in 2001; or among Bacon, the talking metal head, and t he network of demons that he forces to help him build it. But whereas researchers discuss most such distributed environments as centered on a human who uses a “cognitive artifact” such as a computer to extend his own capabilities, these networks throw that center into question. As Hutchins notes in his discussion of distributed cognition, the success of a human who uses an artifact to solve a problem depends on her ability to manipulate it (“Cognition, Distributed” 2070). But the manipulability of the networks mentioned above is uncertain at best. This is because of the distribution of agency that occurs as a result of distributed cognition. These are networks that are, in essence, comprised of other networks, each of which have their own internal and competing reasons for action. As such, they become autopoietic rather than allopoietic, and they are reflexive, characterized by “a reciprocal, tautological relation in which the components produce a system through their interactions with each other, and the system produces the components as components (rather than isolated parts) through its operation as a system” (Hayles, “From Self-Organization to Emergence” 137; emphasis added). Because such systems are reflexive, even observers—including their makers—become, by definition, complicit in the system whether they i ntend to or not. Someone like Bacon, therefore, who sees himself as having leverage on his system because he made it and thinks that he and it are therefore completely separate, would be operating under an illusion; for his integration into the system is implicit, as noted above. This makes him more reliant on it than he understands, and it less pliable than he would like to think. Thus, there is a crucial tension between the way he tries to treat its components as a network of tools, as a slave-system meant to turn out a maker-defined product, and the system’s status as a cognitive entity with its own agenda—a living, autonomous system with powers that in some ways outstrip his own. Because of their enormous power, all iterations of this archetypal servant, no matter in which era it appears, convey the message that it is dangerous to charge slave-systems with tasks too close to the master’s own, for this engenders an unsustainable dialectical tension in the master-slave relationship that threatens to upend it—or more precisely, threatens to complete the upending process that the master himself has started by delegating too much responsibility to such a potent servant. The owners of these devices in Greene’s two plays allow them


246


to take actions that should be confined to the masters, such as deciding when to go to war in Alphonsus, or solving difficult philosophical and political problems in Friar Bacon . Because they rely on android servants not just to serve as prosthetic enhancements of their own abilities but also as virtual proxies for themselves, the masters unwittingly dilute their own agency, making them, in a circular fashion, even more vulnerable to further displacement of that agency. A modern example of this warning about our blindness to inappropriate use of artificial servants as proxies is given in a more direct way by Norbert Wiener, the father of cybernetics. While doing work in the 1950s that would lead to our present generation of intelligent, autonomous machines, he agonized about the ends of his labor. He worried that we would eventually depend upon those machines too much and would reap the consequences of putting ourselves at the mercy of devices that can make their own decisions: I have said that modern man, and especially the modern American, however much “know-how” he may have, has very little “know-what.” He will accept the superior dexterity of the machine-made decisions without too much inquiry as to the motives and principles behind these. In doing so, he will put himself sooner or later in the position of the fathe r in W.W. Jacobs’ The Monkey’s Paw , who has wished for a hundred pounds, only to find at his door the agent of the company for which his son works, tendering him one hundred pounds as a consolation for his son’s death at the factory. ( The Human Use of Human Beings 184-85)

It is notable how similar the story Wiener relates about the monkey’s paw is to the story of Gerbert’s misleadingly prognosticating android, pointing to a continuity in at least one type of fear represented by artificial servants. And the expression of such fears among real scientists and thinkers of our day does not stop with Norbert Wiener. The views of optimists such as Kurzweil, Moravec, and Brooks notwithstanding, such reservations have become increasingly common among modern cultural thinkers. The twentieth-century intellectual groundings of this dystopian view of technology can be seen most readily in the philosophical writings of Jacques Ellul and Martin Heidegger that appeared around the same time as Wiener’s. In the latter half of his essay “The Question Concerning Technology” (1954), Heidegger worries that we are developing a symbiotic relationship with technology that causes us to “enframe” the world as a “standing-reserve” of mere materials to be measured, categorized, and used in some instrumental way. As we encounter problems caused by this relationship with technology, however, we worsen things by trying to make the technology better, rather than by trying to understand it differently, in a way that will allow us to break free of this symbiotic enframing. Similarly, in Ellul’s view, we now live in an environment that is not defined by nature or society, as it formerly was, but instead by technology. Since environments do not just provide the means for life but mediate all else, we are in effect mediated by technology (133-34). We have moved from making technology adapted to our needs t oward an existence where “human beings have to adapt to it and accept total change” (136).



247

More recently, authors such as Bill Joy, Theodore Roszak, Neil Postman, and Clifford Stoll have also written about their fears that we will allow ourselves to be destroyed or degraded in some way by our intelligent technology. They worry that our sense of identity will be sapped, our social priorities perverted. Postman captures the spirit of these skeptics well with his thesis that America, in particular, is engaged in “the submission of all forms of cultural life to the sovereignty of technique and technology” (52). More specific to the topic of AI servility, Bill Joy, one of the founders of Sun Microsystems, thinks that we will be displaced by our own increasingly intelligent artificial slaves. Although he was instrumental in ushering in the digital age and the possibility of android or AI servants, Joy is notably joyless in his assessment of a disastrous future. “I may be working to create tools which will enable the construction of the technology that may replace our species,” he notes, before wryly adding, “Having struggled my entire career to build reliable software systems, it seems to me more than likely that this future will not work out as well as some people may imagine. My personal experience suggests we tend to overestimate our design abilities” (4). Indeed, the way we use distributed cognition in our modern world has proven to be perilous in a way eerily similar to the ancient examples we have examined. In an “example of how agency and decision-making has become a distributed function involving both human and non-human actors,” N. Katherine Hayles mentions the most modern jets, which are so aerodynamically unstable t[hat they] cannot be succes sfully flown by a human alone. There are three computers on board all running the same software, and they “vote” on what actions to take. If two of the three agree, the plane is flown according to that decision. (The triple redundancy is to minimize the possibility of fatal computer malfunction). (Hayles, “An interview/dialogue”).

Ironically, a recent disaster involving the A330 Airbus has been blamed on such distributed computing. At least four of these planes have experienced a situation where the plane responds to the one defective “vote” of these three computers and puts the plane into a sudden dive in response to some sort of external sensor malfunction. According to experts, the latest of these glitches is a likely candidate for the recent crash of a French airliner on its way from Brazil in June 2009 (Perrow). This takeover of a crucial control system by one of its subsystems (often called “slave-systems” by information technologists) is the result of the kind of hazard worried about by Joy and Wiener, but also by Aristotle, William of Malmesbury, and Robert Greene in their accounts of artificial slaves. The unifying theme in the archetype of the artificial servant is that the scientists who create these systems do not recognize the great degree of autonomy in them, and so do not acknowledge their potential for destabilizing the overall network in which they operate—a network that has as its center, as an often unrecognized element, the scientist himself, and a network that often is much broader than the scientist realizes. So the failure of a supposedly servile element of the network has much wider consequences than anticipated. This


248


archetypal narrative is identifiable in the premodern fictional accounts already discussed, and it is recognizable in modern fiction—the example of the “Skynet” global digital defense network in the T ERMINATOR series of sf movies (1984, 1991, 2003, 2009) comes to mind. But, more ominously, it is also actualized in the A330 Airbus crash, giving weight t o the worries of thinkers such as Joy and Wiener about our general dependency on smart technology and its simple unreliability. Evidently the Association for the Advancement of Artificial Intelligence considers those worries—the stories old and new of balky artificial servants and actual disasters such as the Air France crash—as more than postulations. What they see is the gradual coming of age of Aristotle’s ancient warnings about powerful slaves and their relationships with their masters. Those warnings were meant to keep his audience aware of the dangerously unstable dialectic that exists in a relationship where master and slave comprise a virtual, corporeal network: in a networked environment, both the implication and, paradoxically, the centrality of its creator-observer are always in question. To put it in terms of modern systems theory, we all live in a condition of distributed, interdependent cognition and action. We are not (and never have been, by the evidence of Aristotle’s notions) autonomous with solid boundaries to our “selves.” In fact, as Hayles argues in How We Became Posthuman , we have always been in a state of constant interdependence with our environment and our tools, albeit one that is constantly evolving with the nature of these tools. In this state, our consciousness has never really been as autonomously in control as we imagine. Rather, we are unconsciously dependent on myriad processes and entities both inside and outside of our own bodies—from our own cells to our tools—that are part of an emergent network. Nevertheless, in the case of our theoretical thinking about artificial servants, we have often tended to assume a cool Olympian stance somewhere above the systems we create. But far from standing outside of and being separate from our intelligent tools, we are, as systems theory indicates, always implicated in them. There is no “outside” standpoint for a system’s creator, and even if a programmer/creator realizes this and plants herself at its center, there may be no stable center to that system either. In a network, central nodes can shift, can be decentralized. The archetypal nature of the artificial servant that I have attempted to demonstrate here indicates that the worries of current AI theorists about AI intractability—indeed about an inversion of agency or potency between the human “master” and the artificial “slave”—appear to be essentially the same as those implied by Aristotle: these concerns have, in other words, appeared and repeated themselves throughout the history of thinking about artificial servants—which is a surprisingly long one. Moreover, the anxious article about “the coming superbrain” with which I began indicates that these worries still exhibit a blindness towards the inherent interconnectedness of maker and tool, master and slave—and this compounds the dangers associated with creating artificial servants. For in denying the connection between programmer and system, creator and artifact, master and servant, makers blind themselves to the danger of the dialectical inversion of the master-slave relationship that may



249

occur precisely because of that connection. In this way, the older accounts of creating artificial slaves are accounts of modernity in the making—a modernity characterized by the project of extending the self and its powers, in which the vision of the extended self is fundamentally inseparable from the vision of an attenuated self. NOTES 1. Some key works that discuss the general prehistory of AI and artificial humanoids are those by John Cohen and by Jessica Riskin. There are also works devoted to androids and human automata that were made or depicted in particular premodern periods, such as those by Douglas J. Bruce and by Michael Camille on the Classical period and the Middle Ages; by Minsoo Kang, Jonathan Sawday, and Elizabeth King on the Renaissance; an article by me that focuses on the Renaissance, and also touches on the Classical and medieval periods; and, f or the period running from the eighteenth century to the present, Gaby Wood. See also the interesting essay by Brooks Landon in the March 2011 issue of SFS . 2. N. Katherine Hayles’s How We Became Posthuman (1999) does imply a historical connection between the notions of cybernetics and slavery, because she sees the antagonistic modern visions of AI-human interaction as the result of liberal humanism’s focus on mastery and control—especially of nature. I seek to show an explicit and historical connection between the notions of cybernetics and slavery. 3. The term “cybernetic,” coined by Norbert Wiener in the mid-twentieth century, comes from the Greek “kubernetes,” or steersman. He notes that he picked this word to denote devices that could self-steer or self-regulate ( Cybernetics 19). 4. The Three Laws of Robotics are: first, that no robot may harm a human or, by inaction, allow a human to come to harm; second, a robot must obey any orders given to it by human beings, except where such orders would conflict with the First Law; and third, a robot must protect its own existence as long as such protection does not conflict with the First or Second Law. 5. I am aware of the sweeping nature of the term “archetype,” and am using it purposely here because the evidence evokes it. As Jung used it in the broader sense, and as I mean it here, the term “archetype” signifies that a certain type of being or entity or symbol persists in the mythos of a culture, revealing it to be a significant product of the collective unconscious. 6. I rely here on the definition of organon given by Irwin and Fine in the glossary at the end of their edition of Aristotle’s works. It reads: “ Organon refers to a craftsman’s tools or instruments, which are designed for achieving the craftsman’s end. Since Aristotle believes in final causation, he applies the same term to parts of the body, since they also work (though without design) for an end—the benefit of the whole organism” (Irwin and Fine 592). WORKS CITED Adams, Fred, and Kenneth Aizawa. “Why the Mind is Still in the Head.” The Cambridge Handbook of Situated Cognition. Cambridge: Cambridge UP, 2009. 7895. Aristotle. The Complete Works. Ed. A.S. Barnes. Princeton, NJ: Princeton UP, 1995. ))))) . Selections. Ed. and trans. Terence Irwin and Gail Fine. Indianapolis, IN: Hackett, 1995. Asimov, Isaac. I, Robot . New York: Bantam, 1950.


250


Brooks, Rodney. Flesh and Machines: How Robots Will Change Us. New York: Vintage, 2003. Bruce, Douglas J. “Human Automata in Classical Tradition and Medieval Romance.” Modern Philology 10 (1913): 511-26. Camille, Michael. The Gothic Idol: Ideology and Image-Making in Medieval Art. Cambridge: Cambridge UP, 1989. Čapek, Karel. R.U.R. (Rossum’s Universal Robots). 1921. Trans. Paul Selver and Nigel Playfair. New York: French, 1923. Clark, Andy, and David J. Chalmers. “The Extended Mind.” Analysis 58 (1998): 10-23. Clarke, Arthur C. 2001: A Space Odyssey. New York: New American Library, 1968. Cohen, John. Human Robots in Myth and Science. New York: A.S. Barnes, 1967. Cole, Michael, and Peg Griffin. “Cultural Amplifiers Reconsidered.” The Social Foundations of Lang uage and Thought . Ed. David R. Olson. New York: Norton, 1980. Corsini, Matteo de. Rosaio della Vita. 1373. Ed. Filippo Luigi Polidori. Florence, 1845 . Dickson, Arthur. Valentine and Orson: A Study in Late Medieval Romance. New York: Columbia UP, 1929. Delrio, Martin. Disquisitionum Magicarum Libri Sex. Louvain, 1599. Ellul, Jacques. What I Believe. 1987. Grand Rapids, MI: Eerdmans, 1989. The Famous Historie of Fryar Bacon . 1627. In Early English Prose Romances . Ed. Edmund Goldsmid. Edinburgh, 1886. Gower, John. “Confessio Amantis.” 1390. The Complete Works of John Gower . Ed. G.C. Macaulay. Vol.2. Oxford: Clarendon, 1899-1902, rept. 1968. Greene, Robert. Alphonsus, King of Aragon. 1599. Ed. W.W. Greg. London: Malone Society/Oxford UP, 1926. ))))) . The Honourable History of Friar Bacon and Friar Bungay 1630. Ed. Daniel Seltzer. Lincoln: U of Nebraska P, 1963. Hayles, N. Katherine. “From Self-Organization to Emergence: Aesthetic Implications of Shifting Ideas of Organization.” Chaos and the Changing Nature of Science and Medicine: An Introduction. Ed. Donald E. Herbert. Woodbury, NY: AIP, 1995. ))))) . “An Interview/Dialogue with Albert Borgmann and N. Katherine Hayles on Humans and Machines.” U of Chicago P (1999). Online. 3 June 2009. ))))) . How We Became Posthuman: Virtual Bodies in Cybernetics, Literature, and Informatics. Chicago: U of Chicago P, 1999. Heidegger, Martin. “The Question Concerning Technology.” 1954. The Question Concerning Technology and Other Essays. Trans. William Lovitt. New York: Harper, 1977. 3-35. Heinlein, Robert A. The Moon is a Harsh Mistress . New York: Putnam, 1966. Homer. The Iliad . Trans. Samuel Butler. The Internet Classics Archive. Web Atomic and Massachusetts Institute of Technology. Online. 1 Aug. 2010. Hutchins, Edwin. “Cognition, Distributed.” IESBS - International Encyclopedia of the Social & Behavioral Sciences . Eds. Neil J. Smelser and Paul B. Baltes. 26 vols. Amsterdam: Elsevier, 2001. Vol. 3, 2968-3072. ))))) . Cognition in the Wild . Cambridge: MIT, 1995. Joy, Bill. “Why the Future Doesn’t Need Us.” Wired 8.04 (April 2000): 1-11. Online. 13 February 2008. Kang, Minsoo. “Wonders of Mathematical Magic: Lists of Automata in the Transition from Magic to Science.” Comitatus: A Journal of Medieval and Renaissance Studies 33 (2002): 113-39.



251

King, Elizabeth. “Clockwork Prayer: A Sixteenth-Century Mechanical Monk.” Blackbird: An Online Journal of Literature and the Arts. Dept. of English, Virginia Commonwealth University and New Virginia Review, Vol. 1.1 (Spr. 2002). Online. 7 Jan. 2010. Krishnan, Armin. Killer Robots: Legality and Ethicality of Autonomous Weapons. Farnham, UK: Ashgate, 2009. Kurzweil, Ray. The Singularity is Near: When Humans Transcend Biology. New York: Penguin, 2006. LaGrandeur, Kevin. “The Talking Brass Head as a Symbol of Dangerous Knowledge in Friar Bacon and in Alphonsus, King of Aragon.” English Studies 80.5 (1999): 40822. Landon, Brooks. “Slipstream Then, Slipstream Now: The Curious Connections between William Douglas O’Connor’s ‘The Brazen Android’ and Michael Cunningham’s Specimen Days.” SFS 38.1 (Mar. 2011): 67-91. Lenoir, Timothy. “Makeover: Writing the Body into the Posthuman Technoscape. Part One: Embracing the Posthuman.” Configurations 10:2 (2002): 203-20. Malmesbury, William of. William of Malmesbury’s Chronicle of the Kings of England . 1123. Ed. J.A. Giles. London: 1847. Markoff, John. “The Coming Superbrain.” New York Times 23 May 2009, Week in Review. Online. 18 Dec. 2009. ))))) . “Scientists Worry Machines may Outsmart Man.” New York Times 25 July 2009: Science section. Online. 18 Dec. 2009. Maturana, Humberto R., and Francisco J. Varela. Autopoiesis and Cognition: The Realization of the Living. Vol. 42. Dordrecht, Germany: D. Reidel, 1980. Moravec, Hans. Mind Children. Cambridge, MA: Harvard UP, 1990. Perrow, Charles B. “Can We Trust Our Computers?: Technica l Failures Have Resulted in Several Recent Loss-of-Life Accidents.” Waterloo Region Record , Kitchener, Ont.: 2 Jul. 2009. A13. Postman, Neil. Technopoly: The Surrender of Culture to Technology. New York: Random House, 1993. Riskin, Jessica, ed. Genesis Redux: Essays in the History and Philosophy of Artificial Life. Chicago: U of Chicago P, 2007. Roszak, Theodore. The Cult of Information: A Neo-Luddite Treatise on High-Tech, Artificial Intelligence, and the True Art of Thinking. 2nd ed. Berkeley: U of California P, 1994. Sawday, Jonathan. Engines of the Imagination: Renaissance Culture and the Rise of the Machine. London: Routledge, 2007. Singer, P.W. Wired for War: The Robotics Revolution and Conflict in the 21st Century. New York: Penguin, 2009. Stoll, Clifford. Silicon Snake Oil: Second Thoughts on the Information Highway. New York: Doubleday, 1995. Warwick, Kevin. I, Cyborg. Champaign, IL: U of Illinois P, 2004. Wiener, Norbert. Cybernetics: Or Control and Communication in the Animal and the Machine. Cambridge, MA: MIT, 1948. ))))) . The Human Use of Human Beings: Cybernetics and Society. 2nd ed. Garden City, NY: Doubleday, 1954. Wood, Gaby. Edison’s Eve: A Magical History of the Quest for Mechanical Life. New York: Anchor, 2003. Youings, Joyce. Sixteenth Century England. London: A. Lane, 1984.


252


ABSTRACT This article surveys and analyzes the pre-industrial history of artificial humanoid servants and their historical persistence. The idea of artificial slaves—and questions about their tractability—is present not only in the literature of modern times but also extends all the way back to ancient Greek sources; and it is present in the literature and oral history of the Middle Ages and Renaissance as well. Furthermore, at each of these intervals, this idea is connected with an emotional paradox: the joy of self-enhancement is counterpoised with the anxiety of self-displacement that comes with distribution of agency. The idea of rebellious and dangerous artificial slaves is an archetype that spans Western history and persists not only in the pre-modern and modern imaginations, via stories about rebellious AI servants, but also in ancient scientific accounts and in modern systems theory, which is the basis for real AI.


The Persistent Peril of the Artificial Slave

Recommend Documents