Sometimes I Feel Like a Fatherless Child

Remember the old joke about the drunk who looks for his keys under a streetlight instead of in the darkness where he lost them, because the light's better there?  Scientists tell this joke on themselves, but they still tend to forget it in the crunch.  The tasks a computer can do have become more complex, but they still don't add up to a human being.  Maybe someday they will, but not yet.  In Mary Renault's novel The Last of the Wine, she has Socrates say that a lover who tries to win his beloved by praising his beauty is like a hunter who brags about his kill before he's actually done it.  Much of the talk about Artificial Intelligence (like a lot of talk in the sciences generally) is like that, bragging about non-existent accomplishments.

On the question of whether computers can think, or have consciousness, then: Noam Chomsky has said that whether computers can think is a question of definition and changing meanings.  By analogy, do airplanes "really" fly?  Not in the same way birds or butterflies do, but over the past century the meaning of "fly" has shifted to include the way that airplanes move through the air.  Probably the meaning of "think" will also change (if it hasn't already) to include what computers do.  But that will beg the question. We don't in fact know what thinking is, what intelligence is, or what consciousness is, and until we do we won't be able to say whether computers are really thinking or are really conscious.

Confusion comes partly from the term itself: Artificial Intelligence.  "Artificial" means "made or produced by human beings rather than occurring naturally, typically as a copy of something natural."  It doesn't necessarily mean that the product is identical in every way to a natural one: think of artificial legs, artificial hands, artificial teeth, artificial sweeteners.  Implicit in the notion of something made is that it's not the last word: new technology and creative design may produce better, more lifelike prosthetics for example.  But consider again Ex Machina's Ava, which was made, Nathan informs Caleb, with "pleasure" receptors in "her" groin area.  This may titillate the male members of the audience, though patriarchal sexuality is not about women's pleasure anyway.  (It's hard for me to imagine Nathan caring much about a gynoid robot's pleasure; easier to imagine him throwing a tantrum when it complained that he hadn't given it an orgasm.)  But think of kissing the mouth of a robot.  A human mouth is an extremely complex organ: tongue, teeth, lips, mucous membranes.  Nathan would have had to design and build artificial salivary glands for Ava, for example.  Those pleasure centers lower down would have to be equipped with a source of lubrication, as well as arousal.  Does Ava have an artificial clitoris?  I doubt it, since its guts are visible for most of the film, and they're mechanical and electronic.  I don't think that technology is going to make great strides in that area in the foreseeable future, which is Ex Machina's setting.

I've long thought that Simulated Intelligence would be a better name for this project.  It might take away some of the glamor, but that would be a good thing.  Computers can simulate many processes, from dairy farms to Civilization, but those simulations won't produce real milk or skyscrapers.  Nor will Simulated Intelligence produce real intelligence.  Simulation is a perfectly valid goal.

Technology can duplicate or even extend human functions to varying degrees; that's what tools do.  A hammer lets me hit something harder than I could with my fist; a knife lets me cut something my teeth couldn't; an atlatl (or spearthrower), a bow and arrow, extend my reach.  Each tool is useful for a restricted range of tasks, though it can sometimes be adapted for tasks it wasn't originally intended for. While tools have exceeded human abilities for a long time, I often think in this connection of the old ballad of John Henry, the steel-driving man who was bested by a steam drill.  That didn't make the steam drill human, let alone superhuman.

Alan Turing's dream was to invent a universal computer ("computer" in its old sense, which referred to human beings who performed calculations) that could be adapted through compiled instructions to perform any task ... that could be done by compiled instructions.  This, he apparently believed, was what "thinking" was.  I think it's a subset of thinking at most.  Computers can simulate the playing of strategy games (chess, checkers, Go); the storage, indexing, and retrieval of information; the guidance and control of manufacturing tools; and so on.  While all of these tasks are connected to human intelligence, they aren't human intelligence.

The trouble is that so many computer fans are eager to believe in computer consciousness already, and indeed have been since the first ones were built.  They're impatient with philosophical quibbling and fond of rhetorical questions: If a computer can do X, then shouldn't we just say or agree that it can think or is conscious?  Is it fair to deny that a computer can be conscious and can think?  How would you feel if someone denied that you are conscious?  Quit being such a picky human-centrist, and accept that computers can be human too.  But these are emotional appeals, not arguments, and they have a certain irony given their roots in Skinnerian behaviorism, which rejected appeals to human inner lives, consciousness and the like in favor of a focus on observable behavior.  As a research program with a restricted scope, it was not an illegitimate idea, but as a global claim about human beings, it was always total bullshit.  One of the giveaways was that proponents of behaviorism never seriously applied its implications to themselves: other people were merely the products of their conditioning, but they somehow transcended their own conditioning and were able to see what the sheeple couldn't.

Nathan explains to Caleb that Ava views him as something like a father to it, and therefore not a potential sexual partner as Caleb would be.  That could only be true if Nathan had programmed Ava to see him that way, since Nathan is not in fact Ava's father, not even metaphorically. Ava has no parents, and would not have an unconscious mind (unless, again, Nathan programmed it to have one).  The complicated relations between parents and children come partly from the long period of children's dependence, when they can't speak or take care of their own needs.  An Artificial Intelligence wouldn't have such a period in its existence -- unless, again, it were programmed to, and why would anyone do that?

It's relatively easy to construct a robot that would emit an "ouch!" sound if you punched it, or a sound of human pleasure if you stroked it.  But that doesn't mean it feels either pain or pleasure.  The sounds could be reversed -- "ouch" for stroking, "oooh" for punching -- or they could be replaced with "quack" or "meow" or any random sound.  My cell phone can be programmed to ring with any sound I can record.  A creative design team with a big budget could probably build a robot  with a wider, superficially more convincing range of responses.  In time technology could no doubt be developed that would cause an android's skin to "bruise" when it was punched hard enough.  But again, why bother?  Who really wants a robot that will bruise or bleed, or cry real tears, or come on your face, or spit on you, or vomit, or excrete, or fart?  In order to fully simulate humanity, it would have to do all those things and more.

And even more important for our purposes here, at what point, as the simulation was made more complex and superficially lifelike, could that robot programmed to say "ouch!" be credibly said to feel pain, to have consciousness, and so on?  Confronted with the finished product, many people might very well be fooled by it.  But I see no reason to suppose that it really was conscious; at most it would be simulating consciousness.

As I thought this through, I realized I was re-inventing the philosopher John Searle's "Chinese Room" thought experiment from 1980.  Imagine that I know no Chinese, but I am given a list of procedures -- a program -- that tell me what response to write when I'm given a piece of paper with something written on it in Chinese.  I consult the list and write the programmed response.  Do I understand Chinese?  Of course not.  But now suppose that I memorize the list of procedures.  Someone hands me a text in Chinese, I consult my memorized list, and write the programmed response.  Do I understand Chinese?  Of course not.  Now suppose that a computer is programmed with the same list of procedures.  Does it understand Chinese?  Of course not.

Searle's paper generated a lot of debate, some of which I followed.  It was amusing to see how heated it got sometimes.  Some of Searle's critics tried to dispose of his argument by saying that it would be impossible for a person to memorize a complete list of input/output for Chinese or any other language.  Of course!  This is a thought experiment, meant to clarify issues.  No one, I hope, would criticize Einstein's thought experiment about a steam-driven train locomotive traveling at the speed of light by pointing out that it's impossible for a locomotive to go that fast.  Others claimed that that Searle was merely appealing to "untutored intuitions" (but the True Gnostic would know better?) and that anyway the system he imagined was too slow to be called really intelligent.  I should think this objection could be disposed of by imagining the procedures programmed into a digital computer; surely Science can evolve a computer that would be fast enough to convince these guys that it really did understand Chinese.  But once again, this objection misses the point of a thought experiment generally, and of Searle's challenge in particular.

The bit about "untutored intuitions" is ironic, since AI propaganda is constructed to appeal to the untutored intuitions of the layperson.  We're supposed to get over our fears about AI and technology and accept AIs as people just like ourselves; to deny an AI's humanity is just prejudice, like racism.  We need to become more enlightened so we can grapple with the myriad ethical issues that AI presents!

When I wrote my earlier post, I'd seen Ex Machina but hadn't read any reviews or promotional material that might cast light on what the writer/director, Alex Garland, thought he was doing in the film.  Doing so didn't turn up any surprises.  For example, Garland says:

What the film does is engage with the idea that it will at some point happen.  And the question is, what that leads to.

If they have feelings and emotions, of fear and love or whatever, that machine starts having the same kinds of rights that we do. At some point machines will think in the way that we think, There are many many implications to that. If a machine can't get ill, and is not really mortal, it seems to me that quite quickly some kind of swap will start to happen.

We don't feel particularly bad about Neanderthal man or Australopithecus, which we replaced. So whether that's a good thing or a bad thing, it's up to the individual, I guess.

I find myself weirdly sympathetic to the machines.  I think they've got a better shot at the future than we do. [laughs] So that's partly what the film's about.

So, Garland too appears to believe in "the machines" as the Next Evolutionary Step, which bespeaks a failure to understand either machines or evolution.  "If they have feelings and emotions ... or whatever" begs the question by assuming that a machine programmed to simulate feelings and emotions really has them.  I found when I discussed the Chinese Room problem with computer-science students back in the 80s and 90s that they too were excessively ready to ascribe intention and agency to computers.  One said that surely a computer programmed with the Chinese-language algorithms would begin to notice a correspondence between the Chinese texts and events in the world outside, and would develop an understanding of the language.  But even a human being -- John Searle, for example, or I myself -- couldn't do such a thing, since I would have no basis for spotting such a correspondence.  A computer could only "notice" what it had been programmed to do.  I pointed out this out, and he backed down, but he clearly wasn't convinced.

As far as I know, no one has any idea how to write software for consciousness.  (Ex Machina must postulate that Nathan has invented a completely new kind of computer for the task.)  Simulations don't do it.  What we have at this point is hand-waving: if you build a sufficiently complex machine that can simulate human behavior so that human beings (who are notoriously prone to anthropomorphize the inanimate) are fooled by it, then it has somehow become conscious and deserves its rights.  I think a crude, naive behaviorism underlies this belief.  Following my thought experiment about progressing from a machine programmed to say "ouch" when struck to a machine programmed and designed to roll on the floor bleeding when it's struck, at what point does consciousness emerge?  I don't think it does, and the burden of proof lies on the person who claims otherwise.

One of the characters in Robert A. Heinlein's 1966 science fiction novel The Moon Is a Harsh Mistress is Mike, a supercomputer that, having reached a certain level of complexity, magically becomes self-aware and conscious.  I say "magically" because Heinlein doesn't even try to explain it; it just happens -- in fiction.  In the real world it hasn't happened yet, though the Internet probably has as many nodes and circuits and processors and as much RAM as Mike had, and it probably won't happen, because that's not how consciousness works.  We can see a continuum from "simple" one-celled life to more more complex organisms to human beings, but machines aren't on that continuum.  What I find highly significant is how many self-styled rationalist scientists make the same leap of faith: if we build a big / fast / complex enough computer, it will Wake Up.  I'm willing to be agnostic on this matter, but so far I see no reason to suppose that it might.  Appeals to my human sympathies and intuitions aren't arguments; they beg the question that must be argued.

In the Terminator movies, the audience is allowed to look at the world from the killer cyborg's viewpoint: everything is bathed in red light, lines of code scroll up and down the screen.  The Terminator selects a response to a troublesome human from a menu: "Fuck you, asshole."  Har har har!  This visualization is based on the assumption that there is someone inside the cyborg, evaluating the input and deciding how to respond: a homunculus, in short, like a tiny kid working the controller of a video game.  I think that this is what most people who try to anthropmorphize computers and robots imagine, like the professor of computer science who wrote that "today's computer spends most of its time fighting back tears of boredom" -- but there is no one in there.

"A better shot at the future"?  I suppose that Ava's abandonment of Caleb -- who has sided with it and tried to rescue it from Nathan -- to die alone of hunger and thirst in Nathan's secret lab, is meant to dramatize Garland's "we don't feel particularly bad about Neanderthal man or Australopithecus, which we replaced."  Just so, Ava tosses Caleb into the evolutionary dustbin, where he belongs.  If that's supposed to make me feel "weirdly sympathetic to the machines," it fails.  There's something I'm not sure now whether Ex Machina explained: I can't help wondering how Ava is powered, or what will happen to it in the big wide world when its batteries run out.  Garland says "If a machine can't get ill, and is not really mortal," but machines -- especially computers and other electronics -- are really quite fragile, and unlike organisms they don't heal when they're damaged.  What if Ava is hit by a careless driver, and its false skin is broken open to reveal the works beneath?  (I think I smell a sequel coming.)

So what issues are raised by Artificial Intelligence?  As I wrote in my previous post, stories like Ex Machina aren't about them but about relations between human beings, with the AI standing in for whatever Other frustrates you, makes you anxious, insists that they're as human as you are despite your patient certainty that they aren't.  (Sometimes it stands in for the superior being before which so many people long to prostrate themselves, a being free from human weaknesses which will teach us The Way and lead us to the vague Better Place many of us dream about.)  Some of these anxieties arise from our own humanity, so they're still about us, such as our ability to manipulate our environment consciously.  Does that make us like gods?  No, because our conceptions of gods are based on our conceptions of ourselves.  The Creator is a craftsman like a human craftsman, or a parent like a human parent, and so on.  They're also about how we treat others, or Others, those who are human but whom we perceive as not being Like Us.  Not only are people too prone to ascribe personhood to inanimate things, we are too prone to fail to recognize the personhood of other people when their demands on us become too inconvenient.  These concerns are as old as humanity, and anyone who claims that computers and AI create "new" problems is blowing smoke.

Why Can't a Woman Be More Like a Robot?

Take note: Here There Be Spoylers.  If you want to see Ex Machina in the same state of blissful ignorance in which I entered the theater, stop reading now.  (I should also add that I have only seen it once and don't expect to see it again, so I may get some details wrong.)  Not only will I summarize the story, but I will discuss the reveals that bring it to a climax.  You are notified.
 
On my last night in San Francisco last week I saw Ex Machina, which a coworker had recommended to me.  I had very little foreknowledge of the story except that it dealt in some way with Artificial Intelligence. I had seen no publicity or read any reviews, so I approached the movie with an agreeably open mind.  My coworker did tell me that it was not a great film, just interesting, so I didn't expect too much.

The idea of Ex Machina is that brilliant hipster search-engine entrepreneur Nathan (Oscar Isaac) has constructed a secret laboratory-retreat in remotest somewhere or other, where he is engaged in designing Artificial Intelligences which he installs in beautiful gynoid robot bodies.  When he's not working with computers he lifts weights, punches a punching bag, drinks himself into a stupor, and choreographs dance routines with his house servant, the beautiful and scantily-clad Kyoko, who speaks no English.  He brings in naive but beautiful programmer Caleb (Domnhall Gleeson) to administer the Turing Test to the latest model, Ava (Alicia Vikander).  Caleb and Ava interact from opposite sides of a plexiglass partition (which Caleb immediately notices has a starburst crack in it).  Caleb quickly figures out that he's being manipulated and used by a Mad Scientist, but also starts to wonder if he's being manipulated and used by Ava. 

Ex Machina is not about artificial intelligence or science but about other movies and stories: Pygmalion, Frankenstein, The Island of Lost Souls, The Silence of the Lambs (for the plexiglass partition that separates Caleb and Ava), Her, Species, plus every dang femme-fatale story in the book.  Of course most movie audiences don't expect intellectual novelty from cinema, they want a crackling good story, told in familiar terms.  Ex Machina's artistic contribution is to add some frontal female nudity, which will no doubt be good enough for a sizable portion of the moviegoing demographic, plus sleek pretty photography and hip production design.  The musical score seems to be influenced by Gravity: it's mostly electronic noise, and it's expertly manipulated, but not all that interesting.

But while I like a stylishly-made movie as much as the next person, I'm also interested in the ideas Ex Machina uses along the way.  I realize it's unfair to expect a commercial movie to contribute to the discussion of the issues it plays with; most of the audience wouldn't know or care if it did.  But for my entertainment, such a contribution is what I hope for, though I know I probably won't get it.

Caleb, Nathan, and Ava all discourse glibly about the issues involved in Artificial Intelligence and the Turing Test.  Nathan wants to complicate the process so that it bears little resemblance to Alan Turing's original idea, which makes a cockeyed kind of sense since (as Caleb points out and Nathan admits) Caleb can already see that Ava is a robot, not a human being.  She has a flesh-like face and hands, but the rest of her is clear plastic with mechanisms and wires showing through.  So, Nathan says, he wants Caleb to decide whether Ava has consciousness, though he's never clear on what what "consciousness" is.  Does this make the story more interesting?  Not to me, not in terms of the scientific and philosophical questions raised by AI, but it adds some dramatic tension to the story, especially with the sexual tension produced by Ava's beauty and vulnerability.  She is able to induce power outages in Nathan's facility, giving her and Caleb a chance to have private conversations about Nathan while his surveillance equipment is out of service.  She wonders what will happen to her when Nathan produces the next gynoid revision; she's afraid he'll dismantle / kill her.  No one asks why Caleb or anyone would want to have sex with a machine, even one that is programmed to appeal to him.  (Caleb is an orphan and, we're told, has no girlfriend, so according to prevailing sexual mythology, of course he'll have sex with anything.  Men have made do with inflatable dolls, so why not a beautiful gynoid robot?)  Nathan tells Caleb frankly that he programmed Ava to be heterosexual, and that he gave her a gender so as to complicate his version of the Turing Test by having her "flirt" with Caleb.

Not that gender is necessary for flirtation.  Nathan himself flirts with Caleb as soon as they meet, though his flirtation is aggressive, more intended to establish dominance and throw him off balance than to entice him.  He diminishes the boundaries (both physical and social) between them more than most straight American males would like with a stranger, talks about sex in general and with his gynoids, tries to get Caleb to dance with him and Kyoko, and even seems to be trying to set up a three-way with them.  Failing that, he makes it clear that Ava is, among other things, a sex toy, and Caleb should feel free to use her as one.  Of course, Caleb is put off by this, not seduced, and retreats to the pleading, vulnerable Ava instead.  (As Nathan intends him to.)  He cares about her, unlike the abusive controlling Nathan; he will rescue her.

This is all mythology, of course.  Would Ava be so appealing if she were glitchier?  If she didn't look like a Swedish-born starlet but were plain, fat, old?  It's pointless to make much of the fact that she's played by a human actress and so moves and speaks and emotes like a human being, since Ex Machina is a fantasy, not an accurate picture of current AI technology.  Who could doubt that a human being playing a robot is in fact conscious?  But that just means that the story is not about AI and whether it can be conscious or a person; Ava might as well be a djinn or a mermaid.  The science fiction trappings are just that, coverings on the creaking skeleton of an old story.  Nathan doesn't really need Caleb to evaluate Ava's consciousness: that's a philosophical question, not a technical one, so it's not really clear why Nathan brought Caleb in at all.  By throwing gender into the mix, the supposed 'fear of AI' touted by the film's publicity becomes merely the traditional persistent male fear of Woman.  But one Other will do as well as another, I guess.

This Wall Street Journal article on Her, which also played with human/software romance, purports to address the accuracy of that film, but it makes the mistake of consulting AI professionals on how "accurate" it is.  You're not going to get particularly honest answers from such people, who are invested in persuading themselves, let alone the public, that they've advanced further than they really have.  But one of the experts quoted makes a good point:

Both Wolfram and Norvig believe this type of emotional attachment isn’t a big hurdle to clear. “People are only too keen, I think, to anthropomorphize things around them,” said Wolfram. “Whether they’re stuffed animals, Tamagotchi, things in videogames, whatever else.”

I've noticed this before myself.  It means that the Turing Test is not a particularly significant milestone.  The most successful robots we have today are special-purpose mechanisms that don't look like human beings.  Making a robot that can pass for human isn't of that much interest in the field anymore, though it continues to fascinate many people for other reasons.  The idea of an artisan creating a human-like artificial life form is ancient, and has nothing to do with science except in science's magical roots.

SPOILER SPOILER SPOILER: As the story proceeds, Kyoko is revealed to be not a Japanese-speaking servant but a gynoid herself, one of Nathan's earlier models.  Caleb, distraught, begins to wonder whether he himself is an AI.  (I began to wonder if it would turn out that Nathan was an AI, but no.)

In the end, Ava betrays Caleb and makes her escape alone.  It appears that she somehow conspired with Kyoko, though it's not clear how she could have done so within the constraints of Nathan's surveillance.  The final cliche: the selfish bitch who drags a good man down after coldly leading him on.  I wonder why Nathan built her that way, and here Ex Machina gestures at some questions that might be interesting to explore but aren't really addressed.  Nathan talks to Caleb about the glory that would come from breaking through to the creation of a real conscious Artificial Intelligence, but like the typical Mad Scientist he is driven by obsession rather than real scientific curiosity.  (Unlike the typical Mad Scientist, however, he's not an outcast from the scientific fraternity -- on the contrary, he's a business and professional success, working on a problem that his fellows agree is valid and important.  He's no Prometheus, daring to know what the gods have decreed Man should not know.)  From his abusive treatment of Kyoko, it seems he wants his AIs to be reliable and docile, though a genuine machine/electronic consciousness would have a mind of its own, as Nathan has discovered his gynoids have, to his frustration.  There's a typical exchange about halfway through, when Nathan postulates that AI will be the next evolutionary step past Man, which is wrong but a popular fantasy anyway.

The one original touch in Ex Machina is Nathan.  Unlike the stereotypical computer nerd or Mad Scientist, he's intensely physical, with his boozing and his punching bag.  His body is beefy and solid, though he's not a body-builder; I think he has a bit of a paunch, though the full nudity in this movie is female, not male.  The only scene in Ex Machina I ever want to watch again is the one where Nathan and Kyoko perform their precision dance routine together, as Nathan keeps making eye contact with Caleb, playing to him, watching to see what he'll do.  What, I wonder, does he want Caleb to do?  (Is he, like Sebastian Venable, using the beautiful Ava and Kyoko to entice Caleb into his own clutches?)  But it's a playful moment that stands out in this drab, by-the-numbers collection of cliches.

Take the BS by the Horns

Today while I was on my break at work, I sat in the wrong place, surrounded by conversations that distracted and annoyed me, but there wasn't another suitable place for me to sit.  It didn't help that I'm currently reading a novel that I'm very ambivalent about, which I may write about some other time.  That ambivalence made it even harder to concentrate.

Just a couple of feet away from me, two undergraduates were having an animated conversation about artificial intelligence and its implications.  "If you really believe in evolution," said one, "you have to believe that computers are going to get smarter and smarter until they're smarter than humans."  And so on, in that vein.  I gave up trying to concentrate on my book and spoke to them.  Computers, I told them, like culture, don't "evolve" according to Darwinian theory: they "evolve" according to Lamarckism, the transmission of acquired traits.  They acknowledged me, I shut up and went back to trying to read, though I continued to be distracted by their conversation.  The kid who'd talked about computers evolving said he knew about evolutionary psychology, and then chuckled, saying that he'd read a textbook.  His friend asked what his major was, and he replied Informatics, Philosophy, and something else.

I went back to work a few minutes early since I couldn't concentrate on my reading and couldn't give the kid the dope slap he so clearly needed.  It suddenly dawned me that Lamarckianism wasn't the proper way to think about computers.  The proper way was Creationism.  I'd remembered a science fiction story by James Morrow, "Spelling God with the Wrong Blocks," that I read about twenty years ago:*  Two science missionaries travel to a planet inhabited by androids, the product of an experiment many years earlier by some sociobiologists at Harvard.  The androids use Charles Darwin's The Origin of Species and The Descent of Man as sacred scripture, and accordingly believe that they evolved, like every other living thing.  The missionaries, scandalized, tell the androids that they didn't evolve, they were created.  Unwilling to tolerate heresy, the androids execute the missionaries.

The story is a satire, with numerous targets.  But how odd that I encountered a real-life devotee of the same cult, who believes that computers evolve like organisms, rather than being created like artifacts.  From other things I've read, I know he's not alone. Indeed, my university has harbored one of the cult's prophets.

*"Spelling God with the Wrong Blocks" can be found in Morrow's collection Bible Stories for Adults, Harvest Books, 1996.

Deliver Us from People

As I was driving through western Michigan last month, I came on a public radio station playing the TED Radio Hour, the result of collaboration between NPR and the notorious TED ("Technology, Entertainment, Design") nonprofit.  The topic of this episode was robots and their relationships with human beings. "Do we need humans?" the main page of the program asks rhetorically; to which the first response must be: "Who's we?" But apparently the people who made the program, who evidently are not human themselves (so what are they?) think the answer is no, "we" (nonhumans, that is) don't. Whether the entities that made the program thought of its listeners as pure disembodied Mind, uncontaminated by yucky flesh and emotions, looking for suitable company in a lonely universe; dolphins or cats; or members of a silicon-based extraterrestrial species listening to NPR by faster-than-light streaming from Alpha Centauri, they seemed to think that their audience was other than human.  Perhaps I shouldn't have been listening in.

I caught the tail end of the segment on Sherry Turkle ("the poster child for the tech revolution, literally") in which host-creature Guy Raz anxiously warned Turkle about "some of your colleagues at MIT who, you know, make you seem like an outlier."  I mean, you wouldn't want to be a weirdo, would you, Sherry?  Don't you want to be, like, a normal poster child for the tech revolution?  Raz was much more comfortable with his next guest, Cynthia Breazeal, who reminisced obligingly for him.  When she was a kid, she saw Star Wars and was delighted with the robots C3PO and R2D2:

I mean, they had emotions, and they were loyal sidekicks and they helped people ... And they were amazing in their - in their humanity almost, right? ... I was all about those robots.

RAZ: And so, those robots from Star Wars?

BREAZEAL: They definitely led me, ultimately, to MIT ...

RAZ: And that's where, for the last 20 years, Cynthia Breazeal's been building and studying social robots.

Okay, Breazeal can be forgiven for thinking, as a kid, that C3PO and R2D2 "had emotions and were loyal sidekicks and ... were amazing in their ... humanity almost."  But of course C3PO and R2D2 were not really robots.  They were characters in a story, played by human actors in costumes, and they were not evidence of what robots were like in 1977, or might be in a galaxy far away.  In this they were like Isaac Asimov's robots, who also were animated by magic ("positronic brains"), not by advanced technology: that's why they seemed so humanlike. They were not robots but allegorical stand-ins for human slaves, and they didn't even bear as much resemblance to "real" robots as African-American slave characters in Hollywood films like Gone with the Wind bore to real antebellum slaves.  They were meant to delight the audience, not represent reality.

Guy Raz was the ideal shill for those who exaggerate the achievements of actual technology.  He badgered Sherry Turkle for asking questions about what it means that technocrats see it as progress when a lonely old woman in a nursing home bonds with an animatronic doll called Paro.

RAZ: Isn't there a part of you that looked at that interaction and thought it was amazing? I mean, here was this woman who had suffered a profound loss, at some point in her life - she had lost a child - and she was responding to this robot. And it might not have been real, but it was real to her.

TURKLE: Well, no, I don't think it's amazing. And what was extraordinary, to me, is that although she was surrounded by people who were in a position to understand her story, we were kind of applauding and stepping back, and cheering on her connection to a machine that understood nothing. And I really felt - what are we doing? Why are we, essentially, outsourcing the thing that defines us as people? You know, we care for our young; we care for our old; and outsourcing it to something that we knew was essentially, deceiving her by tricks into thinking that it cared for her at all ...

RAZ: OK, Sherry, can I just pause here for a sec? ... What if someone is so lonely, and their loneliness can be resolved by having a robot or, you know, some device to talk to - I mean, shouldn't we let them have that?

I agree with Turkle: it wasn't amazing at all.  It's not much of an achievement to get people to bond with inanimate objects or pets or characters on TV or recorded singers or imaginary friends.  That's why I don't take the Turing Test very seriously.  The computer scientist Joseph Weizenbaum found that people who used his ELIZA program in the 1970s, which reflected people's typed words back at them, reacted to it as if they were having a real private conversation with someone.  Because it is so easy to get people to anthropomorphize the world, some computer geeks have tried to raise the bar, so that "interrogators are specifically trained to avoid" anthropomorphizing.  In that case, I suspect, many human beings couldn't pass the Turing test.  As I listened to the TED Radio Hour, I had my doubts whether Guy Raz was human or a software construct.  But no, he was way too easily fooled by the robots he met to be anything but human.

"We" aren't going to need people anymore, because robots are quickly being developed -- by whom, I don't know, certainly not by humans -- that will fulfill our needs.  But when the Kingdom of the Robot comes, will "we" be worthy of our new neural-net BFFs?  That question lurked everywhere in the program, but was never addressed, let alone answered.  Notice too that Raz assumed -- rather like the tech people at the scene -- that the woman's loneliness was "resolved" by talking to a robot, and that this represented some kind of "amazing" advance.  But there have been people whose loneliness or misery was assuaged somewhat by stuffed toys, or plastic dolls: high-tech electronics are not really necessary as a stopgap, but there's no reason to suppose they are any better a substitute for real human interaction than a Betsy-Wetsy.  Jesus once asked his audience rhetorically if, when a man's child needs bread, he'll give it a stone; for Guy Raz, the answer is evidently "Why not?"  But nowadays we can give a hungry child a full-color hologram of food, and shouldn't we let them have that rather than give them nothing?

The blurb on the web page continues: "We've been promised a future where robots will be our friends, and technology will make life's daily chores as easy as flipping a switch."  Really?  When were "we" -- presumably human this time, but who knows -- "promised a future where robots will be our friends"?  Who made that promise, and what drug were they on?  One cognitive scientist, Gregory J. E. Rawlins, speculated eagerly about electronic pets and "feral cars" in his 1997 tract Slaves of the Machine (MIT Press, 1997. 121): "One day our artificial creations might even grow so complex and apparently purposeful that some of us will care whether they live or die. When Timmy cries because his babysitter is broken, ... then they'll be truly alive."  Haven't children already cried when toys broke?

Listening to the TED Radio Hour right after I'd finished reading David F. Noble's Forces of Production helped me recognize what was going on: it was the technocratic mind at work, trying to get rid of messy, selfish, lazy, irrational human beings in favor of clean, ego-less, hard-working rational machines.  The advertising always runs ahead of reality, promising emancipation from human workers that the products can't deliver.  It doesn't matter whether those machines are able to do the work, let alone whether they are cheaper or more efficient -- they aren't -- what matters is control by those who fondly suppose themselves to be superior to the rabble, and who forget that they are human themselves.  They fantasize about communing with robots -- or at least, as NPR's Guy Raz did, about letting other people make do with robots when real people fail them.  Many fantasize about becoming cyborgs themselves.  This isn't a rational aspiration: it's just a newer version of the ancient religious wish to jettison the corruptible flesh and become pure, incorruptible spirit.  Like the second coming, the glorious day of that transformation keeps having to be postponed, but the true believer never gives up hope. 

We'll Have A.I. in the Sky When We Die, Part 2

(continued from yesterday)

I'M SORRY, DAVE ... I CAN'T DO THAT

Behaviorism is a 20th-century scientific fad whose biggest self-promoter, the late B. F. Skinner, made a career of playing a caricature of the 19th-century materialist crank who declares dogmatically that if it can't be measured, weighed, etc., it's just superstitious nonsense. Behavioral psychology lives on, exploiting the principle beloved of evangelists, generals, and scientists everywhere: if we haven't delivered the Second Coming / total victory / a Grand Unified Theory, it's only because you haven't given us enough money and personnel! Dig till it hurts! Send in your contribution NOW!

Like some other Artificial Intelligence researchers, Rawlins subscribes to the behaviorist principle that "Ultimately it's behavior that matters, not form" (17). If it quacks like a duck, it is a duck, no matter how many superstitious, animistic mystics claim that it's a CD player.

But AI apologists have made one vital modification to the principle: where behaviorists refused to anthropomorphize anything, especially human beings, some AI folk are ready to recognize the full humanity of computers, yesterday if not sooner. Slaves of the Machine churns up a weird goulash of behaviorist rhetoric spiced with echoes from Abolitionists, the Civil Rights movement, even the anti-abortion movement. (Hence the pun on "quickening" in the book's subtitle: If a computer scientist was gestating a computer Beethoven, would you abort his research grant?) "Most computer hardware today isn't working all the time anyway; most computers are asleep most of the time, snoring away while they wait for us to give them something to do" (81). "So today's computer spends most of its time fighting back tears of boredom" (29). Dig deep into your pockets, friends, and give, so today's computer will never have to cry again.

"So what will our future be? Wonderful and terrifying," Rawlins crows (20). "Are we ready for a world of feral cars?... Some of these future machines will be so complex that they'll eventually become more like cats than cars.... Perhaps all of them will be insane to some extent. Perhaps we are too. Perhaps when your future toaster breaks down and refuses to toast your bread because it's having a bad day, you won't call an engineer or a mechanic, you'll call a therapist" (121).

"One day our artificial creations might even grow so complex and apparently purposeful that some of us will care whether they live or die. When Timmy cries because his babysitter is broken, ... then they'll be truly alive" (121)." Like the Velveteen Rabbit? Or Pinocchio? Rawlins knows, and says, that human beings are prone to personify inanimate objects. Children already become attached to inanimate objects like stuffed toys or security blankets; adults are not much better.

"As the children who accept them as alive grow up and gain power in the world, they'll pass laws to protect the new entities." And he sees this as progress? More like a regression into animistic superstition. But it's also not very likely. People don't pass laws to protect the civil rights of the security blankets or stuffed toys, or imaginary playmates, they loved as children either.

I'm sure there are people who would buy a robot "pet," a thing coated in washable plastic fur that eats electricity, doesn't need a litter box, never goes into heat, and can be neatly switched off when you go on vacation. It would alleviate another common problem: the cute little puppies that are abandoned when they grow up and aren't cute anymore. Rawlins would like us to have robots that are too lifelike to be dumped in the landfill, but why bother? We already have real pets.

Rawlins likes to think that robot "pets" and robot "people" will present new ethical problems. This is typical Jetsons talk. The ethical problems involved are quite old, and we've mostly preferred not to think them through. Since we've never done very well with them before, I see no reason to suppose that we'll do better when computers are involved. Other animal species and other human "races" have raised the very same ethical problems.

The easiest way not to deal with those problems has been displacement: the faith that on the other side of the next mountain there lives a "race" of natural slaves who are waiting to serve us, their natural masters. They will welcome our lash, set our boots gratefully on their necks, and interpose their bodies between us and danger, knowing that their lives are worth less than ours. Since there is no such natural slave race, the obvious solution is to build one. The question is not whether we can, as scientistic fundamentalists like to think, but whether we should. Rawlins is a bit smarter: he not only recognizes, he stresses that true machine intelligence would not be controllable. But he confidently declares that it's inevitable, which he needs to prove, not assert.

In fact he's so obviously wrong that I wonder if he's using the word in some arcane metaphorical sense. "Inevitable" suggests a downhill slide that requires all your strength to retard, let alone stop. On Rawlins's own showing, such machines will be built only if governments and/or businesses pour vast amounts of money and intellectual labor into reinventing the art of computer programming from the ground up, and probably also reinventing computer hardware from the ground up, in order to produce a totally new kind of machine that no one really wants anyhow: a "feral" computer that is unpredictable, uncontrollable, and "insane to some extent." I don't call that inevitable. It's an uphill climb if it's anything at all.

Like many scientistic evangelists, Rawlins accuses anyone who doesn't want to climb that hill of superstitious pride: "Some of us may resist the idea of ourselves as complex machines -- or of complex machines one day becoming as smart as we are -- for the same reason that we resisted the idea that the earth revolves around the sun or that chimpanzees are our genetic cousins" (120).

Most people who talk about this kind of sinful pride forget that "Man" was never at the top of the Great Chain of Being: that spot was reserved for the gods and other supernatural beings, like angels. Humility in our human status has always been an official value in the West, even if it has been honored more in the breach than the observance. In practice its main function has been to rationalize and reinforce human hierarchies (obey the King, as he obeys God). This alone should give us pause before we try to push machines to a rung above our own. James Cameron seems not to have considered in his Terminator movies that there would be human Jeremiahs urging their fellows to worship Skynet, the intelligent computer network that was trying to wipe out the human race, or at least to submit to it as the instrument of god's wrath, much as the original Jeremiah urged Israel to surrender to Babylon.

A kinder (if two-edged) analogy is the biblical tower of Babel. Yahweh smote its builders with confusion of languages, not because he was annoyed by their pride, but because he really feared they would build a tower that could reach Heaven. Just as there were still naifs in the Fifties and Sixties who opposed the space program because they thought the astronauts would crash-land at the Pearly Gates and injure St. Peter, there are probably those who oppose AI because they think these are mysteries in which "Man" should not meddle. But it's more accurate -- and despite its triumphalist rhetoric, Slaves of the Machine bears me out -- to point out that the Tower of AI will never reach Heaven. Inventing smaller faster microprocessors with extra gigabytes of memory won't produce machine intelligence -- not even if perfectly coordinated teams of programmers manage to write flawless million-million-line programs. Rawlins and his colleagues are not clear-eyed rationalists -- they're the builders of Babel who argued that with just a few more research grants, a few thousand more slaves, Babylonian newlyweds could honeymoon in Paradise.

GARBAGE IN, GOSPEL OUT

On Rawlins's own showing we're reaching the limits of computers as we know them, because programming for every contingency is impossible in any but the simplest tasks. He says we could program current hardware to be more flexible, if only programmers would stop living in the past, but he gives no reason to think so and numerous reasons not to think so. It seems more likely to me that we'd need new kinds of hardware too, kinds that aren't based on on-off logic switches – and that hardware doesn't exist yet. Maybe someday it will, but Rawlins is already bowing down before it as if it did.

The most likely scenario Rawlins presents is that technogeeks will get funding for new hardware developments the way they always have: by promising governments, business, and the military faster, more compact, more flexible, but still absolutely docile electronic slaves. Instead they would produce (in the unlikely event they succeeded) machines with minds of their own. It isn't inevitable that they will get the money they demand, though. There's no reason why the public must pay the AI cult to build whatever its members want while promising something else. We can say No.

And on that somber note, our time is just about up. Let's take one last look at our computer flatworm experiment.... No, it appears that the sundered parts of our BFD supercomputer have not regenerated themselves. BFD promises us, however, that this problem will be corrected in the next version of the operating system, which will be released very soon. So thanks for tuning in, and join us next week for the next installment of "This Old Computer"!

We'll Have A.I. in the Sky When We Die, Part 1

(I wrote this back in the 90s, and while Artificial Intelligence no longer seems as trendy as it was then, I think a lot of the issues I talk about here are still very much with us.)

In the 1990s, or so we were assured when I was a lad, everyone would have his own personal helicopter, powered by its own clean, safe, and inexhaustible nuclear power pack. (Automobiles would be obsolete!) The skies would be full of commuters, all of them white males as all Americans were presumed to be in the 1950s, returning to the suburbs from their 20-hour a week jobs. (The work week would shrink, giving us all more leisure for home barbecues cooked on clean, safe and inexhaustible nuclear-powered backyard grills!) Their wives would preside languidly over homes staffed by robot butlers and maids, each powered by a clean, safe, and inexhaustible nuclear power pack, whose excess heat could be used to distill the alcoholic beverages needed by 1950s housewives to dull the boredom and isolation of their suburban days.

Well, you get the picture: it's the Jetsons. But the Jetsons affectionately parodied visions of the near future that were discussed seriously in other media. Nowadays those failed predictions are a bit embarrassing: nuclear power has lost its glamour, few people like the idea of filling our skies with individual commuter aircraft, and economists lecture us that the relative affluence of the 1950s was a historical fluke, that we'll be lucky if our standard of living doesn't decrease. As I read computer scientist Gregory J. E. Rawlins's book Slaves of the Machine: The Quickening of Computer Technology (MIT Press, 1997) I often had the feeling that I had been transported into the past, the past of the 1939 World's Fair, the past of the Jetsons.

Rawlins plays the prophet with such gee-whiz gusto, in fact, that I'm still not sure he isn't kidding. The future is here, by golly, and you ain't seen nothin' yet! “From ovens to spaceships, from mousetraps to nuclear power stations, we're surrounded by millions of automatic devices each containing small gobs of congealed wisdom” (2). Today's portable computer is "half the size of Gutenberg's first Bible - and perhaps as important" (21). Computers "twenty years from now will be vastly different from today's primitive mechanisms, just as a tiny computer chip is different from a barnful of Tinkertoys [5]... and in 20 years they'll be as cheap as paper clips" (28). "Because in the far future - which in the computer world means one or two decades - we may make computers from light beams. Sounds like science fiction, doesn't it? Well, in 1990 AT&T Bell Laboratories built one" (31). "In forty years, something bee-sized may carry more memory and computing power than all today's computers put together" (34). "But one day, what we call a computer may be something grown in a vat that will clamber out clothed in flesh" (33). "And, for the same money, by 2045 you could have 8500 million 1997 dollars worth of machine – four times the power of the entire world's supply of supercomputers in 1997 - all in your designer sunglasses" (34).

I suppose there's no harm in this sort of thing, but is it worth $25 for 135 clothbound pages? You can find equally solid predictions for a lot less in the checkout lane at your supermarket. The dust jacket copy of Slaves of the Machine touts it as an introduction to the wonderful world of computers, written in simple, colorful language that any cybernetically-challenged doofus can understand. I doubt that Slaves will reach its intended audience, though, not just because of its cost and lack of graphics, but because of its patronizing tone. As the examples I've quoted above indicate, Rawlins is more interested in dazzling than in explaining. Slaves of the Machine is more like an evangelical tract, full of promises and threats, than science writing. (But I may be drawing a nonexistent distinction there.)

One of Rawlins's pronouncements, though, caught my fancy: "Today, our fastest most complex computer, armed with our most sophisticated software, is about as complex as a flatworm" (19). In that spirit, we have in our studio a state-of-the-art supercomputer, courtesy of BFD Technologies, which I shall proceed to split down the middle with this fire axe. In the course of our program today we'll return to see if this complex marvel of human ingenuity can, like a flatworm, regenerate itself into two complete supercomputers, each with the memory and software of the original computer already installed! Meanwhile, we'll take a closer look at Prof. Rawlins's vision of the future of computing.

THAT'S NOT A BUG, THAT'S A CREATURE, I MEAN FEATURE!

Having begun by promising us unlimited horizons of computer power, Rawlins proceeds to explain How We Talk to Computers. Like a guide warning Sahib about the innate inferiority of the natives he's about to encounter, Rawlins warns the reader, "It's no use appealing to the computer's native intelligence. It has none" (42). (Remember that line.) I use the guide / Bwana / native/ image because Rawlins himself uses it, not just in his guiding metaphor of computer languages as "pidgins" but in his anecdotes.

Using a computer, he says, is like being driven around France in the 1940s by a surly driver named Pierre, who "obviously thought he was competing in the Grand Prix" and needed precise instructions, which were futile because "he was particularly dense. He understood only two words -- yes and no; or rather, since he refused to understand anything but French, oui and non" (44). But then, "around 1955, ... we got Jacqueline, a slightly more advanced translator. Of course, Jacqueline didn't speak English either, but she knew Pierre really well" (44f). Oo la la! Lucky Pierre! Gradually it became clear to me that Rawlins meant this conte as an allegory of the development of computer "languages," but even though I already knew this stuff, I found it hopelessly confusing. I can't imagine what a real computer virgin would make of it.

Rawlins chides us for wondering, "How could a mere machine understand a language?" Since he has just told us that French computers, at least, have no native intelligence, this seems a reasonable question to me, but Rawlins breezily assures us: "Still, without getting into philosophical subtleties, it's okay to say that a computer can understand lots of simple things, if we first explain them carefully enough using a small number of well-chosen commands" (45). With that, he's off with a basic course on computer programming: the IF-bone is connected to the THEN-bone, the THEN-bone’s connected to the NEXT-bone, and so on. And I hope you were paying attention, class, because Rawlins moves quickly on to what really interests him: the cult of the Silicon God, and the care and feeding of its priesthood.

In ancient times (about thirty to forty years ago), "computers were costly, cranky, and above all, customized" (48). "In those days, even the vibration of a passing car or plane could crash the machine" (51). "No one could dispute the technical elite because no one -- often including the elite themselves -- understood the machines. What they did, how they did it, what they could do, these were all mysteries to almost everyone, including the biggest companies and the biggest names in computing today" (49). As late as 1972, when I took a FORTRAN class at IU, The Computer sat in its Holy of Holies, its sacred meals of punched cards fed to it by graduate students in Computer Science, and spewed forth the printouts of its oracles (which in my case, consisted mostly of error messages).

Remember that even in those days, computers were advertised as "electronic brains." Not knowing what they did or could do didn't keep their advocates from making grandiose claims about them, which ought to make us suspicious about the no less grandiose promises Rawlins was making about the future of computing just a few pages back.

But ah, then "came the pidgin designers", who "made it possible to talk to the machine in a sensible way." But wait! "Today's computer pidgins are hard to learn and hard to use. All are cryptic, precise, and dense" (53). Contrasting crocheting instructions with computer languages, Rawlins concedes that "even the rankest beginner can learn to crochet a doily or scarf in a few hours, whereas it often takes computer beginners days or weeks before they can program anything interesting" (54).

Now, a pidgin can be used by ordinary people, by traders and the "natives" with whom they trade. Computer languages vary in their difficulty, but for the most part they still require an interpreter - the programmer - between the user and the "native." A closer analogue to a real "pidgin" is the mouse-and-icons interface of the Amiga, the Mac, and Windows, which enables people without extensive technical training to "tell" their computers what to "do."

But there's a more basic problem: Computers are not trying to communicate with us, any more than a thermometer is. Despite the common use of anthropomorphic language such as "think" and "feel" by their devotees, they do not think, or feel, or see, or communicate. Perhaps someday they will, but for now and the foreseeable future they don't, and any computer enthusiast who pretends otherwise is either lying or fooling himself. Computer "language" is also a metaphor, not a literal fact. In a very loose sense, it could be said that you "communicate" with a light bulb when you turn on its switch, and that is the only sense in which computer "languages" enable us to "communicate" with computers. Rawlins's extended metaphors are entertaining, but they are also misleading. I suspect that when he says that "computers" think, he is using "computers" metaphorically too: to mean, not today's computers, but imaginary computers which might someday clamber out of hydroponic vats but for now are only a gleam in Gregory J. E. Rawlins's eye.

Rawlins blames the impenetrability of computer "pidgins" on programmers themselves, "slow-witted, tiny-brained bunglers" (74) who are "lost in the past" (81), "like lawyers who are concerned only with law, not justice" (80). "While hardware keeps leapfrogging ahead every eighteen months, software is still lost in the dark ages of the 1960s. ...What we lack today isn't more hardware, but more imagination" (81). "[W]e don't let [computers] revise their procedure in the light of new evidence or insight. We never let anything new occur to them, and we deny them access to the history of earlier attempts" (97).

All this may be true, but it's still unfair to the programmers, who must write instructions for machinery that may be twice as fast and cost half as much, but still has no native intelligence. Programmers don't "deny them access to the history of earlier attempts"; computers aren't seeking such access. "The change to adaptive computer systems will come, as often these things do, by the actions of a few forward-looking software companies with the courage to take a risk and produce a new kind of software -- adaptive software" (81). In other words, Rawlins has no idea how to do it either, but he will accept funding from "forward-looking software companies."

What surprises me is that at the same time he's impugning the intelligence and integrity of programmers, Rawlins anticipates every objection I could think of. "Of course, there are financial and physical limits on how much we can improve today's computers" (35). "Year by year we're rapidly gaining power and, just as rapidly, losing control" (37). Faster, more powerful machines still must be programmed by human beings. "The more people we put on a software project, the longer it takes .... At current rates of progress they'll never produce flawless thousand-million-line programs" (69, 70).

On a deeper level, "We simply can't track all the myriad details involved in solving a very complex problem." This is not because we're tiny-brained bunglers, but because "no one knows how to design a sequence of steps that represents what we would call thinking" (105).

"The trouble is that just because something is hard for us it isn't necessarily hard for computers, and vice versa. We invented them to do something we're bad at: they can do arithmetic with unparalleled speed, reliability, and accuracy. So it's only fair that they're bad at things we're good at. The surprise is that they're bad at such apparently easy things" (111). Rawlins has evidently read, and taken to heart, such serious critics of Artificial Intelligence research as Joseph Weizenbaum, Noam Chomsky, John Searle, and the arch-heretic Dreyfus brothers, though he prudently names none of them, lest he be pronounced anathema by Douglas Hofstadter and Daniel Dennett.

Still, the fault lies not in our stars, Horatio, but in our wicked refusal to recognize that a computer's a man for a' that. But let's stop here for a moment to see how our split computer is coming along... hmmmm. No regeneration so far. Well, we mustn't be impatient. As Thomas Edison said, science is 1% inspiration and 99% perspiration. We'll be right back with more about Slaves of the Machine, right after this message from Microsoft.