Sometimes I Feel Like a Fatherless Child

Remember the old joke about the drunk who looks for his keys under a streetlight instead of in the darkness where he lost them, because the light's better there?  Scientists tell this joke on themselves, but they still tend to forget it in the crunch.  The tasks a computer can do have become more complex, but they still don't add up to a human being.  Maybe someday they will, but not yet.  In Mary Renault's novel The Last of the Wine, she has Socrates say that a lover who tries to win his beloved by praising his beauty is like a hunter who brags about his kill before he's actually done it.  Much of the talk about Artificial Intelligence (like a lot of talk in the sciences generally) is like that, bragging about non-existent accomplishments.

On the question of whether computers can think, or have consciousness, then: Noam Chomsky has said that whether computers can think is a question of definition and changing meanings.  By analogy, do airplanes "really" fly?  Not in the same way birds or butterflies do, but over the past century the meaning of "fly" has shifted to include the way that airplanes move through the air.  Probably the meaning of "think" will also change (if it hasn't already) to include what computers do.  But that will beg the question. We don't in fact know what thinking is, what intelligence is, or what consciousness is, and until we do we won't be able to say whether computers are really thinking or are really conscious.

Confusion comes partly from the term itself: Artificial Intelligence.  "Artificial" means "made or produced by human beings rather than occurring naturally, typically as a copy of something natural."  It doesn't necessarily mean that the product is identical in every way to a natural one: think of artificial legs, artificial hands, artificial teeth, artificial sweeteners.  Implicit in the notion of something made is that it's not the last word: new technology and creative design may produce better, more lifelike prosthetics for example.  But consider again Ex Machina's Ava, which was made, Nathan informs Caleb, with "pleasure" receptors in "her" groin area.  This may titillate the male members of the audience, though patriarchal sexuality is not about women's pleasure anyway.  (It's hard for me to imagine Nathan caring much about a gynoid robot's pleasure; easier to imagine him throwing a tantrum when it complained that he hadn't given it an orgasm.)  But think of kissing the mouth of a robot.  A human mouth is an extremely complex organ: tongue, teeth, lips, mucous membranes.  Nathan would have had to design and build artificial salivary glands for Ava, for example.  Those pleasure centers lower down would have to be equipped with a source of lubrication, as well as arousal.  Does Ava have an artificial clitoris?  I doubt it, since its guts are visible for most of the film, and they're mechanical and electronic.  I don't think that technology is going to make great strides in that area in the foreseeable future, which is Ex Machina's setting.

I've long thought that Simulated Intelligence would be a better name for this project.  It might take away some of the glamor, but that would be a good thing.  Computers can simulate many processes, from dairy farms to Civilization, but those simulations won't produce real milk or skyscrapers.  Nor will Simulated Intelligence produce real intelligence.  Simulation is a perfectly valid goal.

Technology can duplicate or even extend human functions to varying degrees; that's what tools do.  A hammer lets me hit something harder than I could with my fist; a knife lets me cut something my teeth couldn't; an atlatl (or spearthrower), a bow and arrow, extend my reach.  Each tool is useful for a restricted range of tasks, though it can sometimes be adapted for tasks it wasn't originally intended for. While tools have exceeded human abilities for a long time, I often think in this connection of the old ballad of John Henry, the steel-driving man who was bested by a steam drill.  That didn't make the steam drill human, let alone superhuman.

Alan Turing's dream was to invent a universal computer ("computer" in its old sense, which referred to human beings who performed calculations) that could be adapted through compiled instructions to perform any task ... that could be done by compiled instructions.  This, he apparently believed, was what "thinking" was.  I think it's a subset of thinking at most.  Computers can simulate the playing of strategy games (chess, checkers, Go); the storage, indexing, and retrieval of information; the guidance and control of manufacturing tools; and so on.  While all of these tasks are connected to human intelligence, they aren't human intelligence.

The trouble is that so many computer fans are eager to believe in computer consciousness already, and indeed have been since the first ones were built.  They're impatient with philosophical quibbling and fond of rhetorical questions: If a computer can do X, then shouldn't we just say or agree that it can think or is conscious?  Is it fair to deny that a computer can be conscious and can think?  How would you feel if someone denied that you are conscious?  Quit being such a picky human-centrist, and accept that computers can be human too.  But these are emotional appeals, not arguments, and they have a certain irony given their roots in Skinnerian behaviorism, which rejected appeals to human inner lives, consciousness and the like in favor of a focus on observable behavior.  As a research program with a restricted scope, it was not an illegitimate idea, but as a global claim about human beings, it was always total bullshit.  One of the giveaways was that proponents of behaviorism never seriously applied its implications to themselves: other people were merely the products of their conditioning, but they somehow transcended their own conditioning and were able to see what the sheeple couldn't.

Nathan explains to Caleb that Ava views him as something like a father to it, and therefore not a potential sexual partner as Caleb would be.  That could only be true if Nathan had programmed Ava to see him that way, since Nathan is not in fact Ava's father, not even metaphorically. Ava has no parents, and would not have an unconscious mind (unless, again, Nathan programmed it to have one).  The complicated relations between parents and children come partly from the long period of children's dependence, when they can't speak or take care of their own needs.  An Artificial Intelligence wouldn't have such a period in its existence -- unless, again, it were programmed to, and why would anyone do that?

It's relatively easy to construct a robot that would emit an "ouch!" sound if you punched it, or a sound of human pleasure if you stroked it.  But that doesn't mean it feels either pain or pleasure.  The sounds could be reversed -- "ouch" for stroking, "oooh" for punching -- or they could be replaced with "quack" or "meow" or any random sound.  My cell phone can be programmed to ring with any sound I can record.  A creative design team with a big budget could probably build a robot  with a wider, superficially more convincing range of responses.  In time technology could no doubt be developed that would cause an android's skin to "bruise" when it was punched hard enough.  But again, why bother?  Who really wants a robot that will bruise or bleed, or cry real tears, or come on your face, or spit on you, or vomit, or excrete, or fart?  In order to fully simulate humanity, it would have to do all those things and more.

And even more important for our purposes here, at what point, as the simulation was made more complex and superficially lifelike, could that robot programmed to say "ouch!" be credibly said to feel pain, to have consciousness, and so on?  Confronted with the finished product, many people might very well be fooled by it.  But I see no reason to suppose that it really was conscious; at most it would be simulating consciousness.

As I thought this through, I realized I was re-inventing the philosopher John Searle's "Chinese Room" thought experiment from 1980.  Imagine that I know no Chinese, but I am given a list of procedures -- a program -- that tell me what response to write when I'm given a piece of paper with something written on it in Chinese.  I consult the list and write the programmed response.  Do I understand Chinese?  Of course not.  But now suppose that I memorize the list of procedures.  Someone hands me a text in Chinese, I consult my memorized list, and write the programmed response.  Do I understand Chinese?  Of course not.  Now suppose that a computer is programmed with the same list of procedures.  Does it understand Chinese?  Of course not.

Searle's paper generated a lot of debate, some of which I followed.  It was amusing to see how heated it got sometimes.  Some of Searle's critics tried to dispose of his argument by saying that it would be impossible for a person to memorize a complete list of input/output for Chinese or any other language.  Of course!  This is a thought experiment, meant to clarify issues.  No one, I hope, would criticize Einstein's thought experiment about a steam-driven train locomotive traveling at the speed of light by pointing out that it's impossible for a locomotive to go that fast.  Others claimed that that Searle was merely appealing to "untutored intuitions" (but the True Gnostic would know better?) and that anyway the system he imagined was too slow to be called really intelligent.  I should think this objection could be disposed of by imagining the procedures programmed into a digital computer; surely Science can evolve a computer that would be fast enough to convince these guys that it really did understand Chinese.  But once again, this objection misses the point of a thought experiment generally, and of Searle's challenge in particular.

The bit about "untutored intuitions" is ironic, since AI propaganda is constructed to appeal to the untutored intuitions of the layperson.  We're supposed to get over our fears about AI and technology and accept AIs as people just like ourselves; to deny an AI's humanity is just prejudice, like racism.  We need to become more enlightened so we can grapple with the myriad ethical issues that AI presents!

When I wrote my earlier post, I'd seen Ex Machina but hadn't read any reviews or promotional material that might cast light on what the writer/director, Alex Garland, thought he was doing in the film.  Doing so didn't turn up any surprises.  For example, Garland says:

What the film does is engage with the idea that it will at some point happen.  And the question is, what that leads to.

If they have feelings and emotions, of fear and love or whatever, that machine starts having the same kinds of rights that we do. At some point machines will think in the way that we think, There are many many implications to that. If a machine can't get ill, and is not really mortal, it seems to me that quite quickly some kind of swap will start to happen.

We don't feel particularly bad about Neanderthal man or Australopithecus, which we replaced. So whether that's a good thing or a bad thing, it's up to the individual, I guess.

I find myself weirdly sympathetic to the machines.  I think they've got a better shot at the future than we do. [laughs] So that's partly what the film's about.

So, Garland too appears to believe in "the machines" as the Next Evolutionary Step, which bespeaks a failure to understand either machines or evolution.  "If they have feelings and emotions ... or whatever" begs the question by assuming that a machine programmed to simulate feelings and emotions really has them.  I found when I discussed the Chinese Room problem with computer-science students back in the 80s and 90s that they too were excessively ready to ascribe intention and agency to computers.  One said that surely a computer programmed with the Chinese-language algorithms would begin to notice a correspondence between the Chinese texts and events in the world outside, and would develop an understanding of the language.  But even a human being -- John Searle, for example, or I myself -- couldn't do such a thing, since I would have no basis for spotting such a correspondence.  A computer could only "notice" what it had been programmed to do.  I pointed out this out, and he backed down, but he clearly wasn't convinced.

As far as I know, no one has any idea how to write software for consciousness.  (Ex Machina must postulate that Nathan has invented a completely new kind of computer for the task.)  Simulations don't do it.  What we have at this point is hand-waving: if you build a sufficiently complex machine that can simulate human behavior so that human beings (who are notoriously prone to anthropomorphize the inanimate) are fooled by it, then it has somehow become conscious and deserves its rights.  I think a crude, naive behaviorism underlies this belief.  Following my thought experiment about progressing from a machine programmed to say "ouch" when struck to a machine programmed and designed to roll on the floor bleeding when it's struck, at what point does consciousness emerge?  I don't think it does, and the burden of proof lies on the person who claims otherwise.

One of the characters in Robert A. Heinlein's 1966 science fiction novel The Moon Is a Harsh Mistress is Mike, a supercomputer that, having reached a certain level of complexity, magically becomes self-aware and conscious.  I say "magically" because Heinlein doesn't even try to explain it; it just happens -- in fiction.  In the real world it hasn't happened yet, though the Internet probably has as many nodes and circuits and processors and as much RAM as Mike had, and it probably won't happen, because that's not how consciousness works.  We can see a continuum from "simple" one-celled life to more more complex organisms to human beings, but machines aren't on that continuum.  What I find highly significant is how many self-styled rationalist scientists make the same leap of faith: if we build a big / fast / complex enough computer, it will Wake Up.  I'm willing to be agnostic on this matter, but so far I see no reason to suppose that it might.  Appeals to my human sympathies and intuitions aren't arguments; they beg the question that must be argued.

In the Terminator movies, the audience is allowed to look at the world from the killer cyborg's viewpoint: everything is bathed in red light, lines of code scroll up and down the screen.  The Terminator selects a response to a troublesome human from a menu: "Fuck you, asshole."  Har har har!  This visualization is based on the assumption that there is someone inside the cyborg, evaluating the input and deciding how to respond: a homunculus, in short, like a tiny kid working the controller of a video game.  I think that this is what most people who try to anthropmorphize computers and robots imagine, like the professor of computer science who wrote that "today's computer spends most of its time fighting back tears of boredom" -- but there is no one in there.

"A better shot at the future"?  I suppose that Ava's abandonment of Caleb -- who has sided with it and tried to rescue it from Nathan -- to die alone of hunger and thirst in Nathan's secret lab, is meant to dramatize Garland's "we don't feel particularly bad about Neanderthal man or Australopithecus, which we replaced."  Just so, Ava tosses Caleb into the evolutionary dustbin, where he belongs.  If that's supposed to make me feel "weirdly sympathetic to the machines," it fails.  There's something I'm not sure now whether Ex Machina explained: I can't help wondering how Ava is powered, or what will happen to it in the big wide world when its batteries run out.  Garland says "If a machine can't get ill, and is not really mortal," but machines -- especially computers and other electronics -- are really quite fragile, and unlike organisms they don't heal when they're damaged.  What if Ava is hit by a careless driver, and its false skin is broken open to reveal the works beneath?  (I think I smell a sequel coming.)

So what issues are raised by Artificial Intelligence?  As I wrote in my previous post, stories like Ex Machina aren't about them but about relations between human beings, with the AI standing in for whatever Other frustrates you, makes you anxious, insists that they're as human as you are despite your patient certainty that they aren't.  (Sometimes it stands in for the superior being before which so many people long to prostrate themselves, a being free from human weaknesses which will teach us The Way and lead us to the vague Better Place many of us dream about.)  Some of these anxieties arise from our own humanity, so they're still about us, such as our ability to manipulate our environment consciously.  Does that make us like gods?  No, because our conceptions of gods are based on our conceptions of ourselves.  The Creator is a craftsman like a human craftsman, or a parent like a human parent, and so on.  They're also about how we treat others, or Others, those who are human but whom we perceive as not being Like Us.  Not only are people too prone to ascribe personhood to inanimate things, we are too prone to fail to recognize the personhood of other people when their demands on us become too inconvenient.  These concerns are as old as humanity, and anyone who claims that computers and AI create "new" problems is blowing smoke.

Deliver Us from People

As I was driving through western Michigan last month, I came on a public radio station playing the TED Radio Hour, the result of collaboration between NPR and the notorious TED ("Technology, Entertainment, Design") nonprofit.  The topic of this episode was robots and their relationships with human beings. "Do we need humans?" the main page of the program asks rhetorically; to which the first response must be: "Who's we?" But apparently the people who made the program, who evidently are not human themselves (so what are they?) think the answer is no, "we" (nonhumans, that is) don't. Whether the entities that made the program thought of its listeners as pure disembodied Mind, uncontaminated by yucky flesh and emotions, looking for suitable company in a lonely universe; dolphins or cats; or members of a silicon-based extraterrestrial species listening to NPR by faster-than-light streaming from Alpha Centauri, they seemed to think that their audience was other than human.  Perhaps I shouldn't have been listening in.

I caught the tail end of the segment on Sherry Turkle ("the poster child for the tech revolution, literally") in which host-creature Guy Raz anxiously warned Turkle about "some of your colleagues at MIT who, you know, make you seem like an outlier."  I mean, you wouldn't want to be a weirdo, would you, Sherry?  Don't you want to be, like, a normal poster child for the tech revolution?  Raz was much more comfortable with his next guest, Cynthia Breazeal, who reminisced obligingly for him.  When she was a kid, she saw Star Wars and was delighted with the robots C3PO and R2D2:

I mean, they had emotions, and they were loyal sidekicks and they helped people ... And they were amazing in their - in their humanity almost, right? ... I was all about those robots.

RAZ: And so, those robots from Star Wars?

BREAZEAL: They definitely led me, ultimately, to MIT ...

RAZ: And that's where, for the last 20 years, Cynthia Breazeal's been building and studying social robots.

Okay, Breazeal can be forgiven for thinking, as a kid, that C3PO and R2D2 "had emotions and were loyal sidekicks and ... were amazing in their ... humanity almost."  But of course C3PO and R2D2 were not really robots.  They were characters in a story, played by human actors in costumes, and they were not evidence of what robots were like in 1977, or might be in a galaxy far away.  In this they were like Isaac Asimov's robots, who also were animated by magic ("positronic brains"), not by advanced technology: that's why they seemed so humanlike. They were not robots but allegorical stand-ins for human slaves, and they didn't even bear as much resemblance to "real" robots as African-American slave characters in Hollywood films like Gone with the Wind bore to real antebellum slaves.  They were meant to delight the audience, not represent reality.

Guy Raz was the ideal shill for those who exaggerate the achievements of actual technology.  He badgered Sherry Turkle for asking questions about what it means that technocrats see it as progress when a lonely old woman in a nursing home bonds with an animatronic doll called Paro.

RAZ: Isn't there a part of you that looked at that interaction and thought it was amazing? I mean, here was this woman who had suffered a profound loss, at some point in her life - she had lost a child - and she was responding to this robot. And it might not have been real, but it was real to her.

TURKLE: Well, no, I don't think it's amazing. And what was extraordinary, to me, is that although she was surrounded by people who were in a position to understand her story, we were kind of applauding and stepping back, and cheering on her connection to a machine that understood nothing. And I really felt - what are we doing? Why are we, essentially, outsourcing the thing that defines us as people? You know, we care for our young; we care for our old; and outsourcing it to something that we knew was essentially, deceiving her by tricks into thinking that it cared for her at all ...

RAZ: OK, Sherry, can I just pause here for a sec? ... What if someone is so lonely, and their loneliness can be resolved by having a robot or, you know, some device to talk to - I mean, shouldn't we let them have that?

I agree with Turkle: it wasn't amazing at all.  It's not much of an achievement to get people to bond with inanimate objects or pets or characters on TV or recorded singers or imaginary friends.  That's why I don't take the Turing Test very seriously.  The computer scientist Joseph Weizenbaum found that people who used his ELIZA program in the 1970s, which reflected people's typed words back at them, reacted to it as if they were having a real private conversation with someone.  Because it is so easy to get people to anthropomorphize the world, some computer geeks have tried to raise the bar, so that "interrogators are specifically trained to avoid" anthropomorphizing.  In that case, I suspect, many human beings couldn't pass the Turing test.  As I listened to the TED Radio Hour, I had my doubts whether Guy Raz was human or a software construct.  But no, he was way too easily fooled by the robots he met to be anything but human.

"We" aren't going to need people anymore, because robots are quickly being developed -- by whom, I don't know, certainly not by humans -- that will fulfill our needs.  But when the Kingdom of the Robot comes, will "we" be worthy of our new neural-net BFFs?  That question lurked everywhere in the program, but was never addressed, let alone answered.  Notice too that Raz assumed -- rather like the tech people at the scene -- that the woman's loneliness was "resolved" by talking to a robot, and that this represented some kind of "amazing" advance.  But there have been people whose loneliness or misery was assuaged somewhat by stuffed toys, or plastic dolls: high-tech electronics are not really necessary as a stopgap, but there's no reason to suppose they are any better a substitute for real human interaction than a Betsy-Wetsy.  Jesus once asked his audience rhetorically if, when a man's child needs bread, he'll give it a stone; for Guy Raz, the answer is evidently "Why not?"  But nowadays we can give a hungry child a full-color hologram of food, and shouldn't we let them have that rather than give them nothing?

The blurb on the web page continues: "We've been promised a future where robots will be our friends, and technology will make life's daily chores as easy as flipping a switch."  Really?  When were "we" -- presumably human this time, but who knows -- "promised a future where robots will be our friends"?  Who made that promise, and what drug were they on?  One cognitive scientist, Gregory J. E. Rawlins, speculated eagerly about electronic pets and "feral cars" in his 1997 tract Slaves of the Machine (MIT Press, 1997. 121): "One day our artificial creations might even grow so complex and apparently purposeful that some of us will care whether they live or die. When Timmy cries because his babysitter is broken, ... then they'll be truly alive."  Haven't children already cried when toys broke?

Listening to the TED Radio Hour right after I'd finished reading David F. Noble's Forces of Production helped me recognize what was going on: it was the technocratic mind at work, trying to get rid of messy, selfish, lazy, irrational human beings in favor of clean, ego-less, hard-working rational machines.  The advertising always runs ahead of reality, promising emancipation from human workers that the products can't deliver.  It doesn't matter whether those machines are able to do the work, let alone whether they are cheaper or more efficient -- they aren't -- what matters is control by those who fondly suppose themselves to be superior to the rabble, and who forget that they are human themselves.  They fantasize about communing with robots -- or at least, as NPR's Guy Raz did, about letting other people make do with robots when real people fail them.  Many fantasize about becoming cyborgs themselves.  This isn't a rational aspiration: it's just a newer version of the ancient religious wish to jettison the corruptible flesh and become pure, incorruptible spirit.  Like the second coming, the glorious day of that transformation keeps having to be postponed, but the true believer never gives up hope. 

Small Changes, Very Small

I'm almost done rereading Marge Piercy's third novel, Small Changes, originally published in 1973.  I first read it in the mid-1970s.  It's a long complex book, built around two main characters but with many other secondary ones.  (Two important male characters are Vietnam veterans, interestingly.)  It's typical now to dismiss it as one of the feminist classics of the 1970s, and many contemporary readers claim that it's dated.  I don't think so.  Relations between women and men have changed somewhat since the book was written, and middle-class women are more likely to have careers outside the home than they were previously.  But men still expect emotional and other service from women, and woman still find it difficult to refuse it, let alone demand emotional and other service from men in return.  And there's still reaction, a powerful movement to return us all to the Fifties, though the Fifties were an aberration, a diversion from tradition rather than Tradition itself.  We're not likely to go back to the days when many women could stay at home as housewives because most men can't earn enough to support a household by themselves.  It wasn't feminism that destroyed the "traditional" family, it was capitalism and the Reagan administration.

But although feminism is one of the core themes of Small Changes, it's about other things too.  It's partly an account of reaction against the Sixties, including state repression that most Americans paid no attention to at the time.  One of the characters must testify before a grand jury empaneled to destroy the resistant social movements that had scared our rulers so badly.  This was officially justified, insofar as anyone bothered, because of the violent segments of those movements: the Black Panthers, the Weathermen.  (Of course, the far greater State violence, against American citizens as well as foreigners, was of no great concern to most citizens, even after it was exposed.)  As one character explains:

It isn't like a regular jury.  They're older, richer white people always, and they even run a credit check on them.  They're to bring in indictments that the government wants.  Ninety-five per cent of the time that's what they do -- agree to prosecute.  But Anita says that mainly they're a device for investigating.  They're a way of forcing people to testify against each other.  The F.B.I. can't force you to talk, the police can't, though they try.  But a grand jury can send you to jail if you don't answer their questions [505].*

And Wanda, the character called before the grand jury, explains:

"This is a fishing expedition, Beth.  The questions they want to ask me aren't just about deserters.  They're asking about the whole web of connections in the movement.  They want to know who's friendly with who, who lives where, in what commune, who sleeps with who, who gives money.  They want to fill in all the missing connections so that people can't disappear underground any more."

"But they must know all that anyhow, they have all the phones in the country tapped!"

"Don't exaggerate, Beth.  They don't know everything.  Even when they 'know,' half of what they know is garbage and they want confirmation.  Look, the questions they're asking me include every address I've lived at since the year zero.  Who else lived there?  What vehicles have I owned and who borrowed them?  Who was at the women's conference last year in Boston and where did they stay?  How did I travel from Boston to New York for the health care conference that summer?  What meetings did I attend in that fall that plotted riots, demonstrations, or street actions -- all lumped as one! ... [507].

Yes, the novel is often didactic, but no more than any Robert Heinlein book.  No wonder that I, who grew up loving Heinlein, also love Piercy.  Heinlein's advocates have often pointed to his highly intelligent and competent characters as a feature of his work that sets it above most (especially mundane) fiction.  Piercy also focuses on intelligent competent characters, with the difference that she doesn't set them against straw opponents they can easily beat.

Things haven't changed all that much in American politics; we now have the Patriot Act and the surveillance state, and probably all our phones and e-mail are tapped now.

Another key topic is computers.  Miriam, one of the two principal characters, moves from mathematics to computer science.  This is in the late 1960s, when computers were mainframes that filled whole rooms, and programs and data were input on punched cards, before desktop computers much less tablets, before the Internet and WiFi.  Miriam gets a job with a small research firm on Route 128 near Boston, the only female programmer in the company.  (Come to think of it, Miriam is a lot like Heinlein's smart, sexy, male-identified heroines, though again, she lives in a different universe.  But then Heinlein's men are also fantasy figures.)  Their big contract is writing software for the anti-ballistic missile program that the Nixon administration backed at the time.  Miriam gets shuttled onto that team, which she hates.  There's a scene where she meets her old MIT professor Wilhelm Graben, who scoffs when she complains about being "hired by death."

"What nonsense.  It's scuttlebutt that the system will never operate.  For instance, assuming that the computer technology works out perfectly -- for the first time in history, perhaps -- but assuming that, the crucial interface is that combination of radar equipment and computers.  Now there cannot possibly in any real war situation be enough time to discriminate the raw data in the radar inputs about what would be real missiles and what simply extra junk floating about.  All weapons are programmed to fill the heavens with large quantities of objects that induce noise in the radar.  Chaff for instsance -- aluminum foil -- reflects radar beams quite nicely and produces responses that would indicate serious objects approaching.  The electronic countermeasures will be jamming, with strong transmission at the same frequency that the radar operates on ...

"But forget military questions.  Think about the most amusing aspect of the ABM.  Now you are exploding, say, a five-megaton warhead to knock down a missile no bigger than a barn.  Obviously, this does not require a warhead of five hundred million tons of TNT -- Hiroshima was destroyed by twenty thousand tons' equivalent.  Thus you can deduce that accuracy is simply not in it.  They're figuring on exploding a five-megaton bomb to knock down a missile because they are not counting on being in the same state with it -- states imagined to be lines superimposed upon the air ... "  His voice was calm and mocking.  She grew colder and colder ...

"Defense, it's called.  The rhetoric of defense, of course, is that human beings are being defended.  But the type of weaponry we are discussing is absolutely useless in defending human beings.  It would make little difference, I imagine, to someone on the ground whether he was fried alive because an enemy missile exploded twenty miles to his right hand, or because one of 'his' missiles exploded as far overhead.  The temperature on the ground would instantly rise several hundred degrees, in either case.  Defense is defense of missile sites, not of people or the landscape, which would be eliminated ... But you must admit, the rhetoric with which American politicians address their constituencies about defense spending is amusing" [379-380].

I'd forgotten this passage when, sometime in the early 90s, I leafed through the book looking for another passage I remembered.  The same weapons systems, with super space lasers added, were floated again in the 1980s as the Reagan gang's Strategic Defense Initiative or SDI, commonly derided as "Star Wars." There have occasional attempts to resurrect the program under Clinton and perhaps under Bush, even though the same criticisms Piercy put into Graben's mouth still apply.  What counts is pouring millions and billions of dollars into high-tech industry, not "defense ... of people or the landscape." 

There's a lot of quotable, still-relevant material in Small Changes -- I haven't come close to exhausting it here.  On one hand, while there's nothing wrong with books that focus on stereotypical concerns of women like domesticity and romance, concerns that still dominate what is derisively called "chick lit," it should be clear that Piercy's interests and ambitions include those concerns but aren't restricted to them.  Much as I love, for instance, Jennifer Crusie and her smart heroines, none of her books are as large in this sense as Small Changes or most of Marge Piercy's novels.  But then, the same is true of most male writers today.  I think that the reason many people try to dismiss Piercy's work as dated feminist cliches is that her left politics disturb them at least as much as her critical take on men in relation to women.

*Quoted from a Fawcett-Columbine trade paperback reprint from (I think) 1997.

One Word: Plastic; or, My, What a Big Hippocampus You Have!

I'm about eighty pages into a new book, The Blind Giant: Being Human in a Digital World by Nick Harkaway (John Murray, 2012).  So far it appears that Harkwaway is trying hard to be middle of the road: microchips and the Internet for him are neither utopian culture-changing forces nor inevitably dehumanizing nightmares, but something in between.  Fair enough.

The chapter I'm reading now is called "The Plastic Brain."  It took me a moment to realize that Harkaway knows the original meaning of "plastic," before it was applied to mostly organic polymers with a distinctive texture and feel: it originally meant "moldable," and connoted substances that could be shaped and would hold the shape they'd been given.  The thought passed through my mind that "plastic" might be a better adjective for human sexuality than the currently popular buzzword "fluid", except that I'm not sure most people would understand its being used that way.

Harkaway begins by pointing out that

the brain is a versatile and even to some extent a volatile organ.  It does, even in adulthood, alter its shape to take on new skills and abilities at the cost of others.  The phenomenon is called 'neuroplasticity', and it is actually -- to a layman's eye -- remarkable.  By way of example: the anterior hippocampus -- the region associated with spatial memory and navigation of a London taxi driver, seen in a magnetic resonance image, shows pronounced enlargement.  Taxi drivers learn the streets and the flow of traffic, and that learning is reflected in the actual physical structure of their brains [81].

Another example that I find suggestive is that bilingual people show up differently in brain scans depending on whether they acquired both languages before the age of four or five, or acquired the second one later in life.  In the former case, both languages show up localized in the same area of the brain; in the latter, they are spread out to different areas.  This sort of information is important, because I've run into people who've claimed that brain scans show different results in gay and straight people, which they assume to mean that the differences are at least inborn if not genetically determined; but such differences can be acquired as the result of learning.

Having said that, it is important not to overstate the extent of neuroplasticity.  Steven Pinker and Johnstone Professor of Psychology at Harvard, points out in The Blank Slate that 'most neuroscientists believe that these changes take place within a matrix of genetically organized structure.'  However impressive the plasticity of the brain, there are limits.  'People born with variations on the typical plan have variations in the ways their minds work ... These gross features are almost certainly not sculpted by information coming in from the senses, which implies that differences in intelligence, scientific genius, sexual orientation, and impulsive violence are not entirely learned.'  The question is how far the smaller changes within the brain can take one's identity before the brick wall of genetic structure is reached [82; emphasis added].

I can't help wondering what alternative there would be to the brain as a "genetically organized structure."  Like many biological determinists, Pinker here overlooks the interaction between the genes and the environment that produces actual organisms.  It's not a question of either / or.  Does anybody argue that the brain isn't "genetically organized," at some level?  At the same time, it's clear that "differences in intelligence, scientific genius [!], sexual orientation, and impulsive violence" are not entirely "organized" by the genes either.

Analogously, the plastic arts use materials that can be shaped and manipulated -- clay, metal, wood, stone, concrete, and so on -- but there are physical limits to their plasticity, and artists who work with these media have to learn to respect them.  There are people who talk, carelessly, as though organisms are infinitely malleable, and they're wrong.  Nothing material has unlimited plasticity.  But on the other side you have people like Richard Dawkins and his fantasy of organisms as "gigantic lumbering robots" in which hives of genes swarm.  Dawkins tried to minimize this passage here, behind a flurry of rhetoric without a serious argument, especially since robots and computers were much less "intelligent" in the Seventies (when he wrote that passage) than he believes they are now.  The real problem, though, is that he posits a sharp divide between the genes and the organisms they inhabit.

Ironically, it seems to be enthusiasts for genetic manipulation who believe that organisms are, or will be, infinitely malleable if we can just learn enough about the genome, and science enthusiasts generally who talk as though there were no limits to the plasticity of the human brain when it comes to the acquisition of scientific knowledge.  The possibility that there might be some questions that can't be answered, not because there are things Man ought not to know, but because thought has limits imposed by the physical structure of the brain, tends to make them uncomfortable.  I have no idea whether there are such limits, or what they might be if they exist, but I think it's a reasonable question.

We'll Have A.I. in the Sky When We Die, Part 2

(continued from yesterday)

I'M SORRY, DAVE ... I CAN'T DO THAT

Behaviorism is a 20th-century scientific fad whose biggest self-promoter, the late B. F. Skinner, made a career of playing a caricature of the 19th-century materialist crank who declares dogmatically that if it can't be measured, weighed, etc., it's just superstitious nonsense. Behavioral psychology lives on, exploiting the principle beloved of evangelists, generals, and scientists everywhere: if we haven't delivered the Second Coming / total victory / a Grand Unified Theory, it's only because you haven't given us enough money and personnel! Dig till it hurts! Send in your contribution NOW!

Like some other Artificial Intelligence researchers, Rawlins subscribes to the behaviorist principle that "Ultimately it's behavior that matters, not form" (17). If it quacks like a duck, it is a duck, no matter how many superstitious, animistic mystics claim that it's a CD player.

But AI apologists have made one vital modification to the principle: where behaviorists refused to anthropomorphize anything, especially human beings, some AI folk are ready to recognize the full humanity of computers, yesterday if not sooner. Slaves of the Machine churns up a weird goulash of behaviorist rhetoric spiced with echoes from Abolitionists, the Civil Rights movement, even the anti-abortion movement. (Hence the pun on "quickening" in the book's subtitle: If a computer scientist was gestating a computer Beethoven, would you abort his research grant?) "Most computer hardware today isn't working all the time anyway; most computers are asleep most of the time, snoring away while they wait for us to give them something to do" (81). "So today's computer spends most of its time fighting back tears of boredom" (29). Dig deep into your pockets, friends, and give, so today's computer will never have to cry again.

"So what will our future be? Wonderful and terrifying," Rawlins crows (20). "Are we ready for a world of feral cars?... Some of these future machines will be so complex that they'll eventually become more like cats than cars.... Perhaps all of them will be insane to some extent. Perhaps we are too. Perhaps when your future toaster breaks down and refuses to toast your bread because it's having a bad day, you won't call an engineer or a mechanic, you'll call a therapist" (121).

"One day our artificial creations might even grow so complex and apparently purposeful that some of us will care whether they live or die. When Timmy cries because his babysitter is broken, ... then they'll be truly alive" (121)." Like the Velveteen Rabbit? Or Pinocchio? Rawlins knows, and says, that human beings are prone to personify inanimate objects. Children already become attached to inanimate objects like stuffed toys or security blankets; adults are not much better.

"As the children who accept them as alive grow up and gain power in the world, they'll pass laws to protect the new entities." And he sees this as progress? More like a regression into animistic superstition. But it's also not very likely. People don't pass laws to protect the civil rights of the security blankets or stuffed toys, or imaginary playmates, they loved as children either.

I'm sure there are people who would buy a robot "pet," a thing coated in washable plastic fur that eats electricity, doesn't need a litter box, never goes into heat, and can be neatly switched off when you go on vacation. It would alleviate another common problem: the cute little puppies that are abandoned when they grow up and aren't cute anymore. Rawlins would like us to have robots that are too lifelike to be dumped in the landfill, but why bother? We already have real pets.

Rawlins likes to think that robot "pets" and robot "people" will present new ethical problems. This is typical Jetsons talk. The ethical problems involved are quite old, and we've mostly preferred not to think them through. Since we've never done very well with them before, I see no reason to suppose that we'll do better when computers are involved. Other animal species and other human "races" have raised the very same ethical problems.

The easiest way not to deal with those problems has been displacement: the faith that on the other side of the next mountain there lives a "race" of natural slaves who are waiting to serve us, their natural masters. They will welcome our lash, set our boots gratefully on their necks, and interpose their bodies between us and danger, knowing that their lives are worth less than ours. Since there is no such natural slave race, the obvious solution is to build one. The question is not whether we can, as scientistic fundamentalists like to think, but whether we should. Rawlins is a bit smarter: he not only recognizes, he stresses that true machine intelligence would not be controllable. But he confidently declares that it's inevitable, which he needs to prove, not assert.

In fact he's so obviously wrong that I wonder if he's using the word in some arcane metaphorical sense. "Inevitable" suggests a downhill slide that requires all your strength to retard, let alone stop. On Rawlins's own showing, such machines will be built only if governments and/or businesses pour vast amounts of money and intellectual labor into reinventing the art of computer programming from the ground up, and probably also reinventing computer hardware from the ground up, in order to produce a totally new kind of machine that no one really wants anyhow: a "feral" computer that is unpredictable, uncontrollable, and "insane to some extent." I don't call that inevitable. It's an uphill climb if it's anything at all.

Like many scientistic evangelists, Rawlins accuses anyone who doesn't want to climb that hill of superstitious pride: "Some of us may resist the idea of ourselves as complex machines -- or of complex machines one day becoming as smart as we are -- for the same reason that we resisted the idea that the earth revolves around the sun or that chimpanzees are our genetic cousins" (120).

Most people who talk about this kind of sinful pride forget that "Man" was never at the top of the Great Chain of Being: that spot was reserved for the gods and other supernatural beings, like angels. Humility in our human status has always been an official value in the West, even if it has been honored more in the breach than the observance. In practice its main function has been to rationalize and reinforce human hierarchies (obey the King, as he obeys God). This alone should give us pause before we try to push machines to a rung above our own. James Cameron seems not to have considered in his Terminator movies that there would be human Jeremiahs urging their fellows to worship Skynet, the intelligent computer network that was trying to wipe out the human race, or at least to submit to it as the instrument of god's wrath, much as the original Jeremiah urged Israel to surrender to Babylon.

A kinder (if two-edged) analogy is the biblical tower of Babel. Yahweh smote its builders with confusion of languages, not because he was annoyed by their pride, but because he really feared they would build a tower that could reach Heaven. Just as there were still naifs in the Fifties and Sixties who opposed the space program because they thought the astronauts would crash-land at the Pearly Gates and injure St. Peter, there are probably those who oppose AI because they think these are mysteries in which "Man" should not meddle. But it's more accurate -- and despite its triumphalist rhetoric, Slaves of the Machine bears me out -- to point out that the Tower of AI will never reach Heaven. Inventing smaller faster microprocessors with extra gigabytes of memory won't produce machine intelligence -- not even if perfectly coordinated teams of programmers manage to write flawless million-million-line programs. Rawlins and his colleagues are not clear-eyed rationalists -- they're the builders of Babel who argued that with just a few more research grants, a few thousand more slaves, Babylonian newlyweds could honeymoon in Paradise.

GARBAGE IN, GOSPEL OUT

On Rawlins's own showing we're reaching the limits of computers as we know them, because programming for every contingency is impossible in any but the simplest tasks. He says we could program current hardware to be more flexible, if only programmers would stop living in the past, but he gives no reason to think so and numerous reasons not to think so. It seems more likely to me that we'd need new kinds of hardware too, kinds that aren't based on on-off logic switches – and that hardware doesn't exist yet. Maybe someday it will, but Rawlins is already bowing down before it as if it did.

The most likely scenario Rawlins presents is that technogeeks will get funding for new hardware developments the way they always have: by promising governments, business, and the military faster, more compact, more flexible, but still absolutely docile electronic slaves. Instead they would produce (in the unlikely event they succeeded) machines with minds of their own. It isn't inevitable that they will get the money they demand, though. There's no reason why the public must pay the AI cult to build whatever its members want while promising something else. We can say No.

And on that somber note, our time is just about up. Let's take one last look at our computer flatworm experiment.... No, it appears that the sundered parts of our BFD supercomputer have not regenerated themselves. BFD promises us, however, that this problem will be corrected in the next version of the operating system, which will be released very soon. So thanks for tuning in, and join us next week for the next installment of "This Old Computer"!

We'll Have A.I. in the Sky When We Die, Part 1

(I wrote this back in the 90s, and while Artificial Intelligence no longer seems as trendy as it was then, I think a lot of the issues I talk about here are still very much with us.)

In the 1990s, or so we were assured when I was a lad, everyone would have his own personal helicopter, powered by its own clean, safe, and inexhaustible nuclear power pack. (Automobiles would be obsolete!) The skies would be full of commuters, all of them white males as all Americans were presumed to be in the 1950s, returning to the suburbs from their 20-hour a week jobs. (The work week would shrink, giving us all more leisure for home barbecues cooked on clean, safe and inexhaustible nuclear-powered backyard grills!) Their wives would preside languidly over homes staffed by robot butlers and maids, each powered by a clean, safe, and inexhaustible nuclear power pack, whose excess heat could be used to distill the alcoholic beverages needed by 1950s housewives to dull the boredom and isolation of their suburban days.

Well, you get the picture: it's the Jetsons. But the Jetsons affectionately parodied visions of the near future that were discussed seriously in other media. Nowadays those failed predictions are a bit embarrassing: nuclear power has lost its glamour, few people like the idea of filling our skies with individual commuter aircraft, and economists lecture us that the relative affluence of the 1950s was a historical fluke, that we'll be lucky if our standard of living doesn't decrease. As I read computer scientist Gregory J. E. Rawlins's book Slaves of the Machine: The Quickening of Computer Technology (MIT Press, 1997) I often had the feeling that I had been transported into the past, the past of the 1939 World's Fair, the past of the Jetsons.

Rawlins plays the prophet with such gee-whiz gusto, in fact, that I'm still not sure he isn't kidding. The future is here, by golly, and you ain't seen nothin' yet! “From ovens to spaceships, from mousetraps to nuclear power stations, we're surrounded by millions of automatic devices each containing small gobs of congealed wisdom” (2). Today's portable computer is "half the size of Gutenberg's first Bible - and perhaps as important" (21). Computers "twenty years from now will be vastly different from today's primitive mechanisms, just as a tiny computer chip is different from a barnful of Tinkertoys [5]... and in 20 years they'll be as cheap as paper clips" (28). "Because in the far future - which in the computer world means one or two decades - we may make computers from light beams. Sounds like science fiction, doesn't it? Well, in 1990 AT&T Bell Laboratories built one" (31). "In forty years, something bee-sized may carry more memory and computing power than all today's computers put together" (34). "But one day, what we call a computer may be something grown in a vat that will clamber out clothed in flesh" (33). "And, for the same money, by 2045 you could have 8500 million 1997 dollars worth of machine – four times the power of the entire world's supply of supercomputers in 1997 - all in your designer sunglasses" (34).

I suppose there's no harm in this sort of thing, but is it worth $25 for 135 clothbound pages? You can find equally solid predictions for a lot less in the checkout lane at your supermarket. The dust jacket copy of Slaves of the Machine touts it as an introduction to the wonderful world of computers, written in simple, colorful language that any cybernetically-challenged doofus can understand. I doubt that Slaves will reach its intended audience, though, not just because of its cost and lack of graphics, but because of its patronizing tone. As the examples I've quoted above indicate, Rawlins is more interested in dazzling than in explaining. Slaves of the Machine is more like an evangelical tract, full of promises and threats, than science writing. (But I may be drawing a nonexistent distinction there.)

One of Rawlins's pronouncements, though, caught my fancy: "Today, our fastest most complex computer, armed with our most sophisticated software, is about as complex as a flatworm" (19). In that spirit, we have in our studio a state-of-the-art supercomputer, courtesy of BFD Technologies, which I shall proceed to split down the middle with this fire axe. In the course of our program today we'll return to see if this complex marvel of human ingenuity can, like a flatworm, regenerate itself into two complete supercomputers, each with the memory and software of the original computer already installed! Meanwhile, we'll take a closer look at Prof. Rawlins's vision of the future of computing.

THAT'S NOT A BUG, THAT'S A CREATURE, I MEAN FEATURE!

Having begun by promising us unlimited horizons of computer power, Rawlins proceeds to explain How We Talk to Computers. Like a guide warning Sahib about the innate inferiority of the natives he's about to encounter, Rawlins warns the reader, "It's no use appealing to the computer's native intelligence. It has none" (42). (Remember that line.) I use the guide / Bwana / native/ image because Rawlins himself uses it, not just in his guiding metaphor of computer languages as "pidgins" but in his anecdotes.

Using a computer, he says, is like being driven around France in the 1940s by a surly driver named Pierre, who "obviously thought he was competing in the Grand Prix" and needed precise instructions, which were futile because "he was particularly dense. He understood only two words -- yes and no; or rather, since he refused to understand anything but French, oui and non" (44). But then, "around 1955, ... we got Jacqueline, a slightly more advanced translator. Of course, Jacqueline didn't speak English either, but she knew Pierre really well" (44f). Oo la la! Lucky Pierre! Gradually it became clear to me that Rawlins meant this conte as an allegory of the development of computer "languages," but even though I already knew this stuff, I found it hopelessly confusing. I can't imagine what a real computer virgin would make of it.

Rawlins chides us for wondering, "How could a mere machine understand a language?" Since he has just told us that French computers, at least, have no native intelligence, this seems a reasonable question to me, but Rawlins breezily assures us: "Still, without getting into philosophical subtleties, it's okay to say that a computer can understand lots of simple things, if we first explain them carefully enough using a small number of well-chosen commands" (45). With that, he's off with a basic course on computer programming: the IF-bone is connected to the THEN-bone, the THEN-bone’s connected to the NEXT-bone, and so on. And I hope you were paying attention, class, because Rawlins moves quickly on to what really interests him: the cult of the Silicon God, and the care and feeding of its priesthood.

In ancient times (about thirty to forty years ago), "computers were costly, cranky, and above all, customized" (48). "In those days, even the vibration of a passing car or plane could crash the machine" (51). "No one could dispute the technical elite because no one -- often including the elite themselves -- understood the machines. What they did, how they did it, what they could do, these were all mysteries to almost everyone, including the biggest companies and the biggest names in computing today" (49). As late as 1972, when I took a FORTRAN class at IU, The Computer sat in its Holy of Holies, its sacred meals of punched cards fed to it by graduate students in Computer Science, and spewed forth the printouts of its oracles (which in my case, consisted mostly of error messages).

Remember that even in those days, computers were advertised as "electronic brains." Not knowing what they did or could do didn't keep their advocates from making grandiose claims about them, which ought to make us suspicious about the no less grandiose promises Rawlins was making about the future of computing just a few pages back.

But ah, then "came the pidgin designers", who "made it possible to talk to the machine in a sensible way." But wait! "Today's computer pidgins are hard to learn and hard to use. All are cryptic, precise, and dense" (53). Contrasting crocheting instructions with computer languages, Rawlins concedes that "even the rankest beginner can learn to crochet a doily or scarf in a few hours, whereas it often takes computer beginners days or weeks before they can program anything interesting" (54).

Now, a pidgin can be used by ordinary people, by traders and the "natives" with whom they trade. Computer languages vary in their difficulty, but for the most part they still require an interpreter - the programmer - between the user and the "native." A closer analogue to a real "pidgin" is the mouse-and-icons interface of the Amiga, the Mac, and Windows, which enables people without extensive technical training to "tell" their computers what to "do."

But there's a more basic problem: Computers are not trying to communicate with us, any more than a thermometer is. Despite the common use of anthropomorphic language such as "think" and "feel" by their devotees, they do not think, or feel, or see, or communicate. Perhaps someday they will, but for now and the foreseeable future they don't, and any computer enthusiast who pretends otherwise is either lying or fooling himself. Computer "language" is also a metaphor, not a literal fact. In a very loose sense, it could be said that you "communicate" with a light bulb when you turn on its switch, and that is the only sense in which computer "languages" enable us to "communicate" with computers. Rawlins's extended metaphors are entertaining, but they are also misleading. I suspect that when he says that "computers" think, he is using "computers" metaphorically too: to mean, not today's computers, but imaginary computers which might someday clamber out of hydroponic vats but for now are only a gleam in Gregory J. E. Rawlins's eye.

Rawlins blames the impenetrability of computer "pidgins" on programmers themselves, "slow-witted, tiny-brained bunglers" (74) who are "lost in the past" (81), "like lawyers who are concerned only with law, not justice" (80). "While hardware keeps leapfrogging ahead every eighteen months, software is still lost in the dark ages of the 1960s. ...What we lack today isn't more hardware, but more imagination" (81). "[W]e don't let [computers] revise their procedure in the light of new evidence or insight. We never let anything new occur to them, and we deny them access to the history of earlier attempts" (97).

All this may be true, but it's still unfair to the programmers, who must write instructions for machinery that may be twice as fast and cost half as much, but still has no native intelligence. Programmers don't "deny them access to the history of earlier attempts"; computers aren't seeking such access. "The change to adaptive computer systems will come, as often these things do, by the actions of a few forward-looking software companies with the courage to take a risk and produce a new kind of software -- adaptive software" (81). In other words, Rawlins has no idea how to do it either, but he will accept funding from "forward-looking software companies."

What surprises me is that at the same time he's impugning the intelligence and integrity of programmers, Rawlins anticipates every objection I could think of. "Of course, there are financial and physical limits on how much we can improve today's computers" (35). "Year by year we're rapidly gaining power and, just as rapidly, losing control" (37). Faster, more powerful machines still must be programmed by human beings. "The more people we put on a software project, the longer it takes .... At current rates of progress they'll never produce flawless thousand-million-line programs" (69, 70).

On a deeper level, "We simply can't track all the myriad details involved in solving a very complex problem." This is not because we're tiny-brained bunglers, but because "no one knows how to design a sequence of steps that represents what we would call thinking" (105).

"The trouble is that just because something is hard for us it isn't necessarily hard for computers, and vice versa. We invented them to do something we're bad at: they can do arithmetic with unparalleled speed, reliability, and accuracy. So it's only fair that they're bad at things we're good at. The surprise is that they're bad at such apparently easy things" (111). Rawlins has evidently read, and taken to heart, such serious critics of Artificial Intelligence research as Joseph Weizenbaum, Noam Chomsky, John Searle, and the arch-heretic Dreyfus brothers, though he prudently names none of them, lest he be pronounced anathema by Douglas Hofstadter and Daniel Dennett.

Still, the fault lies not in our stars, Horatio, but in our wicked refusal to recognize that a computer's a man for a' that. But let's stop here for a moment to see how our split computer is coming along... hmmmm. No regeneration so far. Well, we mustn't be impatient. As Thomas Edison said, science is 1% inspiration and 99% perspiration. We'll be right back with more about Slaves of the Machine, right after this message from Microsoft.