Peter Watts Is an Angry Sentient Tumor

by Peter Watts

  The more of us who refuse to play along—the more of us who cheat by feeding false data into the system—the less we have to fear from code that would read our minds. And if most people can’t be bothered—if all that clickbait, all those emojis and upward-pointing thumbs are just too much of a temptation—well, we do get the government we deserve. Just don’t complain when, after wading naked through the alligator pool, something bites your legs off.

  I’m going to let Berit Anderson play me offstage:

  “Imagine that in 2020 you found out that your favorite politics page or group on Facebook didn’t actually have any other human members, but was filled with dozens or hundreds of bots that made you feel at home and your opinions validated? Is it possible that you might never find out?”

  I think she intends this as a warning, a dire If This Goes On portent. But what Anderson describes is the textbook definition of a Turing Test, passed with flying colors. She sees an internet filled with zombies: I see the birth of True AI.

  Of course, there are two ways to pass a Turing Test. The obvious route is to design a smarter machine, one that can pass for human. But as anyone who’s spent any time on a social platform knows, people can be as stupid, as repetitive, and as vacuous as any bot. So the other path is to simply make people dumber, so they can be more easily fooled by machines.

  I’m increasingly of the opinion that the second approach might be easier.

  1 https://medium.com/join-scout/the-rise-of-the-weaponized-ai-propaganda-machine-86dac61668b#.f0a4mf5p5

  2 https://www.scientificamerican.com/article/will-democracy-survive-big-data-and-artificial-intelligence/

  3 https://www.theguardian.com/politics/2017/feb/26/robert-mercer-breitbart-war-on-media-steve-bannon-donald-trump-nigel-farage

  4 http://www.eng.ox.ac.uk/about/news/new-study-shows-nearly-half-of-us-jobs-at-risk-of-computerisation

  5 https://www.scientificamerican.com/article/will-democracy-survive-big-data-and-artificial-intelligence/

  Life in the FAST Lane

  Nowa Fantastyka Apr 2015

  Back in 2007 I wrote a story about a guy standing in line at an airport. Not much actually happened; he just shuffled along with everyone else, reflecting on the security check awaiting him (and his fellow passengers) prior to boarding. Eventually he reached the head of the queue, passed through the scanner, and continued on his way. That was pretty much it.

  Except the scanner wasn’t an X-ray or a metal detector: it was a mind-reader that detected nefarious intent. The protagonist was a latent pedophile whose urges showed up bright and clear on the machine, even though he had never acted on them. “The Eyes of God” asks whether you are better defined by the acts you commit or those you merely contemplate; it explores the obvious privacy issues of a society in which the state can read minds. The technology it describes is inspired by a real patent filed by Sony a few years ago; even so, I thought we’d have at least couple more decades to come to grips with such questions.

  I certainly didn’t think they’d be developing a similar system by 2015.

  Yet here we are: a technology which, while not yet ready for prime time, is sufficiently far along for the American University Law Review to publish a paper1 exploring its legal implications. FAST (Future Attribute Screening Technology) is a system “currently designed for deployment at airports” which “can read minds . . . employ[ing] a variety of sensor suites to scan a person’s vital signs, and based on those readings, to determine whether the scanned person has ‘malintent’—the intent to commit a crime.”

  The envisioned system doesn’t actually read minds so much as make inferences about them, based on physiological and behavioral cues. It reads heart rate and skin temperature, tracks breathing and eye motion and changes in your voice. If you’re a woman, it sniffs out where you are in your ovulation cycle. It sees your unborn child and your heart condition—and once it’s looked through you along a hundred axes, it decides whether you have a guilty mind. If it think you do, you end up in the little white room for enhanced interrogation.

  Of course, feelings of guilt don’t necessarily mean you plan on committing a terrorist act. Maybe you’re only cheating on your spouse; maybe you feel bad about stealing a box of paper clips from work. Maybe you’re not feeling guilty at all; maybe you’re just idly fantasizing about breaking the fucking kneecaps of those arrogant Customs bastards who get off on making everyone’s life miserable. Maybe you just have a touch of Asperger’s, or are a bit breathless from running to catch your flight—but all FAST sees is elevated breathing and a suspicious refusal to make eye contact.

  Guilty minds, angry minds, fantasizing minds: the body betrays them all in similar ways, and once that flag goes up you’re a Person of Interest. Most of the AULR article explores the Constitutional ramifications of this technology in the US, scenarios in which FAST would pass legal muster and those in which it would violate the 4th Amendment—and while that’s what you’d expect in a legal commentary, I find such concerns almost irrelevant. If our rulers want to deploy the tech, they will. If deployment would be illegal they’ll either change the law or break it, whichever’s most convenient. The question is not whether the technology will be deployed. The question is how badly it will fuck us up once it has been.

  Let’s talk about failure rates.

  If someone tells you that a test with a 99% accuracy rate has flagged someone as a terrorist, what are the odds that the test is wrong? You might say 1%; after all, the system’s 99% accurate, right? The problem is, probabilities compound with sample size—so in an airport like San Francisco’s (which handles 45 million people a year), a 99% accuracy rate means that over 1,200 people will be flagged as potential terrorists every day, even if no actual terrorists pass through the facility. It means that even if a different terrorist actually does try to sneak through that one airport every day, the odds of someone being innocent even though they’ve been flagged are—wait for it—over 99%.

  The latest numbers we have on FAST’s accuracy gave it a score of 78-80%, and those (unverified) estimates came from the same guys who were actually building the system—a system, need I remind you, designed to collect intimate and comprehensive physiological data from millions of people on a daily basis.

  The good news is, the most egregious abuses might be limited to people crossing into the US. In my experience, border guards in every one of the twenty-odd countries I’ve visited are much nicer than they are in ’Murrica, and this isn’t just my own irascible bias: according to an independent survey commissioned by the travel industry on border-crossing experiences, US border guards are the world’s biggest assholes by a 2 to 1 margin.

  Which is why I wonder if, in North America at least, FAST might actually be a good thing—or at least, a better thing than what’s currently in place. FAST may be imperfect, but presumably it’s not explicitly programmed to flag you just because you have dark skin. It won’t decide to shit on you because it’s in a bad mood, or because it thinks you look like a liberal. It may be paranoid and it may be mostly wrong, but at least it’ll be paranoid and wrong about everyone equally.

  Certainly FAST might still embody a kind of emergent prejudice. Poor people might be especially nervous about flying simply because they don’t do it very often, for example; FAST might tag their sweaty palms as suspicious, while allowing the rich sociopaths to sail through unmolested into Business Class. Voila: instant class discrimination. If it incorporates face recognition, it may well manifest the All Blacks Look Alike To Me bias notorious in such tech. But such artifacts can be weeded out, if you’re willing to put in the effort. (Stop training your face-recognition tech on pictures from your pasty-white Silicon Valley high school yearbook, for starters.) I suspect the effort required would be significantly less than that required to purge a human of the same bigotry.

  Indeed, given the prejudice and stupidity on such prominent display from so many so-called authority figures, outsourcing at least some of their decisions
seems like a no-brainer. Don’t let them choose who to pick on, let the machine make that call; it may be inaccurate, but at least it’s unbiased.

  Given how bad things already are over here, maybe even something as imperfect as FAST would be a step in the right direction.

  1 Rogers, C.A. 2014. “A Slow March Towards Thought Crime: How The Department Of Homeland Security’s Fast Program Violates The Fourth Amendment.” American University Law Review 64:337-384.

  Smashing the Lid Off Pandora’s Box

  Nowa Fantastyka Nov 2018

  I’ve been in a funk since the International Panel on Climate Change released their latest report last October. I’m in a funk as I type these words; chances are I’ll still be in a funk when you read them a month down the road. If you happened to read my blog post of October 261, you’ll know why. If you didn’t, the short version is:

  • There isn’t a hope in hell that we’ll meet the goal set out in the Paris Accords, to limit global warming to 2°C;

  • Even if we did meet that goal, the result would be apocalyptic;

  • The results would be merely disastrous if we managed to keep the increase down to 1.5°C—to give just one example, we’d only lose 70-90% of the world’s corals instead of all of them—but the only way to do that would be to—among other things—go carbon-free within thirty years, cut our meat consumption by 90%, and invent new unicorn technologies to suck gigatonnes of carbon back out of the atmosphere.

  Global disaster is now our best-case scenario, and the chances we’ll even clear that low bar are remote. Meanwhile, people have already shrugged and gone back to posting cat videos on Facebook.

  Think of this month’s column as a kind of coda to October’s blog post. Today’s take-home message is: that post was way too light-hearted. It took the IPCC at its word: that we had twelve years to get started, and if we did there was at least a chance to save something from the fire.

  The reality, it seems, is that there may not be any chance at all.

  The first thing to keep in mind is that IPCC reports are scientific, and science is innately conservative. If a result is only 90% certain, science rejects it. The usual threshold for statistical significance is 95%, often 99%; anything less is chalked up to random chance. Meaning that—especially in complex, noisy systems like a planetary ecosphere—real effects get lost in noise and statistical rigor. Your living room could be in flames and nothing might show up in the stats; only when your bed catches fire do the results become “significant”.

  The second thing to keep in mind is that even among—especially among—those who accept the dangers posed by climate change, there exists an almost pathological compulsion to remain upbeat no matter what. A good example is the reaction to an article by David Wallace-Wells—“The Uninhabitable Earth”—that appeared in New York Magazine back in 2017. It was an article that pulled no punches, that laid out the future in store for us without regard to the conservative blinkers of the 95% threshold—and scientists and activists alike decried it as gloomy and counterproductive.

  October’s IPCC report has largely vindicated Wallace-Wells, but the Pathology of Hope remains. We still want to save the world, after all, and too much despair just paralyzes people: if you don’t offer hope, you’ll never inspire folks to change. If you don’t sugarcoat things a bit, people won’t have a reason to try and make things better.

  You don’t tell people they’re doomed. Even if they are.

  Which brings us to Jem Bendell, of the University of Cumbria. Bendell’s PhD is in International Policy, not climate science, but he can read the writing on the wall as well as anyone (and our predicament’s rooted in politics after all, not science). He notes that we could cut our CO2 emissions by 25% and those savings would be wiped out by the heating that’s already resulted from the ongoing loss of Arctic ice (and consequent lowered albedo). He points out that IPCC forecasts have always proven too optimistic, that observed trends always seem to end up being worse than the worst-case scenarios predicted just a few years earlier. He notes that many IPCC projections assume linear increases, while the observed data are more consistent with nonlinear—possibly exponential—ones.

  Bendell doesn’t think we have twelve years to start, doesn’t give us until mid-century to zero out our emissions. He says widespread societal collapse is inevitable, and it’s going to start in just ten years. (Interestingly, this is about the same time that a variety of pathogens—following warmer temperatures into new environments—are expected to kick off a series of pandemics that hollow out the world’s major cities, according to parasitologist and evolutionary biologist Daniel Brooks.) Bendell says it’s time to give up on futile hopes of saving society as we know it. He coins the term “Deep Adaptation” to describe the processes by which we might deal with the collapse of modern society: ways to prioritize the things we might save, accept the loss of everything else, salvage what we can and hope to build something more sustainable from the wreckage.

  The first step, he says, is to grieve for the things we’ve lost. I myself would put it less charitably: for the things we’ve destroyed, I would say.

  Bendell wrote up his analysis in a paper titled “Deep Adaptation: A Map for Navigating Climate Tragedy.” He couldn’t get it published. Or rather, the journal he sent it to would only publish it if he rewrote the text to make it less “disheartening”.2

  Of course, it’s easy to discount one voice. The problem is there might well be a whole chorus backing him up, a chorus we haven’t been allowed to hear thanks to well-intentioned censors who insist facts pass some kind of Hope Test before they’re allowed out in public. The house burns down around us. The fire department was shut down during the latest round of Austerity Cuts. Doesn’t matter. Can’t let people lose hope.

  I have hope, though a distant one. The Earth has experienced mass extinctions before. Five times past the planet has lost 70-90% of its species, and it has always sprung back. A few weedy, impoverished survivors have always been enough to pick up the baton, speciate and bloom into brand-new ecospheres full of wonder and biodiversity. It may take ten or twenty million years, but it happens eventually. It will happen this time too.

  My hope is that nothing like us will be around next time, to fuck it all up again.

  1 Not available in this volume, unfortunately. But you can find it at https://www.rifters.com/crawl/?p=8433.

  2 You can find it on Bendell’s website, though: go to https://jembendell.wordpress.com/2018/07/26/the-study-on-collapse-they-thought-you-should-not-read-yet/

  The Split-brain Universe

  Nowa Fantastyka Aug 2018, extended Sept 12 2018

  The year is 1982. I read Isaac Asimov’s newly-published Foundation’s Edge with a sinking heart. Here is the one of Hard-SF’s Holy Trinity writing—with a straight face, as far as I can tell—about the “consciousness” of rocks and trees and doors, for chrissakes. Isaac, what happened? I wonder. Conscious rocks? Are you going senile?

  No, as it turned out. Asimov had simply discovered physical panpsychism: a school of thought which holds that everything—rocks, trees, electrons, even Donald Trump—is conscious to some degree. The panpsychics regard consciousness as an intrinsic property of matter, like mass and charge and spin. It’s an ancient belief—its roots go all the way back to ancient Greece—but it has recently found new life among consciousness researchers. Asimov was simply ahead of his time.

  I’ve always regarded panpsychism as an audacious cop-out. Hanging a sign that says “intrinsic” on one of Nature’s biggest mysteries doesn’t solve anything; it merely sweeps it under the rug. Turns out, though, that I’d never really met audacious before. Not until I read “The Universe in Consciousness” by Bernardo Kastrup, in the Journal of Consciousness Studies1.

  Kastrup goes panpsychism one better. He’s not saying that all matter is conscious. He’s saying that all matter is consciousness—that consciousness is all there is, and matter is just one of its manifestations. “Nothing exists outsid
e or independent of cosmic consciousness,” he writes. “The perceivable cosmos is in consciousness, as opposed to being conscious.” Oh, and he also says the whole universe suffers from Multiple Personality Disorder.

  It reads like some kind of flaky New Age metaphor. He means it literally, though.

  He calls it science.

  Even on a purely local level, there are reasons to be skeptical of MPS2 (or DID, as it’s known today: Dissociative Identity Disorder). DID diagnoses tend to spike in the wake of new movies or books about multiple personalities, for example. Many cases don’t show themselves until after the subject has spent time in therapy—generally for some other issue entirely—only to have the alters emerge following nudges and leading questions from therapists whose critical and methodological credentials might not be so rigorous as one would like. And there is the—shall we say questionable—nature of certain alternate personalities themselves. One case in the literature reported an alter that identified as a German Shepherd. Another identified—don’t ask me how—as a lobster. (I know what you’re thinking, but this was years before the ascension of Jordan Peterson in the public consciousness.)

  When you put this all together with the fact that even normal conscious processes seem to act like a kind of noisy parliament—that we all, to some extent, “talk to ourselves,” all have different facets to our personalities—it’s not unreasonable to wonder if the whole thing didn’t boil down to a bunch of overactive imaginations, being coached by people who really should have known better. Psychic CosPlaying, if you will. This interpretation is popular enough to have its own formal title: the Sociocognitive Model.

  There could be a sort of psychiatric Sturgeon’s Law at play here, though; the fact that 90% of such studies are crap doesn’t necessarily mean that all of them are. Brain scans of “possessed” DID bodies show distinctly different profiles than those of professional actors trained to merely behave as though they were: the parts of the brain that lit up in actors are associated with imagination and empathy, while those lighting up in DID patients are involved with stress and fear responses3. I’m not entirely convinced—can actors, knowingly faking a condition, really stand in for delusional people who sincerely believe in their affliction? Still, the stats are strong; and it’s hard to argue with a different study in which the visual centers of a sighted person’s brain apparently shut down in a sighted person when a “blind” alter took the controls.