Moonwalking With Einstein: The Art and Science of Remembering Everything

by Joshua Foer

  So if photographic memory is just a myth, what about the Russian journalist S? If he wasn’t taking snapshots in his mind, what exactly was he doing?

  Sʹs exceptional memory wasn’t the only strange feature of his brain. He also suffered from a rare perceptual disorder known as synesthesia, which caused his senses to be bizarrely intertwined. Every sound S heard had its own color, texture, and sometimes even taste, and evoked “a whole complex of feelings.” Some words were “smooth and white,” others “as orange and sharp as arrows.” The voice of Luria’s colleague, the famous psychologist Lev Vygotsky, was “crumbly yellow.” The cinematographer Sergei Eisenstein’s voice resembled a “flame with fibres protruding from it.”

  Words set S’s mind ablaze with mental imagery. When you or I hear someone mention the word “elephant” or read the word on this page, we understand immediately that the referent is a large, gray pachyderm with thick legs and an oversize proboscis. But under most circumstances we don’t actually conjure up an image of an elephant in our mind’s eye. We might, if we choose to, but it takes a little extra effort, and in the course of normal conversation or reading, there’s usually no point to it. But that’s exactly what S did, automatically and instantaneously, with every word he heard. He couldn’t help it. “When I hear the word green, a green flowerpot appears; with the word red I see a man in a red shirt coming toward me; as for blue, this means an image of someone waving a small blue flag from a window,” he told Luria. Because every word summoned up an accompanying synesthetic image—sometimes also a taste or smell—S lived in a kind of waking dream, once removed from reality. While one universe unfolded around him, another universe of images blossomed in his mind’s eye.

  These images that populated S’s head were so powerful that they felt at times indistinguishable from reality. “Indeed, one would be hard put to say which was more real for him: the world of imagination in which he lived, or the world of reality in which he was but a temporary guest,” Luria wrote. All S had to do was imagine himself running after a train to make his pulse race, or envision sticking his hand in a hot oven to make his temperature rise. He claimed even to be able to abolish pain with his images: “Let’s say I’m going to the dentist ... I sit there and when the pain starts I feel it ... it’s a tiny, orange-red thread. I’m upset because I know that if this keeps up the thread will widen until it turns into a dense mass ... So I cut the thread, make it smaller and smaller, until it’s just a tiny point. And the pain disappears.”

  Even numbers had their own personalities for S: “Take the number 1. This is a proud, well-built man; 2 is a high-spirited woman; 3 a gloomy person (why, I don’t know); 6 a man with a swollen foot; 7 a man with a mustache; 8 a very stout woman—a sack within a sack. As for the number 87, what I see is a fat woman and a man twirling his mustache.” But while numbers were brought to life by S’s synesthesia, he had trouble understanding abstract concepts and metaphors. “I can only understand what I can visualize,” he explained. Words like “infinity” and “nothing” were beyond his grasp. “Take the word something for example. For me this is a dense cloud of steam that has the color of smoke. When I hear the word nothing, I also see a cloud, but that one is thinner, completely transparent. And when I try to seize a particle of this nothing, I get the most minute particles of nothing.” S was simply unable to think figuratively. An expression like “weigh one’s words” evoked images of scales, not prudence. Poetry was virtually impossible to read, unless it was completely literal. Even simple stories proved difficult to understand because his irrepressible image-making would bog him down as he tried to visualize every word, or else send his brain hurtling off to some other associated image, and some other memory.

  All of our memories are, like S’s, bound together in a web of associations. This is not merely a metaphor, but a reflection of the brain’s physical structure. The three-pound mass balanced atop our spines is made up of somewhere in the neighborhood of 100 billion neurons, each of which can make upwards of five to ten thousand synaptic connections with other neurons. A memory, at the most fundamental physiological level, is a pattern of connections between those neurons. Every sensation that we remember, every thought that we think, transforms our brains by altering the connections within that vast network. By the time you get to the end of this sentence, your brain will have physically changed.

  If thinking about the word “coffee” makes you think about the color black and also about breakfast and the taste of bitterness, that’s a function of a cascade of electrical impulses rocketing around a real physical pathway inside your brain, which links a set of neurons that encode the concept of coffee with others containing the concepts of blackness, breakfast, and bitterness. That much scientists know. But how exactly a collection of cells could “contain” a memory remains among the deepest conundrums of neuroscience.

  For all the advances that have been made in recent decades, it’s still the case that no one has ever actually seen a memory in the human brain. Though advances in imaging technology have allowed neuroscientists to grasp much of the basic topography of the brain, and studies of neurons have given us a clear picture of what happens inside and between individual brain cells, science is still relatively clueless about what transpires in the circuitry of the cortex, the wrinkled outer layer of the brain that allows us to plan into the future, do long division, and write poetry, and which holds most of our memories. In our knowledge of the brain, we’re like someone looking down on a city from a high-flying airplane. We can tell where the industrial and residential neighborhoods are, where the airport is, the locations of the main traffic arteries, where the suburbs begin. We also know, in great detail, what the individual units of the city (citizens, and in this metaphor, neurons) look like. But, for the most part, we can’t say where people go when they get hungry, how people make a living, or what any given person’s commute looks like. The brain makes sense up close and from far away. It’s the in-between—the stuff of thought and memory, the language of the brain—that remains a profound mystery.

  One thing is clear, however: The nonlinear associative nature of our brains makes it impossible for us to consciously search our memories in an orderly way. A memory only pops directly into consciousness if it is cued by some other thought or perception—some other node in the nearly limitless interconnected web. So when a memory goes missing or a name gets caught on the tip of the tongue, hunting it down can be frustrating and often futile. We have to stumble in the dark with a flashlight for cues that might lead us back to the piece of information we’re looking for—Her name begins with an L ... She’s a painter ... I met her at that party a couple years ago—until one of those other memories calls to mind the one we’re looking for. Ah yes, her name was Lisa! Because our memories don’t follow any kind of linear logic, we can neither sequentially search them nor browse them.

  But S could. S’s memories were as regimentally ordered as a card catalog. Each piece of information he memorized was assigned its own address inside his brain.

  Let’s say I asked you to memorize the following list of words: “bear,” “truck,” “college,” “shoe,” “drama,” “garbage,” and “watermelon.” You might very well be able to remember all seven of those words, but it’s less likely you’d be able to remember them in order. Not so with S. For S, the first piece of information in a list was always, and without fail, inextricably linked to the second piece of information, which could only be followed by the third. It didn’t matter whether he was memorizing Dante’s Divine Comedy or mathematical equations; his memories were always stored in linear chains. Which is why he could recite poems just as easily backward as forward.

  S kept his memories rigidly organized by mapping them onto structures and places he already knew well. “When S read through a long series of words, each word would elicit a graphic image. And since the series was fairly long, he had to find some way of distributing these images of his in a mental row or sequence,” wrote Luria. “Most often ..
. he would ‘distribute’ them along some roadway or street he visualized in his mind.”

  When he wanted to commit something to memory, S would simply take a mental stroll down Gorky Street in Moscow, or his home in Torzhok, or some other place he’d once visited, and install each of his images at a different point along the walk. One image might be placed at the doorway of a house, another near a streetlamp, another on top of a picket fence, another in a garden, another on the ledge of a store window. All this happened in his mind’s eye as effortlessly as if he were placing real objects along a real street. If asked to memorize those same seven words—“bear,” “truck,” “college,” “shoe,” “drama,” “garbage,” and “watermelon”—he would conjure up an image associated with each of them, and scatter them along one of his many mental paths.

  When S wanted to recall the information a day, month, year, or decade later, all he would have to do was rewalk the path where that particular set of memories was stored, and he would see each image in the precise spot where he originally left it. When S did, on rare occasions, forget something, “these omissions ... were not defects of memory but were, in fact, defects of perception,” wrote Luria. In one instance, S forgot the word “pencil” amid a long list of words that he was supposed to have memorized. Here’s his own description of how he forgot it: “I put the image of the pencil near a fence ... the one down the street, you know. But what happened was that the image fused with that of the fence and I walked right on past without noticing it.” On another occasion, he forgot the word “egg.” “I had put it up against a white wall and it blended in with the background,” he explained.

  S’s memory was a beast that indiscriminately gobbled up everything it was fed, and had trouble disgorging those pieces of information that were too trivial to be worth keeping. The greatest challenge S faced was learning what Luria called “the art of forgetting.” The rich images that every sensation created proved frustratingly indelible. He experimented with different techniques to wipe them from his mind. He tried writing things down, with the hope that he would then no longer feel a need to remember them. When that didn’t work, he tried burning the pieces of paper, but he could still see numbers hovering among the embers. Eventually he had an epiphany. One evening, while feeling particularly pestered by a chart of numbers he had earlier memorized, he figured out the secret of forgetting. All he had to do was convince himself that the information he wanted to forget was meaningless. “If I don’t want the chart to show up it won’t,” he exclaimed. “And all it took was for me to realize this!”

  One might assume that S’s vacuum-cleaner memory would have made him a formidable journalist. I imagined if I could only take notes without taking notes and have at my fingertips every fact I’d ever digested, I’d be immensely better at my job. I’d be better at everything.

  But professionally S was a failure. His newspaper gig didn’t last long, and he was never able to hold down a steady job. He was, in Luria’s estimation, “a somewhat anchorless person, living with the expectation that at any moment something particularly fine was to come his way.” Ultimately, his condition made him unemployable as anything but a stage performer, a theatrical curio like the mnemonist of Alfred Hitchcock’s The 39 Steps. The man with the best memory in the world simply remembered too much.

  In his short story “Funes the Memorious,” Jorge Luis Borges describes a fictional version of S, a man with an infallible memory who is crippled by an inability to forget. He can’t distinguish between the trivial and the important. Borges’s character Funes can’t prioritize, can’t generalize. He is “virtually incapable of general, platonic ideas.” Like S, his memory was too good. Perhaps, as Borges concludes in his story, it is forgetting, not remembering, that is the essence of what makes us human. To make sense of the world, we must filter it. “To think,” Borges writes, “is to forget.”

  While S’s capacious memory for facts seems almost unbelievable, he was in fact taking advantage of the well-developed spatial memory we all possess. If you visit London, you’ll occasionally cross paths with young men (and less often women) on motor scooters, blithely darting in and out of traffic while studying maps affixed to their handlebars. These studious cyclists are training to become London cabdrivers. Before they can receive accreditation from London’s Public Carriage Office, cabbies-in-training must spend two to four years memorizing the locations and traffic patterns of all 25,000 streets in the vast and vastly confusing city, as well as the locations of 1,400 landmarks. Their training culminates in an infamously daunting exam called “the Knowledge,” in which they not only have to plot the shortest route between any two points in the metropolitan area, but also name important places of interest along the way. Only about three out of ten people who train for the Knowledge obtain certification.

  In 2000, a neuroscientist at University College London named Eleanor Maguire wanted to find out what effect, if any, all that driving around the labyrinthine streets of London might have on the cabbies’ brains. When she brought sixteen taxi drivers into her lab and examined their brains in an MRI scanner, she found one surprising and important difference. The right posterior hippocampus, a part of the brain known to be involved in spatial navigation, was 7 percent larger than normal in the cabbies—a small but very significant difference. Maguire concluded that all of that way-finding around London had physically altered the gross structure of their brains. The more years a cabbie had been on the road, the more pronounced the effect.

  The brain is a mutable organ, capable—within limits—of reorganizing itself and readapting to new kinds of sensory input, a phenomenon known as neuroplasticity. It had long been thought that the adult brain was incapable of spawning new neurons—that while learning caused synapses to rearrange themselves and new links between brain cells to form, the brain’s basic anatomical structure was more or less static. Maguire’s study suggested the old inherited wisdom was simply not true.

  After her groundbreaking study of London cabbies, Maguire decided to turn her attention to mental athletes. She teamed up with Elizabeth Valentine and John Wilding, authors of the academic monograph Superior Memory, to study ten individuals who had finished near the top of the World Memory Championship. They wanted to find out if the memorizers’ brains were—like the London cabbies’—structurally different from the rest of ours, or if they were somehow just making better use of memory abilities that we all possess.

  The researchers put both the mental athletes and a group of matched control subjects into MRI scanners and asked them to memorize three-digit numbers, black-and-white photographs of people’s faces, and magnified images of snowflakes, while their brains were being scanned. Maguire and her team thought it was possible that they might discover anatomical differences in the brains of the memory champs, evidence that their brains had somehow reorganized themselves in the process of doing all that intensive remembering. But when the researchers reviewed the imaging data, not a single significant structural difference turned up. The brains of the mental athletes appeared to be indistinguishable from those of the control subjects. What’s more, on every single test of general cognitive ability, the mental athletes’ scores came back well within the normal range. The memory champs weren’t smarter, and they didn’t have special brains. When Ed and Lukas told me they were average guys with average memories, they weren’t just being modest.

  But there was one telling difference between the brains of the mental athletes and the control subjects: When the researchers looked at which parts of the brain were lighting up when the mental athletes were memorizing, they found that they were activating entirely different circuitry. According to the functional MRIs, regions of the brain that were less active in the control subjects seemed to be working in overdrive for the mental athletes.

  Surprisingly, when the mental athletes were learning new information, they were engaging several regions of the brain known to be involved in two specific tasks: visual memory and spatial navigation, including the sam
e right posterior hippocampal region that the London cabbies had enlarged with all their daily way-finding. At first glance, this wouldn’t seem to make any sense. Why would mental athletes be conjuring images in their mind’s eye when they were trying to learn three-digit numbers? Why should they be navigating like London cabbies when they’re supposed to be remembering the shapes of snowflakes?

  Maguire and her team asked the mental athletes to describe exactly what was going through their minds as they memorized. The mental athletes recounted a strategy that sounded almost exactly like what S claimed had been happening in his brain. Even though they were not innate synesthetes like S, the mental athletes said they were consciously converting the information they were being asked to memorize into images, and distributing those images along familiar spatial journeys. Unlike S, they weren’t doing this automatically, or because it was an inborn talent they’d nurtured since childhood. Rather, the unexpected patterns of neural activity that Maguire’s fMRIs turned up were the result of training and practice. The mental athletes had taught themselves to remember like S.

  I found myself fascinated by Ed and his quiet friend Lukas, and this formidable-sounding project of theirs to push their memories as hard and as far as they possibly could. And they likewise seemed fascinated with me, a journalist of roughly the same age, who might share their story in some magazine they’d never heard of, and perhaps jump-start their careers as mnemonic celebrities. After Ed’s lecture at the high school, he invited me to follow him and Lukas to a nearby bar, where we met up with an aspiring filmmaker and old boarding school chum of Ed’s who had been trailing them around New York with an 8-mm video camera, documenting their every antic adventure, including Lukas’s attempt to memorize a deck of playing cards on the fiftythree-second elevator ride to the Empire State Building’s observation deck. (“We wanted to see if the fastest lift in the world was faster than the Austrian speed cards champion,” Ed deadpanned. “It wasn’t.”)