9
MULTISENSORY AND NONVISUAL ILLUSIONS
Our nervous system gathers all different kinds of sensory information from the outside world and from our own bodies, but visual inputs take up the most processing power. Though the experience of viewing an object may seem simple and effortless, our brain’s cortex alone dedicates more than two dozen areas to analyzing, filling in, transforming, and relaying visual inputs. Such extraordinary allocation of neural muscle can make us lose sight—pun intended—of the fact that our experience of the world is fundamentally multisensory. Scientists and philosophers since Aristotle have pondered how the brain is able to integrate different kinds of sensory information into a unified perception. In the initial stages of neural processing, the inputs from the different senses (sight, touch, hearing, taste, and smell) are kept separate, in different processing streams that lead to different cortical areas of the brain (the visual cortex, the somatosensory cortex, the auditory cortex, and others). These areas are physically distinct and largely disconnected from one another. Even within a given sensory modality, such as in the visual system, specific cortical areas and populations of neurons deal with particular features, such as motion, color, texture, and faces. How the brain puts together different kinds of information so that when we encounter a puppy we can see it wagging its tail, feel its fur under our hands as we pet it, and hear its excited bark all at once, as a single perception, is a neuroscientific mystery known as “the binding problem.” Some of the illusions in this chapter illustrate the paradoxical features of our nonvisual perception. Others demonstrate the unusual ways in which our brain integrates (or sometimes fails to integrate) the information arriving from our different senses.
THE DISAPPEARING HAND TRICK
BY ROGER NEWPORT, HELEN GILPIN, AND CATHERINE PRESTON
UNIVERSITY OF NOTTINGHAM, U.K.
2012 FIRST PRIZE
A trusting young woman puts her hands in a box with a transparent top. She is participating in a science experiment, but this one has the aura of a magic show. The researchers ask her to hold her hands steady between two vertical blue lines. She does so, watching her hands carefully. They do not appear to move, nor does she feel as if they are moving. The investigators flick a switch and the right side of the box darkens, obscuring her right hand. They ask her to reach across with her left hand and touch her now invisible right one. She complies—but her eyes widen with alarm. All she feels is empty space. “Where’s my hand gone?” she asks anxiously. Then, suddenly, she explodes with laughter. Still, just to be sure, she pulls both her hands from the box to check that they are still there—they are.
This scene, captured on video, helped the inventors of the mirage multisensory illusion box win first prize at the 2012 Best Illusion of the Year Contest. Like many illusions, this one was discovered by accident. Three psychologists created the box to study how the brain integrates visual input, tactile information, and proprioception, or bodily sensations that tell us about the position of our limbs in space. One day, Roger Newport, one of the psychologists, was trying to fix a misalignment in the box. He discovered that his right hand was in the wrong place and his left one was out of sight. “I tried to touch my left hand with my right and missed it. I was so surprised I decided to see whether I could re-create the feeling experimentally,” he says. Equipped with a camera, a mirror, and a monitor, the box created the illusion that the woman was looking at her own hands when in fact she was seeing a video re-creation of them. The hand images, manipulated by computer software, moved slowly inward. To compensate, the woman moved her hands outward—although it all happened so gradually that she did not notice. In less than a minute, the space between her hands became much greater than she realized. Then she was surprised when she tried to touch her hand and discovered that it was not where she thought it was.
In everyday experience, sensations such as sight, touch, and proprioception work together to inform us about the location of our various body parts. Think of how jerky a baby’s early movements are, and how unsteady she is as she learns to walk. She dynamically adjusts and re-adjusts virtually every muscle in her body as she struggles to remain upright. You do the same, even when sitting still, just more smoothly and without conscious oversight. The mirage box created by Newport and his colleagues shows that the brain is easily confused when our touch and bodily sensations are dissociated from the visual input. For a demonstration, visit the Best Illusion of the Year Contest website. An amusing video showing the best reactions from participants in the Disappearing Hand Trick is available on YouTube.
THE KNOBBY SPHERE ILLUSION
BY PETER TSE
DARTMOUTH COLLEGE, U.S.A.
2013 FINALIST
Illusions are not only perceived by our visual system. They can also be created through the sense of touch, as noticed by the cognitive neuroscientist Peter Tse of Dartmouth College. Try it yourself. Get a pencil and a small, round, hard sphere, such as a ball bearing or a marble. First, squeeze the sides of the pencil very tightly between your thumb and forefinger for sixty seconds or so, until you make deep indentations in your thumb and forefinger pads. Now feel the ball bearing at the location of the indentations by rolling it around. The ball no longer seems round but instead feels as if it has rounded corners—as if the ball were hexagonal in cross section, just like the pencil. When you squeeze the pencil, the array of touch receptors in your skin takes on its shape. Your brain assumes that the skin where your receptors are located is smooth and round, and it misattributes these perceived edges as belonging to the ball.
FISHBONE ILLUSION
BY MASASHI NAKATANI
KEIO UNIVERSITY, JAPAN
2011 FINALIST
This tactile illusion will give you a very fishy feeling. At the 2011 Best Illusion of the Year Contest, the scientist Masashi Nakatani, who studies haptic perception (i.e., the sense of touch), showed up dressed in spearfishing attire and passed out business cards embossed with the shape of a stylized fish. With the flourish of a magician, Nakatani approached the emcee, the vision scientist Peter Thompson, onstage and told him, “Rub your finger up and down the spine. Up and down the spine. You feel a groove there? But … there … is … none!” Thompson rubbed the fish’s spine as instructed. “How do you feel?” Nakatani asked.
“I feel dirty for feeling up this fish,” Thompson deadpanned. “But I do feel a groove here.”
Nakatani was intrigued by the possibility that the brain might experience tactile as well as visual illusions. He thought he could create a texture that was not a circle but would feel like one. The fishbone pattern (without a head or tail) was one of many botched attempts. Disappointed, Nakatani took the sample to a senior colleague, Susumu Tachi, and described his failure. Tachi agreed that the texture did not feel like a circle, but he noticed a central groove where there was none. He encouraged Nakatani to change his dissertation project to study the fishbone illusion full-time. By testing a variety of configurations and textures, Nakatani and his colleagues determined that the illusion arises from how tactile receptors in your skin compare smooth and rough textures. Even though the spine and ribs of the fish are embossed exactly the same way, the ribs feel rougher to the touch because they are discontinuous; the spine is continuous from beginning to end, so it feels smoother than the ribs. Your brain interprets the smooth spine to be lower than the rough ribs of the fish—and you’re hooked.
DISAPPEARING SMOKE— DISAPPEARING PLEASURE!
BY SIDNEY PRATT, MARTHA SANCHEZ, AND KARLA ROVIRA
SIN HUMO, COSTA RICA
2013 FINALIST
The clinical psychologists Sidney Pratt, Martha Sanchez, and Karla Rovira are members of Sin Humo, an anti-smoking treatment program in Costa Rica whose name translates to “Without Smoke.” Their illusion was inspired by a relaxation technique in which a person closes his or her eyes while smoking. This small act, surprisingly, decreases the enjoyment typically felt from a cigarette—and less enjoyment, they reasoned, would lead
to weaker addiction.
Pratt’s team wondered, was it relaxation per se that reduced the patients’ gratification? Or was it primarily the fact that the patients’ eyes were closed, preventing them from seeing the burning cigarette? To find out, they simply asked the patients to smoke while blindfolded, and learned that the blindfold did in fact account for the reduction in the pleasure from smoking.
The contest judges classified this demonstration as an illusion because the smokers’ perceptions did not match the biochemical reality of smoking. In other words, we assume that the amount of pleasure derived from a cigarette should be based on the amount of nicotine it delivers. That amount, of course, is the same with or without the blindfold, and so the patients’ brains were effectively tricked.
In light of these findings, the researchers reasoned that seeing the smoke from a cigarette might strengthen nicotine addiction because the smoke is associated with past experiences of feeling satisfied. Their discovery also suggests that it’s very rare for any experience to affect a single sensory modality. All of our senses may be engaged in powerful, unexpected ways when we interact with the world around us. The blindfolding technique is now the cornerstone of the team’s anti-smoking treatment plan.
10
ATTENTION ILLUSIONS
For a neuroscientist, the trouble with cocktail parties is not that there are cocktails or that it’s a party (many neuroscientists love both). Instead, what we call “the cocktail party problem” is the mystery of how anyone can have a conversation at a cocktail party at all.
Consider a typical scene: You have a dozen or more lubricated and temporarily uninhibited adults telling loud, improbable stories at increasing volumes. Interlocutors guffaw and slap backs. Given the decibel level, it is a minor neural miracle that any of these revelers can hear and parse even one word emitted by their friends.
The alcohol does not help to solve this complicated problem, but it is not the main source of confusion. The issue is that there is simply too much going on at once. How can our brains filter out the noise and focus on specific information?
This problem is a central one for neuroscientists—and not just during cocktail parties. The entire world we live in is quite literally too much to take in. Yet the brain does gather all of this information somehow and sorts it in real time, usually seamlessly and correctly. The sounds and sights around you include at least as much noise as signal, but the conversation or object that interests you remains in clear focus—at least this is how you perceive it.
How does the brain accomplish this feat? One critical component is that our neural circuits actively suppress anything that is not task-relevant. Our brains pick their battles. They stomp out irrelevant information so that the good stuff has a better chance of rising to awareness. This process, colloquially called “attention,” is how the brain sorts the wheat from the chaff. You pick and choose what you want to see and hear, and your attention system filters out the noise—which is everything else.
In collaboration with the laboratories of the neuroscientists Jose-Manuel Alonso of the SUNY College of Optometry and Harvey Swadlow of the University of Connecticut, we discovered some of the initial circuits that mediate attention in the primary visual cortex of the brain. To do so, we observed neurons in this area, some of which encourage activity in other brain cells, so-called excitatory neurons, and others that tamp down activity, known as inhibitory neurons. We found that when someone attends to a specific location, the inhibitory neurons take action, suppressing responses to other visual locations. In short, the brain depends on inhibitory neurons to enable focus.
Even more interesting was our finding that the harder you concentrate, the greater the suppression. One fundamental role of cognition is to select what your brain goes on to process. It does that, at least in part, by blocking irrelevant information.
But that is not attention’s only role. As the neural activity associated with attention travels down throughout the visual system’s circuits, it can also affect how we perceive and interpret the color and brightness of objects. The illusions in this chapter illustrate some of the numerous perceptual consequences of our brain’s attentional mechanisms.
James Randi (aka the Amaz!ng Randi), on the left, performed at the 2010 Best Illusion of the Year Contest. Dan Simons, on the right, is posing in his signature gorilla costume.
THE MONKEY-BUSINESS ILLUSION
BY DANIEL SIMONS
UNIVERSITY OF ILLINOIS, U.S.A.
2010 FINALIST
In a famous experiment done in 1999, Daniel Simons and Christopher Chabris, both then at Harvard University, asked subjects to watch two groups of people dribbling and passing a basketball among themselves. Three players wore white shirts; three wore black. The watchers were asked to count the number of passes by the players in white shirts. About halfway through the exercise, a gorilla—that is, a person in a gorilla suit—walked into the ball-passing scene, beat its chest while facing the camera, then walked out.
Simons and Chabris were shocked to discover that about half of the people counting passes failed to notice the gorilla. Their spectacular demonstration became an instant classic, spreading to conferences, university courses, and textbooks. It is an excellent example of inattentional blindness, a phenomenon in which the brain ignores information that is not relevant to its current task. The gorilla illusion is so well-known that Simons decided to create a variation for the 2010 illusion contest. He appeared at the gala dressed as a gorilla, flinging bananas to the audience before he took the stage. “You are all good vision scientists,” he said. “You know that when people are passing basketballs you should be looking for gorillas.” The audience roared with laughter at the inside joke.
People can only experience the invisible gorilla illusion once. After you know to look for a gorilla, you’ll never miss it again. Does knowledge of the impending occurrence of unexpected events help you detect other unexpected events? Simons’s latest demonstration, called the Monkey-Business Illusion, shows that the answer is no. People who know to look for a gorilla are of course more likely to spot the gorilla. But these same viewers will fail to notice other unanticipated happenings—and are even more likely to do so than viewers who are unfamiliar with the illusion. Again, the harder you pay attention during a task, the more powerfully your visual system suppresses distracting information. The more you watch out for the gorilla that you expect to appear, the more you will miss other changes that are unpredicted.
As the gorilla-suited Simons explained, there are several changes that most people overlook when they watch the Monkey-Business Illusion: the background of the image changes color from red to gold, and one of the three black-shirted players leaves the game by discreetly backing out of the scene. Simons had one final surprise: “Did any of you spot a pirate?” Simons asked the audience. The spectators groaned, rolled their eyes, and shook their heads at yet another impossible oversight. But the undetected pirate was not in the video. Simons pointed to stage right, where a spotlight beamed down on a pirate, previously unnoticed yet completely out in the open for all to see, holding a sword to the neck of the contest’s technical director (Steve). The illusion had spilled out of the video onto the stage!
Still frames from the illusion show (top left) the scene before the gorilla appears, (top right) the gorilla entering and one of the players in black backing out of the scene, (middle left) the gorilla thumping its chest, (middle right) the gorilla exiting, and (bottom) the scene after the gorilla has left. The color of the curtain has changed, and now only two black-shirted players remain.
ATTENTION TO BRIGHTNESS
BY PETER TSE
DARTMOUTH COLLEGE, U.S.A.
2005 FINALIST
At the 2005 illusion contest, Peter Tse presented one of the simplest but most important illusions ever discovered: three semitransparent overlapping circles. Look carefully at the blue dot in the center of the three intersecting disks while directing your attention to each of the three disk
s in turn. If you are paying attention to the bottom disk, for example, you will see that it looks brighter than the other two disks. The same is true when you turn your attention to one of the other disks. Before Tse discovered this illusion, neurophysiologists believed that people cast a spotlight of attention on a specific location, leaving the rest of the world in relative darkness. Tse showed that the spotlight concept was not just a useful metaphor. When we direct our attention to a specific object or area of an image, it appears more salient than the regions we don’t attend to. Even the apparent contrast of the focused area is higher than the rest of the picture! The neural mechanisms responsible for this phenomenon are the same that prevent you from seeing the gorilla when counting basketball passes in Simons and Chabris’s original demonstration. Our brain actively suppresses the parts of a scene that we don’t pay attention to, so that the regions that we do focus on appear more prominent in comparison.
ATTENTION TO COLOR
BY PETER TSE
DARTMOUTH COLLEGE, U.S.A.
2012 FINALIST
Seven years after creating the Attention to Brightness Illusion, Tse discovered a related color effect. Three colored disks overlap in the center like a Venn diagram. If you fix your gaze on the central intersection and focus on one disk, that entire disk will appear to become uniform in color—for instance, the disk with the blue outline and outer region will appear uniformly blue. It will also seem to be floating transparently above the other disks, even though the colors are mixed in some regions—and the center of the composition is actually gray!
Champions of Illusion Page 6
