The New Science of the Mind

by Mark Rowlands

  A change in a visually presented scene, like a photograph, would normally cause a transient signal in your visual apparatus. This transient signal is detected by low-level (i.e., unconscious) visual mechanisms in this apparatus, and the result is that your attention is automatically attracted to the location of the change. This is why you notice the change. In a change-blindness experiment, however, a way is found of nullifying the role of the visual transient. This can be done in several ways (and if you visit O'Regan's Web site, you will see all of these). One method involves superimposing a very brief global flicker over the whole visual field at the moment of the change. In other words, the scene is momentarily replaced by a simple gray frame. During the period of gray-out, a change is made in the photograph. Another method involves creating a number of simultaneous local disturbances-which appear something like mud splashes on the scene-that act as decoys and so minimize the effect of the local transient. The same sort of effect can also be achieved by making the change in the photograph coincide with an eye saccade, an eye blink, or a cut in a film sequence. In all these cases, a brief global disturbance swamps the local transient and thus prevents it from playing its normal role of grabbing your attention.

  Experiments conducted by O'Regan (1992) and others (e.g., Blackmore et al. 1995; Rensink, O'Regan, and Clark 1997) showed that under these sorts of conditions, observers have great difficulty seeing changes, even though they are large and occur in full view. If you are at all typical, you will find some of the changes easier to spot than others. But some of them will be very difficult to spot, and all of them will be harder to spot than if you were just looking at a changing scene where the change was not masked. In one version of the experiment, O'Regan even showed that an observer could be looking directly at the change at the moment it occurs and still not notice it (O'Regan et al. 2000).

  Simons and Levin (1997) have show that the same sorts of results can be obtained when one switches from computer screens to real-world situations. One striking experiment turned on a slapstick scenario-of which Simons and Levin are the undisputed world heavyweight champions-in which an experimenter, pretending to be lost on the Cornell Campus, would approach an unsuspecting passerby to ask for directions. Once the passerby started to reply, two people carrying a large door would walk between the inquirer and the passerby. During the walk through, however, the original inquirer is replaced by a different person. Only 50 percent of the direction-givers noticed the change. Yet the two experimenters were of different heights, wore different clothes, had different voices, and so on. Moreover, those who did notice the change were students of roughly the same age and demographics as the two experimenters. In a follow-up study, the students failed to spot the change when the experimenters appeared as construction workers, placing them in a different social group. The conclusion that Simons and Levin (1997, 266) draw is that our failures to detect change arise because "we lack a precise representation of our visual world from one view to the next." We encode only a "rough gist" of the current scene-just enough to support a broad sense of what's going on insofar as it matters to us, and to guide further intelligent information retrieval as and when it is needed.

  In a related experiment, one that gave rise to a paper with almost certainly the best title ever to grace the pages of an professional academic journal-the paper was called "Gorillas in Our Midst"-Simons and Chabris (2000) had their experimental subjects watch a video of two teams (one in white, one in black) passing basketballs (one per team). Each subject was asked to count the number of successful passes made by the white team. Afterward, subjects are asked whether they saw anything else, anything unusual. A short time into the film (about 45 seconds) an intruder will make an appearance-walking through the middle of the players. The intruder may be the semitransparent figure of a woman holding an umbrella or a semitransparent man in a gorilla suit. On some trials, the woman or man in gorilla suit were presented in fully opaque form. In the semitransparent condition, 73 percent of subjects failed to see the gorilla, and even in the opaque condition 35 percent of subjects failed to see it (Simons 2000, 152). Simons concludes, "We do not realize the degree to which we are blind to unattended and unexpected stimuli, and we mistakenly believe that important events will automatically draw our attention away from our current task or goals" (2000, 154). This is an example of inattentional blindness, a phenomenon closely related to change blindness.

  In the next chapter, we shall examine attempts to explain these phenomena in more depth. For present purposes, however, two points are worthy of note. First, the phenomena of change and inattentional blindness seem to indicate-at least arguably-that the role traditionally assigned to visual representations has been somewhat overplayed. Instead of providing us with complex and detailed simulacra of the visual world, the role of internal visual representations is to provide us with the "rough gist" of the situation. That is, in general, visual representations provide us with a map of the perceived world that is partial, incomplete, and lacking in the sort of detail we take as unproblematically given in our visual experience. We do not, after all, visually experience the rough gist of the world around us: we experience it in all its concrete, rich, and detailed glory. Therefore, visual representations, by themselves, cannot explain the phenomenological character of our visual experience.

  Second, the slack left by this downgrading of the role of visual representations, and the resulting gap between visual representations and visual phenomenology, is taken up by the ability of an organism to act on the world around it. We do not need to reproduce internally the complexity, detail, and richness that we take, correctly, to be part of the phenomenology of visual experience. Instead, we make use of the complexity, detail, and richness that make up the visually detectable world around us. By constantly directing and redirecting our attention to this world, we avail ourselves of the complexity, detail, and richness it contains, and thereby obviate the need to reproduce these internally.

  We shall look at these ideas in much more detail in the next chapter. For now, we can simply note that the general explanatory profileattenuation of the role of representation coupled with augmentation of the role of action is one that is replicated throughout the cluster of theories that make up our prospective new science.

  3 The Ecological Theory of Visual Perception

  Some of the most important features of this sort of enactive account of perception discussed in the previous section were anticipated by a Cornell University Gibson. Gibson's account, which he developed from the late 1950s until his death in 1979, became known as the ecological theory of visual perception. During the first two decades of its life, this theory was widely reviled, by psychologists and philosophers alike, as a hopelessly confused, "magical" account of vision. Now, however, it is slowly beginning to be appreciated that the rise of non-Cartesian cognitive science is, in effect, a vindication of the work of James Gibson.

  As we have seen, traditional approaches to visual perception, of the sort exemplified by Marr, adhere closely to the following sort of explanatory framework:

  1. Perception begins with stimulation of the retina by light energy impinging on it.

  2. This results in a retinal image, characterized in terms of intensity values distributed over an array of different locations.

  3. Retinal images carry relatively little information, certainly not enough to add up to genuine perception.

  4. In order for perception to occur, the information contained in the retinal image has to be supplemented and embellished (i.e., processed) by various information-processing operations.

  5. These information-processing operations occur inside the skin of the perceiving organism.

  This framework assumes that visual perception begins with the retinal image. Gibson's first insight-and perhaps his most important one-was to see that this is not so: visual perception begins not with the retinal image but with what Gibson (1966, 1979) called the optic array.

  Light from the sun fills the air-the terre
strial medium-so that it is in a "steady state" of reverberation. In this steady state, the environment is filled with rays of light traveling between the surfaces of objects. At any point in space, light will converge from all directions. Therefore, at each point there is what can be regarded as a densely nested set of solid visual angles, which are composed of inhomogeneities in the intensity of light. Thus, we can imagine an observer, at least for now, as a point surrounded by a sphere, which is divided into tiny solid angles. The intensity of light and the mixture of wavelengths vary from one solid angle to another. This spatial pattern of light is the optic array. Light carries information because the structure of the optic array is determined by the nature and position of the surfaces from which it has been reflected.

  The optic array is divided into many segments or angles. Each of these contains light reflected from different surfaces, and the light contained in each segment will differ from that in other segments in terms of its average intensity and distribution of wavelength. The boundaries between these segments of the optic array, since they mark a change in intensity and distribution of wavelength, provide information about the threedimensional structure of objects in the world. At a finer level of detail, each segment will, in turn, be subdivided in a way determined by the texture of the surface from which the light is reflected. Therefore, at this level also, the optic array can carry information about further properties of objects and terrain.

  The realization of the importance of the optic array is, in many ways, Gibson's essential insight. In effect, the rest of his ecological approach stems from placing the optic array in its proper position of conceptual priority. The optic array is an external information-bearing structure. It is external in the quite obvious sense that it exists outside the skins of perceiving organisms and is in no way dependent on such organisms for its existence. It also carries information about the environment. Indeed, according to Gibson, there is enough information contained in the optic array to specify the nature of the environment that shapes it. Information, for Gibson, is essentially nomic dependence. The structure of the environment depends, in a lawlike (i.e., nomic) way on the structure of the physical environment that surrounds it. In virtue of this dependence, the optic array carries information about this wider environment. The optic array is, as Gibson puts it, specific to the environment. Because of this, an organism whose perceptual system detects optical structure in the array can thereby be aware of what this structure specifies. Thus, the perceiving organism is aware of the structure and not the array and, more importantly, is in a position to utilize the information about the environment embodied in the array.

  Once we allow that the optic array is an external structure that embodies information about the environment, we are, of course, forced to admit that some of the information relevant to perception exists in the environment of the perceiver. This may seem like a mundane observation. However, an important conclusion follows from it. Suppose we are faced with a particular perceptual task. If we accept the idea of the optic array, we must allow that at least some of the information relevant to this task will be located in the array. Perhaps, as Gibson seems to suggest, this information will be sufficient for us to accomplish the task. Perhaps not-in which case we might find it necessary to postulate internal processing operations that supplement or embellish the information contained in the array. Even if this is so, however, one thing is clear: we cannot begin to estimate what internal processing an organism needs to accomplish unless we understand how much information is already available to that organism in its optic array. The more information available to the organism in its optic array, the less internal processing the organism needs to perform. Understanding the internal processes involved in visual perception is logically and methodologically secondary to understanding the information that is available to the perceiving organism in its environment.

  The next stage in understanding visual perception would be to provide an account of how a perceiving organism is able to make the information contained in the optic array available to it. Here we find another distinctively Gibsonian element: the emphasis on action. As Gibson points out, and the enactive approach would echo this point nearly half a century later, perception is inextricably bound up with action. Perceiving organisms are not, typically, static creatures but, rather, actively explore their environment. The optic array is a source of information for any organism equipped to take advantage of it. But the optic array does not impinge on passive observers. Rather, the living organism will actively sample the optic array. When an observer moves, the entire optic array is transformed, and such transformations contain information about the layout, shapes, and orientations of objects in the world.

  By moving, and so effecting transformations in the optic array, perceiving organisms can identify and appropriate what Gibson calls the invariant information contained in the array. This is information that can be made available to an observer not by any one static optic array as such but only in the transformation of one optic array into another. Consider, for example, what is known as the horizon ratio relation (Sedgwick 1973). The horizon intersects an object at a particular height. All objects of the same height, whatever their distance from the observer, are cut by the horizon in the same ratio. This is one example of invariant information. An organism can detect such information only by moving and, hence, bringing about transformations in the optic array. Information, in this sense, is not something inside the organism. It exists as a function of changes in the organism-environment relation.

  Therefore, crucial to Gibson's account are two claims: (i) the optic array, a structure external to the perceiving organism, is a locus of information for suitably equipped creatures; and (ii) a creature can appropriate or make this information available to itself through acting on the array, and thus effecting transformations in it. What the perceiving creature does, in effect, is manipulate a structure external to it-the optic array-in order to make available to itself information that it can then use to navigate its way around the environment. And the claim that the organism makes information available to itself simply means that the organism makes information susceptible, or amenable, to detection by itself. Thus, information about the relative heights of objects is contained in the organism's environment. However, this information is not directly available to the organism. It becomes available-that is, amenable to detection by the organism-only when the organism moves, and thus brings about transformations in the structure of ambient light.

  Gibson has usually been understood as hostile to the idea of mental representations, and it's common to find his account presented as obviating the need for them. However, I think the real value of Gibson's account is safeguarded by another, less sanguine interpretation. Whether or not Gibson would have endorsed it-and I have been made aware that at least some contemporary Gibsonians do note-the interpretation is compatible with his overall theory.

  Recall the general contours of the old science, as represented by Marr's theory of vision. Perception consists in the manipulation and transformation of internal information-bearing structures-representational items ranging from the raw primal sketch all the way through to 3D object representations. Gibson's account replaces at least some of the need for manipulation and transformation of internal information-bearing structures with the manipulation and transformation of external informationbearing structures-the optic array. There is nothing in Gibson's theory itself-as opposed, perhaps, to his statements about his theory-that entails or even suggests that all of the role traditionally assigned to manipulation and transformation of internal information-bearing structures can be taken over by the manipulation and transformation of external informationbearing structures (Rowlands 1995). Precisely how much of the role traditionally assigned to the manipulation and transformation of internal information-bearing structures can be taken over by the manipulation and transformation of external information-bearing structures is, of course, an empirical question. And there is no reason at all for
thinking that Gibson's theory entails that all of the role of the former can be taken over by the latter. However, crucially, we will not know what sorts and how many kinds of internal information-processing operations we need to posit in order to explain an organism's perceptual ability unless we understand the ways in which, and the extent to which, the organism is able to manipulate and transform relevant structures in its environment and so make available to itself information required for the prosecution of its perceptual tasks.

  Therefore, we find in Gibson's work the twin themes characteristic of the nascent new science: attenuation of representation combined with augmentation of action. There is the questioning of, and limited (at least on my interpretation) hostility to, the role played in Cartesian cognitive science by the notion of mental representation. And there is the idea that at least some of the role traditionally played by mental representations can be taken over by the perceiving organism acting on the world in appropriate ways.

  4 From Russia with Love

  That the work of Gibson should turn out to be a version of what I am calling the new science shows that this science is not, in fact, that new. An even more striking example of just how old the new science is can be found in work I briefly mentioned in the previous chapter. This is the work of two Soviet psychologists, Anton Luria and Lev Vygotsky (1930/1992).

  A natural expression of the Cartesian conception is what we might, following Hurley (1998), call the vertical sandwich model of the relation between sensation, cognition, and action. This model was clearly implicated in Marr's account of vision. This is, in essence, an input-output model of the mind. Sensation provides the input to the system; action is the system's output. And cognition consists in information-processing operations-the manipulation and transformation of information-bearing structures in the brain. One noticeable feature of Cartesian cognitive science is that it locates perception in the middle of the sandwich-as a part of cognition, and therefore, quite different from sensation. Nonetheless, one would also have to acknowledge that, intuitively, perception is far more closely connected to the world than other types of cognitive process, such as thinking, reasoning, and remembering. The starting point for perception is provided by sensation; but, one might argue, this is not true of any other type of cognitive process. Therefore, if perception is to be found in the sandwich's filling, it is in the outer part of that filling. Perception is not so much the burger as the lettuce, gherkins, and mayonnaise.