Human Error

by James Reason

  8.2. Questionnaire studies

  Another way of obtaining data about everyday errors is through self-report questionnaires. Most commonly, they present subjects with descriptions (and/or examples) of different slips and lapses and ask them to rate approximately how often they have experienced each one during some specified time period. Of course, the subjects can only respond with general impressions, and these are liable to various types of distortion. Nevetheless, such methods can yield interesting data with regard to individual differences in error proneness, the relatedness of various error types and the organisation of the underlying control mechanisms. In addition, they can provide a rough indication of how people perceive the relative frequencies of particular kinds of cognitive failure. As such, they offer a valuable supplement to the extended diary mode of naturalistic investigation (for a detailed evaluation of these questionnaire methods, see Herrmann, 1982,1984; Morris, 1984).

  As indicated in the Preface, these studies have been recently reviewed elsewhere (Reason, 1988) and will not be discussed in any detail in this book. However, the principal conclusions of this review are summarised later in the chapter.

  Self-reported liability to minor cognitive failures is a relatively stable and enduring feature of the individual (Broadbent, Cooper, FitzGerald & Parkes, 1982; Herrmann, 1982; Reason & Lucas, 1982b, 1984b). Responses to questionnaire items asking about the incidence of a wide variety of minor cognitive failures remain fairly consistent over several months. Moreover, self-assessments are usually confirmed by marital partners. These findings suggest that there is some genuine correspondence between self-reports of error proneness and everyday behaviour.

  Responses to questionnaire items are generally positively correlated. Thus, those people who confess to being particularly liable to one kind of cognitive failure (e.g., memory lapses) also tend to report a high degree of susceptibility to other types as well (e.g., action slips), and conversely. This suggests that error proneness is not specific to any one cognitive domain, but operates more or less uniformly across all types of mental function (see Broadbent et al., 1982; Reason & Mycielska, 1982). The implication is that susceptibility to these usually inconsequential slips and lapses is governed by some global factor relating to the characteristic way in which an individual manages his or her cognitive affairs.

  Observations taken from a wide range of samples, in which people were exposed to different stresses (nurses under training, women facing breast surgery, students preparing for important examinations), support Broadbent’s stress-vulnerability hypothesis: namely, that relatively high levels of cognitive failure in normal everyday life are associated with increased vulnerability to externally imposed stresses. Whatever governs general proneness to everyday slips and lapses also appears to contribute to stress vulnerability. This factor eludes capture by most simple laboratory measures, but it seems to relate closely to the deployment of limited attentional resources in the face of competing task demands. The evidence so far assembled suggests that it is not so much that stress induces a high rate of cognitive failure, but that certain styles of cognitive management can lead to both absent-mindedness and to the inappropriate matching of coping strategies to stressful situations.

  8.3. Laboratory studies

  The relative merits of naturalistic and experimental modes of enquiry and their mutual interdependence were concisely stated by Baars (1980): “Without naturalistic facts, experimental work may become narrow and blind: but without experimental research, the naturalistic approach runs the danger of being shallow and uncertain.”

  Undoubtedly, the most powerful technique for studying underlying mechanisms is through the deliberate elicitation of particular error types under controlled laboratory conditions. Such investigations follow in the footsteps of Helmholtz, Mach and the other great German ‘illusionists’ of the nineteenth century. Some of the most impressive modern studies have involved the deliberate elicitation of speech errors (see Baars, 1980; Fromkin, 1980).

  There are at least two problems associated with the experimental investigation of errors. First, the need to establish precise control over the possible determinants of the error often forces investigators to focus upon rather trivial phenomena (i.e., the large number of studies on the Stroop effect). Second, it is usually the case that the greater the measure of control achieved by the experimenter, the more artificial and unnatural are the conditions under which the error is elicited.

  8.4. Simulator studies

  An important recent development has been the use of computer-based simulations to create within the laboratory many of the dynamic features of real-life, complex decision-making tasks that were hitherto lacking in static, one-shot experimental studies. The pioneers in this field have been Berndt Brehmer and his associates at the University of Uppsala (Brehmer, Allard & Lind, 1983; Brehmer, 1987) and Dietrich Doerner at the University of Bamberg (Doerner, 1978; Doerner & Staudel, 1979; Doerner, 1987).

  The Swedish research has examined the effects of complexity and feedback delay upon decision making in highly dynamic situations, such as directing the fighting of a forest fire. The Bamberg group has been primarily concerned with charting the ‘pathologies’ of individual decision making in complex situations (e.g., being mayor of a small middle-European town). The work of these two groups is discussed further in Chapter 3.

  8.5. Case studies

  As in other branches of psychology, the intensive study of the single case can yield valuable information about the circumstances leading up to catastrophic errors. Where sufficient evidence is available regarding both the antecedent and the prevailing circumstances of a particular event or accident, we are able to study the interaction of the various causal factors over an extended time scale in a way that would be difficult to achieve by other means. Although any one catastrophe may result from the unhappy conjunction of several distinct causal chains, and hence be a truly unique happening, the precise effects of this particular combination of contributing factors teaches us something about the limits of human performance that could not be obtained from either the laboratory or from naturalistic observations.

  Inevitably there are problems. The primary sources of data are accident reports. Not only are these mostly concerned with attributing blame, they also tell a story that may be inaccurate or incomplete, even when the reports are prepared by experienced and relatively open-minded investigators. There are two main difficulties. First, an accident report will always contain less information than was potentially available. Second, a written account has the effect of ‘digitizing’ what in the original was a complex and continuous set of ‘analogue’ events.

  One of the important lessons of these case studies is that disasters are very rarely the product of a single monumental blunder. Usually they involve the concatenation of several, often quite minor, errors committed either by one person or, more often, by a number of people. In general, the errors contributing to human-made disasters recognizably belong to the familiar body of slips, lapses and mistakes to which all of us are prone in the normal course of daily life. Any one of them might have had negligible consequences; but their effects accumulate, each compounding the mischief of its predecessors, so that in retrospect the whole series seems to move inexorably towards its calamitous conclusion. The aetiology of accidents is considered at length in Chapter 7.

  But hindsight, as Fischhoff (1975) has demonstrated, does not equal foresight. Simply knowing how past disasters happened does not, of itself, prevent future ones. However, by combining the knowledge obtained from case studies with a more adequate theory (or theories) of error production, we not only extend our knowledge of cognitive function, we can also begin to assemble a body of principles that, when applied to the design and operation of high-risk technological systems, could reasonably be expected to reduce either the occurrence of errors or their damaging consequences.

  9. Summary

  The more predictable varieties of human fallibility are rooted in the essentia
l and adaptive properties of human cognition. They are the penalties that must be paid for our remarkable ability to model the regularities of the world and then to use these stored representations to simplify complex information-handling tasks. They represent the debit side of the cognitive ‘balance sheet’, where each entry also carries significant advantages.

  Error is intimately bound up with the notion of intention. The term ‘error’ can only be meaningfully applied to planned actions that fail to achieve their desired consequences without the intervention of some chance or unforeseeable agency. Two basic error types were identified: slips (and lapses), where the actions do not go according to plan, and mistakes, where the plan itself is inadequate to achieve its objectives.

  Errors may be classified at any one of three levels: behavioural, contextual and conceptual. But only the last provides a satisfactory foundation, since there is no direct mapping of the surface forms of error onto their underlying cognitive mechanisms. The same mental process may produce quite different behavioural error types; conversely, a particular surface form (e.g., the omission of an intended act) may arise from a variety of underlying mechanisms.

  A distinction was made between error types and error forms. Error types (as will be discussed in Chapter 3) can be distinguished according to the performance levels at which they occur. On the other hand, error forms are evident at all levels of human performance and appear to originate in universal cognitive processes. In Chapters 4 and 5, we will identify these as the automatic retrieval mechanisms, similarity-matching and frequency-gambling, by which long-term memory locates and delivers its products, either to consciousness or as actions in the outside world.

  2 Studies of human error

  * * *

  This chapter examines some significant contributions to the study of human error. Its primary aim is to outline the major influences upon the arguments presented later in this book, particularly in Chapters 3 to 5. But I also hope that it will convey something of the current state of cognitive theorising to those readers who are not themselves working in this field.

  The first part of the chapter attempts to provide a modest historical vantage point from which to view contemporary treatments of error. It is necessarily selective because a complete account would encompass the entire history of psychological ideas. For our purposes, however, it is sufficient to consider the major influences of the past hundred years.

  The second part deals mainly with the resource-limited aspects of human cognition: attention and ‘primary’ (or short-term) memory. This work has largely been carried out in the laboratory within the natural science tradition of experimental psychology. As such, the material discussed in these sections bears the characteristic hallmarks of this mode of investigation: a restricted focus upon well-defined, manipulable phenomena; limited, data-bound models and an abiding concern with resolving differences between theoretical contenders on the basis of their predictive performance.

  The third part is concerned with the representation and deployment of stored knowledge and with the vast ‘community’ of specialist processors governing the more automatic aspects of perception, thought and action. Much of this work has been carried out within the more recent cognitive science tradition. This is too youthful and diverse an enterprise for there to be any universally accepted definition of its aims and methods. But most people who now style themselves as cognitive scientists (or cognitive engineers) would probably agree that if there is a common theme, then it has to do with devising, either in reality or in some notional form, information-processing machines that at least partly mimic the essential properties of human cognition.

  The final two parts set the scene for Chapters 3 to 5, which present this book’s particular view of human error mechanisms. One discusses the principal differences between ‘local’ and ‘framework’ models of cognition. The other outlines some basic assumptions regarding human cognition. These constitute the point of departure for the next three chapters.

  1. Early psychological observers of human error

  Since this is a psychological not a philosophical enquiry, I will focus upon those writers and early investigators who were directly concerned with the mental and behavioural aspects of error, specifically upon turn-of-the-century psychologists who sought to describe the variety of its forms and explain the processes underlying its production.

  1.1. Sully’s Illusions

  In 1881, James Sully, later Grote Professor of Mind and Logic at University College London, published a largely forgotten book entitled Illusions in which he undertook “a wide survey of the field of error, embracing in its view not only the illusions of sense dealt with in treatises on physiological optics, etc., but also other errors familiarly known as illusions, and resembling the former in their structure and mode of origin” (Sully, 1881, preface). These ‘other errors’ included systematic anomalies of memory, belief, thinking and insight.

  Sully defined ‘illusion’ (p. 6) as “any species of error which counterfeits the form of immediate, self-evident, or intuitive knowledge, whether as sense-perception or otherwise.” In the first instance, he sought to separate all knowledge into two regions: primary or intuitive knowledge, and secondary or inferential knowledge. Illusions are thus false or spurious knowledge of the first kind, and fallacies are false or spurious knowledge of the second kind. However, as he points out (p. 6), “the same error may be called a fallacy or an illusion, according as we are thinking of its original mode of production or of the form which it finally assumes; and a thorough-going psychological analysis of error may discover that these two classes are at bottom very similar.”

  Having construed illusion as the counterfeiting of immediate knowledge, Sully based his error classification upon the kind of knowledge that these illusions simulate. Using what were then fashionable philosophical terms, he distinguished two forms of immediate knowledge: presentative and representative. The former variety is obtained through direct perception of external objects or, through introspection, of inner states. Representative knowledge was obtained through the agencies of memory and belief (insofar as this simulates direct knowledge). These four modes of cognition—external perception, introspection, memory and belief—constituted the major dimension of Sully’s error taxonomy. A second dimension, active-passive, was employed to distinguish those errors arising out of spontaneous activity within the individual from those due primarily to external factors. However, along neither of these dimensions did Sully attempt to make rigid categorical distinctions.

  Sully’s treatment of perceptual and memory illusions provides an illustration of the way he sought common features among errors arising from different cognitive domains. Sully identified three major classes of memory illusions:

  (a) False recollections to which there correspond no real events or personal history.

  (b) Recollections that misrepresent the manner of the happening of real events.

  (c) Recollections that falsify the date of the events remembered.

  For each of these, there is a corresponding perceptual analogue: (a) perceptions for which there are no external counterparts (e.g., ocular spectra, sensations of light and hallucinations); (b) perceptions that distort the shape of the external object (e.g., the effects of haze and refracting media) and (c) perceptions that falsify size and distance (e.g., when clear air causes distant mountains to seem far closer than they are, or when intervening ‘clutter’ makes objects appear more distant).

  Sully was probably the first psychologist to attempt to classify the broad spectrum of human error and to seek common explanatory principles. His book, though little known, is a classic in the field of error studies. If its impact had been more widely felt, Charles Spearman, a subsequent Grote professor at University College London, would have had no cause to complain, nearly 50 years later, that: “Whoever would seek for any well-digested information about (errors) may make the whole round of current psychological textbooks ... for all this, he will become but little the
wiser for his pains” (Spearman, 1928, p. 30).

  Actually, Spearman’s complaint was not entirely justified. The 20 years or so straddling the turn of the century saw a number of significant attempts to understand the psychological origins of human fallibility. The most widely known of these was Freud’s investigation of the psychopathology of everyday life. No less important were the pioneering studies of speech errors conducted by Meringer in Vienna and the contributions of psychologists in the United States, notably William James and Hugo Munsterberg (a refugee from Wundt’s laboratory) at Harvard and the long-lived Joseph Jastrow at the University of Wisconsin.

  1.2. The Freudian slip

  Freud first became aware of the meaningfulness of certain everyday slips and lapses in 1896. In a letter to his colleague, Fliess, he wrote that he had “at last grasped a little thing I had long suspected”, namely, the reason why a known name sometimes eludes retrieval and wrong names are dredged up in its place. In this instance, he had been unable to recall the last name of the poet, Joseph Mosen: “I was able to prove (i) that I had repressed the name Mosen because of certain associations; (ii) that material from my infancy played a part in the repression; and (iii) that the substitute names that occurred to me arose, just like a symptom, from both recent and infantile groups of material” (Bonaparte, 1954).

  Having established the significance of this apparently trivial lapse, he began to collect examples of many different forms of everyday error: misreadings, slips of the tongue and pen, misquotations, bungled actions and so on. Gradually he became convinced that all of these normally inconsequential slips and lapses betrayed the presence within the unconscious of repressed impulses. Most of the time they remained hidden, but occasionally they seized upon an opportune moment—in speech, memory retrieval or action—to make themselves known through some error or ‘parapraxis’.