by Lee McIntyre
This idea has caused some push back over the years from both social scientists and philosophers of social science, who have held that one cannot hope to do social science in the same way as natural science. The subject matter is just too different. What we want to know about human behavior is often at odds with the desire merely to reduce actions to their causal forces. So if it were true that there is really only one way to do science—one that is defined by the unique methodology of natural science (if not “scientific method” itself)—it would be easy to see why some might lose hope in the idea that social science could become more scientific.
In earlier work, I spent a good deal of effort trying to identify flaws in those arguments that held there was a fundamental barrier to having a science of human action because of the complexity or openness of its subject matter, the inability to perform controlled social experiments, and the special problems created by subjectivity and free will in social inquiry.2 I continue to believe that my arguments here are valid, primarily because complexity and openness are also a part of natural scientific inquiry, and the other alleged barriers have less effect on the actual performance of social inquiry than their critics might assume. I also believe that these problems are overblown; if they were truly a barrier to inquiry they would likely show that much of natural science should not work either. But for the time being, I wish to focus on a more promising path for defending the possibility of a science of human behavior, for I now realize that what has been missing from the social sciences all these years is not proper method, but the right attitude toward empirical evidence.3
Like Popper, I have never believed that there is such a thing as scientific method, but I have held that what makes science special is its methodology.4 Popper, of course, famously held that the social sciences were not falsifiable, so they could not be scientific.5 I have argued against this and held that in every meaningful way there could be methodological parity between natural and social inquiry. That may be, but it overlooks one crucial point. What makes science special—both natural and social—is not just the way that scientists conduct their inquiry, but the attitude that informs their practices.
Too much social research is embarrassingly unrigorous. Not only are the methods sometimes poor, much more damning is the nonempirical attitude that lies behind them. Many allegedly scientific studies on immigration, guns, the death penalty, and other important social topics are infected by their investigator’s political or ideological views, so that it is all but expected that some researchers will discover results that are squarely in line with liberal political beliefs, while others will produce conservative results that are directly opposed to them. A good example here is the question of whether immigrants “pay their own way” or are a “net drag” on the American economy. If this is truly an empirical question (and I think that it is) then why can I cite five studies that show that immigrants are a net plus, five more that show that they are not, and can probably predict which studies came out of which research centers and by whom they were written?6 I am not here accusing anyone of fraud or pseudoscience. These are purportedly rigorous social scientific studies performed by well-respected scholars—it is just that their findings about factual matters flatly contradict one another. This would not be tolerated in physics, so why is it tolerated in sociology? Is it any wonder that politicians in Washington, DC, are so skeptical about basing their policies on social scientific work and instead cherry pick their favorite studies to support their preferred ideology?
The truth is that such questions are open to empirical study and it is possible for social science to study them scientifically. There are right and wrong answers to our questions about human behavior. Do humans experience a “backfire effect” when exposed to evidence that contradicts their opinion on an empirical (rather than a normative) question, such as whether there were weapons of mass destruction in Iraq or President George W. Bush proposed a complete ban on stem cell research? Is there such a thing as implicit bias, and if so, how can it be measured? Such questions can be, and have been, studied scientifically.7 Although social scientists may continue to disagree (and indeed, this is a healthy sign in ongoing research), their disagreements should focus on the best way to investigate these questions, not whether the answers produced are politically acceptable. Having the scientific attitude toward evidence is just as necessary in the study of human behavior as it is in the study of nature.
Yet there are numerous problems in contemporary social scientific research:
(1) Too much theory: A number of social scientific studies propose answers that have not been tested against evidence. The classic example here is neoclassical economics, where a number of simplifying assumptions—perfect rationality, perfect information—resulted in beautiful quantitative models that had little to do with actual human behavior.
(2) Lack of experimentation/data: Except for social psychology and the newly emerging field of behavioral economics, much of social science still does not rely on experimentation, even where it is possible. For example, it is sometimes offered as justification for putting sex offenders on a public database that doing so reduces the recidivism rate. This must be measured, though, against what the recidivism rate would have been absent the Sex Offender Registry Board (SORB), which is difficult to measure and has produced varying answers.8 This exacerbates the difficulty in (1), whereby favored theoretical explanations are accepted even when they have not been tested against any experimental evidence.
(3) Fuzzy concepts: In some social scientific studies, the results are questionable because of the use of “proxy” concepts for what one really wishes to measure. Recent examples include measuring “warmth” as a proxy for “trustworthiness” (see details in next section), which can lead to misleading conclusions.
(4) Ideological infection: This problem is rampant throughout the social sciences, especially on topics that are politically charged. One recent example is the bastardization of empirical work on the deterrence effect of capital punishment or the effectiveness of gun control on mitigating crime. If one knows in advance what one wants to find, one will likely find it.9
(5) Cherry picking: As we’ve seen, the use of statistics allows multiple “degrees of freedom” to scientific researchers, but this is the most likely to be abused. In studies on immigration, for instance, a great deal of the difference between them is a result of alternative ways of counting the “costs” incurred by immigration. This is obviously also related to (4) above. If we know our conclusion, we may shop for the data to support it.
(6) Lack of data sharing: As Trivers reports, there are numerous documented cases of researchers failing to share their data in psychological studies, despite a requirement from APA-sponsored journals to do so.10 When data were later analyzed, errors were found most commonly in the direction of the researcher’s hypothesis.
(7) Lack of replication: As previously discussed, psychology is undergoing a reproducibility crisis. One might validly argue that the initial finding that nearly two-thirds of psychology studies were irreproducible was overblown (see Gilbert et al. 2016), but it is nonetheless shocking that most studies are not even attempted to be replicated. This can lead to difficulties, where errors can sneak through.
(8) Questionable causation: It is gospel in statistical research that “correlation does not equal causation,” yet some social scientific studies continue to highlight provocative results of questionable value. One recent sociological study, for instance, found that matriculating at a selective college was correlated with parental visitation at art museums, without explicitly suggesting that this was likely an artifact of parental income.11
All of these problems can also be found to some degree in natural scientific work. Some of the other problems identified in earlier chapters (p-hacking, problems with peer review) can be present in social scientific work as well. The issue is not so much that these difficulties are unique to social inquiry, as that some of them ar
e especially prevalent there. Even if natural science suffers from them as well, the problems for social science are proportionally greater.
The problem with social science is not that it is failing to follow some prescribed method, or even to embrace certain scientific procedures, but that a number of its practices are not yet instantiated at the level of group practice so that they demonstrate a discipline-wide commitment to the scientific attitude. Fuzzy concepts or errors from questionable causation may not be such a big problem if one can count on one’s colleagues to catch them, but in many instances—in an environment in which data are not shared and replication is not the norm—too many errors slip by. Social science no less than natural science needs to embrace the scientific attitude toward evidence and realize that the only way to settle an empirical dispute is with empirical evidence. It should be recognized as the embarrassment that it is that there is so much opinion, intuition, theory, and ideology in social scientific research. Just as we now look back on bare-handed surgery and cringe, may we someday ask “Why didn’t someone test that hypothesis?” in social science? Nothing keeps people honest like public scrutiny. We need to do more data sharing and replication in the social sciences. We need better peer review and real scientific controls. And we need to recognize that it is shameful that up until recently much of social science has not even made an attempt to be experimental. Compared to the old neoclassical model in economics, the new behavioral one is a breath of fresh air. And this is all made possible by embracing the scientific attitude.
If social scientists were more committed at both the individual and group level to reliance on evidence and constructing better procedures for capitalizing on this, the social sciences would be better off. In this, they can follow the same path that was earlier laid out by medicine. Whether one is performing a clinical trial or doing fieldwork, the appropriate attitude to have in social inquiry should be that previously noted by Emile Durkheim: “When [we] penetrate the social world … [we] must be prepared for discoveries which will surprise and disturb [us].”12 We must abandon the belief that just because we are human we already basically understand how human behavior works. Where possible, we need to perform experiments that challenge our preconceptions, so that we can discover how human action actually works, rather than how our mathematical models and high theory tell us it should.
And all of this applies equally to qualitative work as it does to quantitative. While it is true that in social science there is some evidence that may be irreducibly qualitative (see Clifford Geertz’s work on “thick description” in his book The Interpretation of Cultures [Basic Books, 1973]), one must still be concerned with how to measure it. Indeed, in the case of qualitative work, we must be especially on guard against hubris and bias. Our intuitions about human nature are likely no more profound than eighteenth-century physicians’ were about infection.13 The data can and should surprise us. Just because a result “feels right” does not mean it is accurate. Cognitive bias and all of the other threats to good scientific work in the study of nature are no less a threat to those who study human behavior. The revolution in social science may be attitudinal rather than methodological, but this does not mean that it should not reach the four corners of how we have been doing our inquiry.
For years, many have thought that one could improve social science by making it more “objective.” The Logical Positivists in particular held fast to the fact–value distinction, which said that scientists should be concerned only with their results and not with how these might be used. But they were wrong. While it is true that we should not let our hopes, wishes, beliefs, and “values” color our inquiry into the “facts” about human behavior, this does not mean that values are unimportant. Indeed, as it turns out, our commitment to the scientific attitude is an essential value in conducting scientific inquiry. The key to having a more rigorous social science is not scientific method, but the scientific attitude.
A Way Forward: Emulating Medicine
If we think back to the state of medicine at the time of Semmelweis, the analogies with social science are compelling. Knowledge and procedures were based on folk wisdom, intuition, and custom. Experiments were few. When someone had a theory, it was thought to be enough to consider whether it “made sense,” even if there was no empirical evidence in its favor. Indeed, the very idea of attempting to gather evidence to test a theory flew in the face of belief that medical practitioners already knew what was behind most illnesses. Despite the shocking ignorance and backward practices of medicine throughout most of its history, theories were abundant and ideas were rarely challenged or put to the test. Indeed, this is what was so revolutionary about Semmelweis. He wanted to know whether his ideas held up by testing them in practice. He understood that knowledge accumulates as incorrect hypotheses are eliminated based on lack of fit with the evidence. Yet his approach was wholeheartedly resisted by virtually all of his colleagues.
Medicine at the time did not yet have the scientific attitude. Does social science now? In some cases it does, but the problem is that even in those instances where good work is being done many feel free to ignore it. In a society in which law enforcement continues to rely on eyewitness testimony and perform nonsequential criminal lineups, despite the alarmingly high rate of false positives with these methods, we have to wonder whether this is just another instance where practice trails theory.14 Public policy on crime, the death penalty, immigration, and gun control is rarely based on actual empirical study. Yet at least part of the problem also seems due to the inconsistent standards that have always dogged social scientific research. This has handicapped the reputation of the social sciences and made it difficult for good work to get noticed. As we have seen, when so many studies fail to be replicated, or draw different conclusions from the same set of facts, it does not instill confidence. Whether this is because of sloppy methodology, ideological infection, or other problems, the result is that even if there are right and wrong answers to many of our questions about human action, most social scientists are not yet in a position to find them. It is not that none of the work in social science is rigorous enough, but when policy makers (and sometimes even other researchers) are not sure which results are reliable, it drives down the status of the entire field.
Medicine was once held in similarly low repute, but it broke out of its prescientific “dark ages” because of individual breakthroughs that became the standard for group practice and some degree of standardization of what counted as evidence. To date, the social sciences have yet to complete their evidence-based revolution. We can find some examples today of the scientific attitude at work in social inquiry that have enjoyed some success, but there has not yet been a discipline-wide acceptance of the notion that the study of human behavior needs to be based on theories and explanations that are relentlessly tested against what we have learned through experiment and observation. As in prescientific medicine, too much of today’s social science relies on ideology, hunches, and intuition.
In the next section, I will provide an example of what a social science that fully embraced the scientific attitude might look like. Before we get there, however, it is important to consider one remaining issue that many have felt to be an insuperable barrier to the pursuit of a science of human behavior. Some have said that social science is unique because of the inherent problem of humans studying other humans—that our values will inevitably interfere with any “objective” empirical inquiry. This is the problem of subjectivity bias. Yet it is important to remember that in medicine we have an example of a field that has already solved this problem and moved forward as a science.
In its subject matter, medicine is in many ways like social science. We have irreducible values that will inevitably guide our inquiry: we value life over death, health over disease. We cannot even begin to embrace the “disinterested” pose of the scientist who does not care about his or her inquiry beyond finding the right answer. Medical scientists desperately hope that some theories will wor
k because lives hang in the balance. But how do they deal with this? Not by throwing up their hands and admitting defeat, but rather by relying on good scientific practices like randomized double-blind clinical trials, peer review, and disclosure of conflicts of interest. The placebo effect is real, for both patients and their doctors. If we want a medicine to work, we might subtly influence the patient to think that it does. But whom would this serve? When dealing with factual matters, medical researchers realize that influencing their results through their own expectations is nearly as bad as fudging them. So they guard against the hubris of thinking that they already know the answer by instituting methodological safeguards. They protect what they care about by recognizing the danger of bias.
The mere presence of values or caring about what you study does not undercut the possibility of science. We can still learn from experience, even if we are fully invested in hoping that one medicine will work or one theory is true, as long as we do not let this get in the way of good scientific practice. We can still have the scientific attitude, even in the presence of other values that may exist alongside it. Indeed, it is precisely because medical researchers and physicians recognize that they may be biased that they have instituted the sorts of practices that are consonant with the scientific attitude. They do not wish to stop caring about human life, they merely want to do better science so that they can promote health over disease. In fact, if we truly care about human outcomes, it is better to learn from experience, as the history of medicine so clearly demonstrates. It is only when we take steps to preserve our objectivity—instead of pretending that this is not necessary or that it is impossible—that we can do better science.
