Scientifical Americans

by Sharon A. Hill

  Sham Inquiry

  ARIGs see bits of evidence differently than others would as they employ a particular framework to interpret it. Pieces of evidence should all come together independently to tell a coherent story that does not fly in the face of what we already know to be true. The evidence should form an explanatory picture with the pieces mutually reinforcing each other (Haack 2003). The paranormal worldview results in evidence that appears to fit a framework that reasonably supports the preferred paranormal cause. This framework provides a pre-existing shape like a wire frame. It skews the investigation and conclusions resulting in a non-scientific and disputable result. The process of inquiry that is undertaken (perhaps unconsciously) results in gathering only “evidence” that can be fit to this particular frame or preferred narrative. Non-conforming evidence is ignored or explained away. The script has already been written, the destination set, so the process of “investigation” is fail-proof. It’s sham inquiry. The sham inquirer knows just the right kind of evidence he seeks. Seek, and he finds.

  If we put weak or misleading evidence in, we must consider what comes out. Garbage in, garbage out. Volume of evidence matters far less than quality. A few carefully done studies are preferred to dozens of low-quality studies that may all be wrong. My favorite analogy for paranormal evidence was made by Benjamin Radford who said that you can’t make a strong cup of coffee by pouring together many weak cups.6 And you can’t make a strong case with a lot of weak evidence. Context is also important: observations or recordings out of context are worse than useless, they are misleading.

  Carl Sagan, writing on UFOs, recognized the lure of fringe ideas. They are charming, would be delightful if true, and carry deep emotional significance to us and our place in nature. Like Harry Houdini who exposed psychics or James Randi who exposed faith healers, Sagan was aware of how strong emotion can overcome our senses and allow us to be deceived. We believe weak evidence simply because we want it to be true or we reject it when we don’t want it to be true. Sagan made an effort to pause and examine before believing or rejecting. We must always strive to do the same (Sagan 1972). His maxim that extraordinary claims require extraordinary evidence still holds.

  ARIGs collect a lot of weak evidence and it may also be pulled from and applied outside a reasonable context (in a paranormal framework). The evidence goes unscrutinized, and is scientifically wasted, lacking coherence with any reasonable overarching model of what could be going on. A paranormal interpretation goes against the rest of reliable knowledge we have, making it highly implausible to be true. The information may be narrowly presented to appear to converge to a paranormal entity, but in the bigger picture of nature, it does not. Ideally, all the evidence ARIGs produce should be critically evaluated on merit with poor stuff tossed and the remainder used to form an independent hypothesis and to generate testable or answerable questions. If a large animal exists here, then I should find certain things in the environment that correspond to that conclusion.

  If this is a haunting, then I should find consistent results between investigators at the same place. ARIGs do not formulate testable hypotheses very often because it’s not their primary goal, which is, instead, to prove the paranormal. Therefore, this work isn’t sound research.

  Most ARIGs are not at all aware they are taking this route of sham inquiry. Instead, they assume they follow a systematic approach that will get them to a solid conclusion. They often demonstrated that they believe they are using the best science and reason they can. The process feels satisfying to them and they get rewarded with their expected result. Going through the motions but not the intellectual effort of inquiry scuttles any chance to get a robust result. When they find the evidence that fits their anticipated result, they stop, and inquiry stops too. At that point, acceptance of a preconceived, probably paranormal, explanation is assured and hard to budge. Prior commitment to a specific belief is a block to even considering other evidence that may conflict with that belief.

  The ARIG will work to maintain the conclusion in the face of critical questioning. They are not open to having that belief challenged and will typically be hostile to skeptical processes or persons whose role it is to do that. In this behavior, ARIGs exhibit tribalism—adherence to the values and practices of their community. Scientists and other communities all fall into tribalist tendencies (McRae 2012). But tribalism can get out of hand, causing the tribe to lose touch with objectivity, ignore important elements, and become hypocritical.

  This behavior manifests in social media interactions among ARIGs and between them and skeptical commentators. Tribalism is another hurdle in moving towards shared knowledge.

  Even though it feels satisfying to make a conclusion that this or that anomaly is attributable to a paranormal cause, that’s not a valid investigation. If failure to meet a reasonable standard of investigation is deliberate or unintentional, it is intellectually dishonest if it’s presented as a fair inquiry into the issue. After decades of research by amateur investigators who seek ghosts, UFOs, and cryptids, we observe that they have not improved their methods to the point where better evidence is obtained. But that doesn’t mean good research can’t be done by ARIGs. A scientific background is not necessary to increase the quality of such investigations. Acceptance of some basic guidelines and goals of critical thinking is required for ARIGs and interested individuals to make progress and contribute to understanding seemingly paranormal experiences.

  Rational Paranormal Investigation: The Alternative to Sham Inquiry

  A great deal more effort and time is required to do rigorous, meaningful research and investigation into ARIG subject areas. The goal of improving research methods and conclusions may not be suitable for many ARIGs who prefer to continue to do what they do now. There is no real incentive to do anything different, as debunkers are not rewarded nearly to the degree that the paranormal view is. The media, seeking drama and controversy, latches on to false positives. ARIG clients may wish to validate their belief instead of gain a reasonable explanation. ARIG participants may be drawing rewards from his or her serious leisure activities and belief in paranormal ideas. Popular culture will elevate the “better story” for its needs. A cultural shift towards heightened value for critical thinking is required before we see due respect for more thoughtful investigation and credible expertise.

  If ARIGs do wish to tighten their methods and limit biases that prevent sound research results, if they wish to do rational inquiry instead of sham inquiry, the following sections are a guide to a revised approach. If information is collected carefully and then shared, considering the core scientific norms, amateur non-scientists can contribute significantly to the body of reliable knowledge. As with citizen science projects, evidence and data can be presented to credentialed scientists who use it to move along the path to produce a scientific conclusion. Scientists better serve the public with regards to these processes (Sykes 2016) since they can provide the time, funds, and expertise to conduct large studies and statistical evaluations—things ARIGs can not do but that are necessary for scientific rigor.

  Loxton and Prothero’s Abominable Science (2013) provides some goals for cryptozoologists to improve on their research. The tenets apply to all paranormal investigation and are as follows:

  Rethink fundamental assumptions. As described thoroughly in this volume, ARIGs often come into an investigation with preconceived notions and strong beliefs in an entity. This short-circuits the investigation process immediately.

  Test the null hypothesis. Is there enough evidence to overthrow conventional explanations in exchange for the alternative? Is there anything to explain?

  Meet the burden of proof. Strong, positive evidence is required when making a claim that could more likely be a hoax or a misinterpretation.

  Collect high-quality data. Credible physical evidence and logical analysis is needed. More stories and eyewitness tales aren’t going to cut it (the weak coffee analogy).

  Publish high-quality reports and w
ork. Proper documentation is needed for the wider public to access and judge the claims and conclusions and to build on that work.

  Be open to criticism. Peer review is a necessary component of science. It is a harsh process, no one likes to be criticized, but it must be undertaken to gain credibility and fix errors. Ignoring skeptical literature and reasonable criticism reflects poorly on fields that strive to be taken seriously.

  Be skeptical of your own data. ARIGs do not recognize how easily they can be fooled and have regularly been duped by those with alternative agendas. They must learn to harshly judge the quality of their own evidence and concede alternatives and flaws. In the end, the ideas must form a solid theory or model that explains and predicts. Facts must fit into that model and the resulting explanation needs to made make sense without requiring baseless assumptions. Research must fill in the gaps and strengthen knowledge, not aim to upset the entire table.

  Finally, we all would benefit from accepting uncertainty. To invent answers is to create misinformation. It’s really OK to say “I don’t know” but let’s continue to look for the best answer.

  Reassessing Goals

  Generalization as to what ARIGs discover is difficult because each person, each group, and each community is seeking their own understanding as it applies to them. Sunk costs (the time and money invested into a cause) make it difficult to walk away from the search. An alternative to doing things the way they have been done for a decade or more is to reassess the goals, and reboot or retool the effort in a way that makes the work still personally fulfilling for ARIG participants but provides useful knowledge to the rest of society.

  Careful research, record keeping, thorough documentation, and archiving should be goals of ARIGs who say they are trying to contribute to science. A reliable, high-quality effort in cataloging reports of paranormal claims would provide a means to classify a phenomenon, spot patterns and similarities, and perhaps formulate predictions that can be tested. Such a collection would be valuable for social and historical context alone.

  Just about anyone can be trained to think more critically and carefully. I offer to consult ARIGs gratis to help them sharpen up their science understanding and improve their methods. I have had two participants in this program. My method involves a first step to establish clear goals and expectations, and plan a path towards achieving the goals. I provide them a series of questions to answer in a group discussion. This first step is daunting and they invariably give up prior to fulfilling it. I suspect they reject the step because such personal reflection is difficult and they realize that doing so would mean going down an uncomfortable path, so they say “forget it” and continue with their current ways. But, I contend that any successful project requires preparation, otherwise, the effort is just a lark and it will not work out well. For ARIGs to honestly assess their reasons for doing this and recognize their inherent biases, an outside motivator is needed or internally-motivated reasoning will quickly take the group off this tough course. The following discussion questions are provided to ARIGs and they are asked to write the answers or otherwise record their discussions for me.

  1. What is your goal on each investigation? (Be very basic, short and to the point.)

  2. Do you consider yourselves scientific in your approach? What does that specifically mean to you? Or if not, what approach do you utilize—spiritual, metaphysical, etc.?

  3. Describe your typical group members or people who are most involved with investigations. Do any have scientific training?

  4. What is your understanding of each piece of equipment you use (including personal feelings you record during each investigation)? What purpose do they serve and what is your basis for using it?

  5. Describe your best evidence and why you think it is convincing. Should I be convinced? Why?

  Framework for Paranormal Investigation

  The five questions in the previous section are meant to make the ideals and values of the ARIG transparent so all participants understand. To break from the parade of cookie-cutter investigators who unfortunately don’t produce any research and conclusion of substance, methods and attitudes must change. I provide a new framework for ARIGs to use that is superior to the TV-based sham inquiry method. As part of this process, I recommend ARIGs study the skeptical literature. (For a good start, I recommend Benjamin Radford’s Scientific Paranormal Investigation (2010) to contrast the characteristics of a solid investigation with a made-for-TV one.)

  It is imperative that any investigator understand these basic concepts:

  • Chance, randomness, and correlation vs. causation

  • Characteristics of scientific research

  • How we fool ourselves (bias, perception errors, and unreliability of memory and anecdotes)

  • Critical thinking processes

  Establishing a clear goal for each event (investigation, ghost hunting trip) will influence the way the investigation plays out and whether it is ultimately satisfying, frustrating, or a waste of time. (Is the goal to have fun? To enhance your belief? To find out if a claim is true? Each of these will require a different approach.) The goal also will illuminate what constitutes a useful result.

  Think about the practical utility of this investigation and whom it benefits. Is the aim to get to the best answer about a claim? Will others who were not part of the investigation find the conclusion satisfactory and useful? What would be the objections from critics?

  Establish the claim clearly and narrow down the questions you are asking about it. Make sure there is something described that you can examine for an explanation. Seek to illuminate the issue with verifiable facts and explanatory details. Stick closely to this claim for investigation instead of looking for ways to enhance it.

  Decide if you are using the right tools. What’s the intended use of each tool? Does this match the use for which it was designed? If not, what is your justification for using it in another way? Can you explain what data it gives you and what that means? Find references for these answers. If they are paranormal-themed references only, find objective technical references for the devices or claims about how they work. If you can’t explain and defend why you are using a tool so that others outside of the paranormal field will understand and except it, ditch it and go back to basic observations, note-taking, and fact-gathering. These exercises in thinking through what you plan to do and why are key to effective investigation, but hardly any ARIGs seem to do this.

  With regards to process of an investigation, the following considerations should be emphasized:

  • Work on establishing the claim exactly through interviews, research, observations.

  • Do proper scholarship. Get facts and quotes right. Seek primary sources. Corroborate facts with evidence. Trust but verify.

  • Utilize critical thinking, evidence analysis, logic, and established scientific methodologies. Identify assumptions that, if false, make the mystery disappear. (The witness may be hoaxing/lying, or they may make a mistake in facts or observation.)

  • Look for negative evidence. What is missing that should be there? (Did the dog not bark? Are there no body or physical traces of a reported animal?)

  • Eliminate natural explanations as best you can. Every place will have some unexplained activity because we are missing information. It is impossible know every possible explanation. Therefore, the logical solution to an unexplained case is “I don’t know.” Use this when necessary.

  • Consider ethical concerns. Never go beyond your limits (people with psychological or health issues, children). This is harmful, morally questionable and legally treacherous. It’s unethical to suggest causes that are implausible and promote your personal belief (demons, angels) and to spread misinformation or fear.

  • Do not assume the existence or behavior of something without evidence to back it up. Get independent confirmation of facts, suggest more than one possible hypothesis and examine each. Beware of lumping observations together, or corr
elating them to a single cause (haunting). Don’t rely on any explanation by default (such as “I can’t explain it, therefore it’s paranormal.”).

  Avoid these mistakes:

  • Assuming all methods are valid. Even though other groups may use a technique, it may be nonsense. Some equipment and processes are bogus, leading to useless evidence.

  • Considering “feelings” as evidence. A person is a subjective “tool.” Our senses are frequently unreliable. Recognize that contagion may be at play—if you say you feel cold or ill, others may be sensitive to suggestion.

  • Failing to consider alternatives adequately. Look for natural explanations. Also, consider: Why does it have to be a ghost? Can it just as likely be aliens or humans from another dimension?

  • Avoid unproven paranormal theories and hypotheses. No one person can reasonably conclude something is “supernatural” because comprehensive testing and systematic reduction of all possibilities is untenable.

  • Counting on orbs, EVPs and any video or audio as best evidence. There are innumerable other possibilities for explanation including equipment glitches.

  Write up your results completely:

  • A complete record of all details is required.

  • Include all carefully recorded notes.

  • Avoid making assumptions, just rely on the facts.

  • Analyze the findings, doing follow-up research if necessary.

  • Use references and cite them.

  • Make the report available to discuss with others. Request a review and critique. To make the strongest report, seek out a skeptical reviewer.