Burned

by Edward Humes

  The owner had a criminal record, bad debts, and had lost his job.

  He had taken out fire insurance on the house.

  Furniture had been taken out of the house just before the fire.

  The family pet had been taken from the house before the fire.

  The owner was home at the time of the fire, but his car was not in its customary place in the driveway or garage.

  Many of the family clothes were missing from the closets rather than lying burned inside the ruin of the house.

  The owner of the house seemed overly distraught.

  The insurance investigator and police detectives saw in this information an obvious case of a homeowner setting a fire to resolve a financial crisis, with the arsonist taking pains to preserve prized possessions just before setting the fire. This was grounds to deny an insurance payout and to issue an arrest warrant, they decided.

  But before an arrest could be made, Reuben conducted the actual origin and cause investigation, relying on fire science instead of the list of red flags. Burn patterns led her to a possible area of origin, and once that was pinpointed, she found the fire had started with a short circuit hidden inside a wall, caused when a nail used to attach exterior shingles had pierced internal wiring. The fire was accidental, she concluded, and all those red flags turned out to be red herrings.

  In the end, a closer look at the supposedly suspicious behavior led to benign explanations: The bad credit was from a mortgage payment that had been just a few days late. The owner had been laid off temporarily, but had been told by his employer he’d be rehired shortly. The couch had been put in the garage because the new family dog had been chewing it. The dog was missing because he had been dropped at the veterinarian. The clothes were gone because they were being laundered in advance of a family vacation. And the car was gone from its customary place because the owner’s wife had taken it and their kids to visit their grandparents.

  “Unlike burn patterns, red flags are indicators,” she wrote, “. . . and like wind socks and weather vanes, can blow this way or that.”

  Bieber’s proposed solution to such biased investigations, as simple as it is despised, is for fire investigators—at least those conducting a science-based examination of a fire scene—to work “in the blind” as Reuben did. To not look for or become aware of such “red flags.” They should not consider or even know about other reasons to suspect that a crime had been committed until after the origin and cause investigation is complete. Anything short of that allows bias to influence the outcome.

  In his report on the Parks case, Ed Nordskog dismisses cognitive bias as “the latest negative buzz phrase used when nothing else can be proven . . . akin to calling someone ‘racist’ and then putting the burden on them to prove otherwise.” He derides the idea of making arson investigators work in the blind as ridiculous and impractical. “Only someone who has no idea how fires are actually investigated would say that,” Nordskog says.

  “Just consider the history,” counters Bieber. “It’s what the scientific method requires.”

  * * *

  • • •

  The notion of blind investigations in general is neither new nor outlandish. The first documented example of a blind scientific investigation was a successful probe of fraudulent claims commissioned by King Louis XVI of France in 1784. Louis wanted to verify or disprove the miracle-cure claims of one of the most famous physicians of his era, Franz Mesmer, who had captivated such luminaries of the day as Wolfgang Amadeus Mozart and the last queen of France, Louis’s wife, Marie Antoinette. Now regarded as the father of hypnosis, from whom the term mesmerize originates, Mesmer claimed he could use magnets and other techniques to manipulate invisible vital fluids inside all living creatures in order to cure physical and mental ailments. He called this property of living things “animal magnetism,” and Mesmer claimed his mastery of it had, among other feats, cured a well-known musician’s hysterical blindness. What Mesmer was actually doing was using music, lighting, and spoken cadences to place individuals and groups into a hypnotic trance, then using hypnotic suggestion to effect cures.

  The king appointed four members of the French Academy of Sciences to investigate Mesmer, led by the US ambassador to France at the time, America’s sardonic and clever founding father, raconteur, and inventor, Benjamin Franklin. Among the commissioners was Dr. Joseph Guillotin, inventor of the world’s most famous beheading device, which would be used after the Revolution to execute Louis, Marie Antoinette, and, ironically, the two other members of the Mesmer commission.

  The commission asked Mesmer’s trainees to don blindfolds, then attempt to distinguish objects, including a tree and a flask, that supposedly had been filled with vital fluids. They failed to do so. The commissioners also tested subjects who believed they had been cured of various ailments, whether or not they had actually received the animal magnetism treatments. The commission considered the resulting cures as a form of self-delusion or imagination, identifying what would later come to be known as the placebo effect.

  The debunking of animal magnetism as a cure or a boon to sound minds and bodies brought an end to Mesmer’s star turns with the royalty and Europe’s wealthy elite, although his practices remained in vogue for the next century.

  A few years later, blind testing was used in Britain to ascertain the effects of nitrous oxide—laughing gas—on humans (with subjects unaware whether they were breathing different concentrations of the gas or just plain air), and to compare the sound quality of a Stradivarius violin to a new guitar-like instrument crafted by a naval engineer, François Chanot. The committee, listening from another room, chose the new violin design.

  In more modern applications, blind auditions are used by most symphony orchestras, with prospective players hidden behind screens. The probability of women landing a chair in America’s symphony orchestras rose by 50 percent since the practice of blind auditions became commonplace, according to a 2001 gender bias study by researchers at Princeton and Harvard universities. Cognitive bias has been recognized as a factor even in pop culture—the television show The Voice has its judges sit in chairs facing away from singing contestants while judging their performances. Testing of new medicines is routinely done in “double blind” experiments, in which one group gets the real drug and one group gets a placebo, and neither the test subjects nor those administering the drugs know which is which.

  The goal of blind testing is to avoid introducing information that could bias the test results. Seeing the legendary Stradivarius (or the marine scientist’s equally distinct rival creation) while comparing their sound would be an obvious way of biasing judges of violin quality. Less obvious is the possible biasing of drug testing that can occur when lab workers know they are giving out placebos, which creates the risk of purposely, accidentally, or unconsciously providing cues to test subjects that they are not receiving the actual medicine, as well as the risk of data and observations being misreported.

  For the same reasons, the criminal justice system is also beginning to recognize the value of blind practices in non-forensic practices: Double-blind methods for securing eyewitness identification of suspects are being adopted by some police departments. The most common characteristic in cases where DNA evidence overturns wrongful convictions are faulty eyewitness identifications (a factor in seven out of ten convictions of the innocent). Multiple studies across decades have shown eyewitness identifications have an error rate of at least one out of five in cases where witnesses are asked to identify strangers in traditional police lineups. In many of the wrongful convictions cases, initially uncertain witnesses were cajoled, coerced, or given subtle, even unconscious cues by police detectives, which led the witnesses to identify the suspects that investigators preferred, rather than relying strictly on memory. As time passed and through repeated retellings, witnesses would also begin to express more confidence in thei
r recollections, though most researchers have found memories become less, not more, accurate over time. In some cases, eyewitnesses are led to alter their descriptions of suspects to better match the detectives’ preferred suspect—though this also can be an unintended and subconscious process rather than deliberate corruption. Double-blind procedures seek to reduce bad eyewitness identifications by keeping both the witnesses and those administering the lineup from knowing which person on display is the actual suspect. The San Diego police, for example, have begun showing witnesses randomly generated photos on a computer screen that the police officers in the room cannot even see. Then the potential witness is asked to select the person most resembling the culprit. In contrast, the Los Angeles Police Department uses the venerable “six-pack” method of lineup identification—a group of six photos chosen by investigating officers and shown to eyewitnesses with the investigators present. LA police agencies have had to pay out several multimillion-dollar lawsuit verdicts for wrongful conviction based on bad eyewitness identifications in recent years, but have resisted changing their methods.

  * * *

  • • •

  The way forensic science is deployed by law enforcement seems at times to be designed to introduce bias into the process. It is both permissible and common for a police investigator or prosecutor to introduce a case to a forensic analyst by explaining who a suspect is, what crimes are alleged, and what results are needed to “make” the case—be it a fingerprint or DNA or fiber or ballistics comparison. In other words, a forensics specialist often is told what the “right” answer is before the testing even begins, before they compare that smudgy partial fingerprint to the “right” suspect’s fingerprint card, or before an arson investigator peers at a single burn pattern in a house where foul play is suspected before the fire scene is fully examined.

  In his testimony in the Parks case, Paul Bieber said he first became aware of the role cognitive bias can play in the criminal justice system through a case he analyzed for the Northern California Innocence Project. George Souliotes of Modesto, California, had been convicted of setting a fire in 1997 that burned down a home he owned and rented out. A mother and her two children died in the fire.

  Investigators assigned to the fire began by assembling information that seemed suspicious to them, but had nothing to do with fire science: a for-sale sign on the lawn; Souliotes’s desire to evict his tenants; erroneous information that the house was going into foreclosure; and questionable eyewitness testimony from a distraught woman who thought she saw a vehicle like Souliotes’s near the house at the time of the fire. This information suggested a financial motive, hostility toward the victims, and presence at the scene of the crime. The investigators suspected arson before any detailed investigation of the fire scene. Once inside, they quickly concluded burn patterns and melted metal door thresholds proved the use of gasoline or other flammable liquids to make the house burn hotter and faster than a normal fire. These conclusions were based on ideas that had been discredited as junk science by 1997, Bieber testified, “yet there was a tremendous amount of certainty.”

  It took sixteen years to finally free Souliotes from prison at age seventy-two. The judge who overturned Souliotes’s conviction singled out “tunnel vision” and confirmation bias as key elements behind the wrongful conviction.

  “I thought at the time that this must be a unique circumstance,” Bieber told Judge Ryan on day one of the Parks hearing. “But I learned that [this] is actually a common occurrence.”

  He added that the sorts of errors he saw in the Souliotes case and in the Parks investigation were “not an example of poor investigator integrity, or that they are not honest about their conclusions. They believe they are correct.”

  But cognitive bias, he added, led investigators astray in both cases.

  In a perfect world, he argued, the fire science experts searching for clues to how a fire started could be protected from such errors—as could the suspects their work implicates—if they were kept away from the nonscientific detective work, and were kept in the dark about the results of the larger criminal investigation. Noting that ten pages of a twelve-page fire investigation report in the Souliotes case detailed non-fire-related information, Bieber said the problems in that case could have been avoided by working in the blind. The presidential commission on forensic science agreed with this idea, Bieber said, finding that only “task relevant” information should be given to the forensic examiners to shield them from possible bias. Knowing about a planned eviction and foreclosure at the scene of a fire may be important information for a prosecutor or jury to consider in a criminal trial, but it adds nothing to the scientific analysis of burn patterns. The patterns do not change when the investigator has such extraneous information in hand, Bieber argues, but the investigator is another story—he and his perceptions can be changed and unconsciously biased by that information.

  When it was time for the district attorney’s office to cross-examine Bieber, the prosecutor’s goal was to show the judge that Bieber’s ideas were extreme, far outside the norms most others in the arson investigation business endorsed or accepted. The twenty-year-veteran deputy district attorney given this task, Sean Carney, Los Angeles’s most experienced arson prosecutor, started by asking Bieber how he conducted a fire investigation. The defense expert gave him the expected answer: He limited himself to the origin and cause determination, and avoided biasing and extraneous information. He said he might find, for example, that a fire’s ignition source was an overheated extension cord that caused flammable materials in a nearby trash can to catch fire. The fire then spread to other fuels in the room, then throughout the house.

  And then, Carney asked, you would classify that short circuit as either deliberate and incendiary, or accidental, right?

  Wrong, replied Bieber. “Others can make that determination and classification.”

  Classification of a fire is typically the last step in a fire investigation and is different from the cause determination. There are currently four possible classifications: incendiary (arson), accidental, natural causes, or undetermined. But Bieber was saying he avoided making such a call in his cases.

  Carney adopted a look of incredulity and asked, “What’s the point of that?”

  Because that conclusion, Bieber said, is not scientific. The lead detective in any criminal investigation, who is also looking at possible motives, the behavior of the homeowner, and other non-scientific information, can put all that together and decide between arson and accident, Bieber suggested. That way the origin and cause inquiry remains pure and unbiased, arrived at through the scientific method.

  A verbal tug-of-war ensued, a tensely polite struggle in which each man sought to portray the other as unreasonable. Carney tried several different hypotheticals intended to show that Bieber’s reasoning led to absurd limits on fire investigators. But Bieber remained adamant, soft-spoken on the stand—unlike his more forceful and salty personality outside the courtroom. The stalemate soon drew the judge in to ask his own questions. It would be the first of many times Judge Ryan would take the wheel away from one or another of the attorneys, because he was either curious or impatient, trying to cut to the chase in one question instead of the six queries the attorneys might ask.

  “Wouldn’t the cause necessarily require you to determine whether it was accidental, incendiary, or natural?” Ryan asked.

  “No, sir. The cause is defined . . . as the identification of a physical ignition source. . . . Not whether the circumstances were intentional or not.” The first part was a scientific finding, he said. The rest was outside the realm of science.

  When the judge expressed continued puzzlement, Bieber added, “It’s kind of like saying to a fingerprint examiner, ‘Why aren’t you figuring out whether the finger was placed there intentionally or accidentally?’ Well, because it is outside the domain of a fingerprint examination.”

  The judge nodd
ed. The nuances were clear now, he said: The origin and cause investigation looks at the fire scene only and seeks to explain what happened to start and spread a fire. The classification looks at the totality of circumstances, scientific and nonscientific, and draws a conclusion.

  This was intolerable to Carney because if the judge accepted Bieber’s view that his was the best possible way of investigating a fire fairly, it meant that Jo Ann Parks had come that much closer to freedom. The Parks team wanted the judge to view the lead arson investigator in the original trial as having gone too far, mingling science with gumshoe detective work, inviting bias and violating the scientific method along the way. So Carney tried another hypothetical: What if you investigate a fire and find clear proof it was caused by a short circuit in a malfunctioning electrical appliance. “Wouldn’t it follow that that would be an accident?”

  Bieber would not concede. “Yes, it would, but the conclusion that it was an accidental cause would not be a scientific, technical, or engineering process. It would be the application of common sense to the totality of the circumstances. That’s not the role of a forensic examiner.”

  “But that’s commonly done in the fire investigation industry?” a frustrated Carney responded.

  “Yes, sir. Just to be clear, it is common for fire investigators to take that additional step and classify the cause of the fire,” Bieber said in a tone that made it clear he believed that most fire investigators were wrong to do so. “The person who is impacted by bias is unaware of it.”

  It was downhill from there. Carney could not shake or rattle Bieber, even when he launched a line of questioning that suggested the National Fire Protection Association had failed to employ the scientific method when it insisted fire investigators use the scientific method. Bieber just shook his head and said he was confused by the notion that asking investigators to follow the scientific method could be unscientific. Judge Ryan started reading paperwork, looking heavy-lidded.