Super Crunchers

by Ian Ayres

  Decline of physician status: Kevin Patterson, “What Doctors Don’t Know (Almost Everything),” N.Y. Times Magazine, May 5, 2002, p. 74.

  Along Came Polly: A trailer with quotation viewable at http://www. apple.com/trailers/universal/along_came_polly/medium.htm.

  Lies, damn lies: This phrase was popularized in the U.S. by Mark Twain, but is attributed to English statesman Benjamin Disraeli.

  Pain points: Harrah’s prediction of customer pain points was discussed in Chapter 1. See also Christopher Caggiano, “Show Me the Loyalty,” CMO Magazine, Oct. 2004.

  Virtual redlining in insurance?: Disclosure: I was a paid economics expert for minority plaintiffs in this case (as well as in other cases bringing similar claims). Owens v. Nationwide Mutual Ins. Co., No. Civ. 3:03-CV-1184-H, 2005 WL 1837959 (N.D. Tex. Aug. 2, 2005) (holding that even if its use of credit scores had a disparate impact on minorities, Nationwide had legitimate, race-neutral business reasons for using credit scores). See also Powell v. American General Finance, Inc., 310 F. Supp. 2d 481 (N.D.N.Y. 2004).

  Justice O’Connor urging “race-neutral means to increase minority participation”: Adarand Constructors, Inc. v. Pena, 515 U.S. 200, 238 (1995); City of Richmond v. J. A. Croson Co., 488 U.S. 469, 507 (1989).

  Veterans’ virtual records disappeared: David Stout and Tom Zeller, Jr., “Vast Data Cache About Veterans Has Been Stolen,” N.Y. Times, May 23, 2006, p. A1.

  Fidelity laptop stolen: Jennifer Levitz and John Hechinger, “Laptops Prove Weakest Link in Data Security,” Wall St. J., Mar. 24, 2006, p. B1.

  AOL user information released: Saul Hansell, “AOL Removes Search Data on Vast Group of Web Users,” N.Y. Times, Aug. 8, 2006, p. C4.

  Frost’s definition of home: Robert Frost, “The Death of the Hired Man,” in North of Boston (1914).

  The end of anonymity: Jed Rubenfeld, “Privacy’s End” (working paper October 2006).

  New facial recognition innovations: Anick Jesdanun, “Facial-ID Tech and Humans Seen as Key to Better Photo Search, But Privacy Concerns Raised,” Associated Press, Dec. 28, 2006.

  179: Gattaca’s vision of genetic predestination: The Science Show, http://www.abc.net.au/rn/scienceshow/stories/2001/262366.htm.

  Google’s mission statement: Google Corporate Information: Company Overview, http://www.google.com/corporate/.

  Americans don’t protect their privacy: Bob Sullivan, “Privacy Under Attack, But Does Anybody Care?” MSNBC, Oct. 17, 2006, http://www.msnbc. msn.com/id/15221095/.

  Mary Rosh criticizes Ayres and Donohue: Tim Lambert, Mary Rosh’s Blog, http://timlambert.org/2003/01/maryrosh/.

  Lott’s claims on guns and crime: John Lott, More Guns, Less Crime (2000).

  Ayres and Donohue response article to Lott: Ian Ayres and John J. Donohue III, “Shooting Down the ‘More Guns, Less Crime’ Hypothesis,” 55 Stanford L. Rev. 1193 (2003); Ian Ayres and John J. Donohue III, “The Latest Misfires in Support of the ‘More Guns, Less Crime’ Hypothesis,” 55 Stanford L. Rev. 1371 (2003).

  More questions about John Lott: Tim Lambert has been tireless in researching and pursuing several Lott-related questions. http://timlambert.org/lott/. For a good summary of the Mary Rosh controversy see Julian Sanchez, “The Mystery of Mary Rosh,” ReasonOnline, May 2003, http://www.reason.com/news/show/28771.htm.

  Senator Craig cites Lott: 146 Cong. Rec. S349 (daily ed. Feb. 7, 2000) (statement of Sen. Craig).

  Lott testifies: Lott has testified before state legislatures in Nebraska (1997), Michigan (1998), Minnesota (1999), Ohio (2002), Wisconsin (2002), Hawaii (2000), and Utah (1999). On May 27, 1999, Lott testified before the House Judiciary Committee that the stricter gun regulations proposed by President Clinton either would have no effect or would actually cost lives, and a number of Republican members of Congress have since included favorable references in their speeches to Lott’s work. Eighteen state attorneys general relied on “the empirical research of John Lott” in claiming: “There is an increasing amount of data available to support the claim that private gun ownership deters crime.” See 145 Cong. Rec. H8645 (daily ed. Sept. 24, 1999) (statement of Rep. Doolittle); Letter from Bill Pryor, Att’y General of Alabama, et al., to John Ashcroft, U.S. Att’y General, July 8, 2002, http://www.nraila.org/media/misc/pryorlet.pdf; Nat’l Rifle Ass’n, Inst. Leg. Action, Right to Carry Fact Sheet 2007, http://www.nraila.org/ Issues/factsheets/read.aspx?ID=18.

  Details on the Super Crunching analysis: Lott started with a very simple model that only let the law have a one-time impact on crime. There’s nothing wrong in starting with this specification. However, we found that, if you tested less constrained formulas that allowed the law to impact crime to varying degrees over time, his result often went away.

  Experts reject Lott’s model: Nat’l Acad. Sci., “Firearms and Violence: A Critical Review,” http://www.nap.edu/books/0309091241/html/.

  Lott sues Levitt: Lott v. Levitt, 2007 WL 92506, at *4 (N.D. Ill. Jan. 11, 2007).

  The offending paragraph from Freakonomics: Steven D. Levitt and Stephen J. Dubner, Freakonomics: A Rogue Economist Explores the Hidden Side of Everything (2005), pp. 134–35. Lott also claimed he was defamed by a private email that Levitt sent to economist John McCall. See Complaint at 7, Lott v. Levitt, No. 06C 2007 (N.D. Ill. Apr. 10, 2006). (“It was not a peer refereed edition of the Journal. For $15,000 he was able to buy an issue and put in only work that supported him. My best friend was the editor and was outraged the press let Lott do this.”). Lott maintains the special issue was peer-reviewed and that scholars with varying viewpoints were invited to contribute articles. As of this writing, the McCall email claim has not been dismissed.

  Levitt’s endnote: The endnotes also cited to one other article, Mark Duggan, “More Guns, More Crime,” 109 J. Polit. Econ. 1086 (2001).

  Devil’s advocacy: Benedict XIV, De Beat. et Canon. Sanctorum, I, xviii (On the Beatification and Canonization of Saints).

  Devil’s advocates in the boardroom: Barry Nalebuff and Ian Ayres, Why Not?: How to Use Everyday Ingenuity to Solve Problems Big and Small (2003), pp. 8–9.

  Heckman’s objections to randomized results: James Heckman et al., “Accounting for Dropouts in Evaluations of Social Programs,” 80 Rev. Econ. and Stat. 1 (1998); James J. Heckman and Jeffrey A. Smith, “Assessing the Case for Social Experiments,” 9 J. Econ. Perspectives 85 (1995).

  How healthy are low-fat diets?: Gina Kolata, “Maybe You’re Not What You Eat,” N.Y. Times, Feb. 14, 2006, p. F1; Gina Kolata, “Low-Fat Diet Does Not Cut Health Risks, Study Finds,” N.Y. Times, Feb. 8, 2006, p. A1.

  WHI study on low-fat diets: Women’s Health Initiative homepage: http://www.nhlbi.nih.gov/whi/. See also B. V. Howard et al., “Low-Fat Dietary Pattern and Weight Change Over 7 Years: The Women’s Health Initiative Dietary Modification Trial,” 295 JAMA 39 (2006); B. V. Howard et al., “Low-Fat Dietary Pattern and Risk of Cardiovascular Disease: The Women’s Health Initiative Randomized Controlled Dietary Modification Trial,” 295 JAMA 655(2006).

  Control vs. experimental low-fat diet groups: Ross L. Prentice et al., “Low-Fat Dietary Pattern and Risk of Invasive Breast Cancer,” 295 JAMA 629 (Feb. 8, 2006) (“Comparison group participants received a copy of Nutrition and Your Health: Dietary Guidelines for Americans and other health-related materials but were not asked to make dietary changes.”).

  The Rolls-Royce of studies: Gina Kolata, “Low-Fat Diet Does Not Cut Health Risks, Study Finds,” N.Y. Times, Feb. 8, 2006, p. A1.

  Calcium supplement study: R. D. Jackson et al., “Calcium Plus Vitamin D Supplementation and the Risk of Fractures,” 354 N. Engl. J. Med. 669(2006), erratum in 354 N. Engl. J. Med. 1102 (2006); 354 N. Engl. J. Med. 2285(2006); Gina Kolata, “Big Study Finds No Clear Benefit of Calcium Pills,” N.Y. Times, Feb. 16, 2006, p. A1.

  But is more information always better?: Indeed, I have also pointed out contexts—for example, with regard to campaign finance contributions—where less information could be valuable. Bruce Ackerman and Ian Ayres, Voting
with Dollars: A New Paradigm for Campaign Finance (2002).

  CHAPTER 8

  Hiking estimates: Ian Ayres, Antonia Ayres-Brown, and Henry Ayres-Brown, “Seeing Significance: Is the 95% Probability Range Easier to Perceive?” Chance Magazine (forthcoming 2007). It turns out that Sleeping Giant has been a wellspring of publication ideas. See also Ian Ayres and Barry Nalebuff, “Environmental Atonement,” Forbes, Dec. 25, 2006. (“Ian was hiking at Sleeping Giant Park outside New Haven. As he exited the car, a tissue fell out of his pocket. Ian stooped to pick up the litter, but a gust of wind blew it a few feet out of reach. He walked over to pick it up and again the tissue skittered beyond his grasp.”)

  Unlike the standard deviation, the variance is not our friend: The variance is the other traditional measure of dispersion. The two concepts are closely related. The variance is simply the standard deviation squared. But the variance is not your friend. It is not at all intuitive. You can see the difference if you just look at the units in which the two are expressed. If I tell you that the average kid in Anna’s class reads ten books in a month, you know what I mean. And very soon, you’ll understand what it means to say that the standard deviation is three books. However, we’ll all be long gone before we ever have an intuition for the fact that the variance is nine “squared books.” So if you’re ever told what some variance is, you should immediately turn it into a standard deviation (by taking its square root) and never, I repeat never, think about the variance again.

  Estimating a standard deviation from your intuitions (the answer): Once you have your range of heights, you’re ready to apply the Two Standard Deviation Rule in reverse. You see, because 95 percent of men fall within two standard deviations (both above and below) the average height, the distance between your upper and lower height represents four standard deviations. I have done this exercise dozens of time in class and most students say that 95 percent of men fall between 5'3" and 6'3", or 5'9" ±6 inches. The Two Standard Deviation Rule applied to this height range says that the standard deviation for male height is probably close to 3 inches. Of course this is just a rough estimate, but we should be pretty confident that the true standard deviation is not 5 inches or 1 inch.

  The man of the future is the man of statistics: Oliver Wendell Holmes, Jr., “The Path of the Law,” 10 Harv. L. Rev. 457 (1897).

  Point shaving in college basketball: Justin Wolfers, “Point Shaving: Corruption in NCAA Basketball,” 96 Am. Econ. Rev. 279 (2006); David Leonhardt, “Sad Suspicions About Scores in Basketball,” N.Y. Times, Mar. 8, 2006, p. C1; David Leonhardt, “The NCAA’s Response,” N.Y. Times, Mar. 7, 2006 (“a player on roughly one out of every five teams has direct knowledge of point shaving.”).

  Approximately normal distributions: Male height and IQ scores are close, but not perfectly normal because the mathematical normal distribution has an infinitesimal chance of having outcomes close to positive or negative infinity. And we know there are no such things as negative heights or IQ scores.

  The biggest problem with my knee-jerk application of the Two Standard Deviation Rule is that unlike the normal bell curve, many real-world distributions are skewed. Instead of being a symmetric bell shape on either side of the mean, a distribution will have a greater chance of taking on either a larger or a smaller number. The Two Standard Deviation Rule doesn’t work as well for skewed distributions. There may not be a 95 percent chance that a variable will be within two standard deviations. Instead, the most that statistics can tell us is that there is at least a 75 percent chance that a non-normal random variable will be within two standard deviations of its mean. Nevertheless, I still apply the 2SD rule as a starting point whenever I’m trying to figure out the intuitive variation in some process.

  Justin Wolfers, superstar: David Leonhardt, “The Future of Economics Isn’t So Dismal,” N.Y. Times, Jan. 10, 2007.

  Amar works backward: One of this country’s greatest scholars on the Constitution, Akhil Amar, used a similar method to back out an estimate of grade inflation. Yale doesn’t release its grade statistics, so Akhil couldn’t directly calculate the average GPA. However, it does release what grades a student needs in a particular year in order to qualify for the limited number of honors. The cutoff last year for magna cum laude (which is awarded to the top 15 percent of the class) was 3.82, while the cutoff for cum laude honors (which is awarded to the top 30 percent of students) was 3.72. Akhil realized that these two cutoffs were enough for him to estimate the average GPA. If he just assumed that the grade distribution at Yale is approximately normal, he could work backward from these two probabilities to estimate underlying mean and standard deviation. Here’s what he said in an email: “If only 30 percent of students have GPAs of 3.72 or higher (approx .5 standard deviation above the mean) and if only 15 percent have GPAs of 3.82 or higher (approximately one standard deviation above the mean), then simple algebra suggests that the standard deviation is roughly .2 and the mean is roughly 3.62.” [The “simple” algebra is that the two cutoffs for honors implicitly give us two equations with two unknowns. The two equations are: 5*sd+mean=3.72 and 1*sd+mean=3.82, where sd is the standard deviation and mean is the average GPA that we’re trying to solve for. By solving the first equation for sd, and then substituting that solution into the second equation (remember “substituting out” from seventh-grade algebra!), we are able to solve for the mean: mean=2*3.72-3.82=3.62.] Of course, this is just an approximation, because the grades may not in fact follow the normal distribution. Still, Akhil’s back-of-the-envelope estimate jibed pretty closely with the student newspaper’s own survey of seniors, which suggested that the median GPA was between 3.6 and 3.7. Kanya Balakrishna and Jessica Marsden, “Poll Suggests Grade Inflation,” Yale Daily News, Oct. 4, 2006, http://www.yaledailynews.com/articles/view/18226.

  The claim that women are “innately deficient”: Cornelia Dean, “Women in Science: The Battle Moves to the Trenches,” N.Y. Times, Dec. 19, 2006.

  The claim that women lack “intrinsic aptitude”: Sara Rimer and Alan Finder, “After 371 Years, Harvard Plans to Name First Female President,” N.Y. Times, Feb. 10, 2007.

  Summers’s controversial presidency: Newspaper reports have pointed to several other reasons why Summers may have felt pressure to resign. Slate columnist James Traub claims, “Summers was forced out of Harvard because he behaved so boorishly that he provided a bottomless supply of ammunition to his enemies.” James Traub, “School of Hard Knocks: What President Summers Never Learned About Harvard,” Slate, Feb. 22, 2006, http://www.slate.com/id/2136778/. Washington Post columnist Eugene Robinson said, “Summers is being forced to resign because, as brilliant as he is—and you don’t become a tenured Harvard professor at twenty-eight, as Summers did, unless you’re ridiculously brilliant—he proved to be a terrible politician.” Eugene Robinson, “The Subject Larry Summers Failed,” Wash. Post, Feb. 24, 2006. Even other parts of his speech about women in science rankled. For example, he proposed that another reason for the dearth of women in science was the relative unwillingness of women “to have a job that they think about eighty hours a week.” Summers also analogized the shortfall to a rather bizarre set of comparisons: “It is after all not the case that the role of women in science is the only example of a group that is significantly underrepresented in an important activity and whose underrepresentation contributes to a shortage of role models for others who are considering being in that group. To take a set of diverse examples, the data will, I am confident, reveal that Catholics are substantially underrepresented in investment banking, which is an enormously high-paying profession in our society; that white men are very substantially underrepresented in the National Basketball Association; and that Jews are very substantially underrepresented in farming and in agriculture.” Lawrence H. Summers, Remarks at NBER Conference on Diversifying the Science and Engineering Workforce, Jan. 14, 2005, http://www.president.harvard.edu/speeches/2005/ nber.htm.

  Studies finding sex differences in IQ standard deviations: Ian J. Deary et al
., “Population Sex Differences in IQ at Age 11: The Scottish Mental Survey 1932,” 31 Intelligence 533 (2003). See also Ian J. Deary et al., “Brother—Sister Differences in the G Factor in Intelligence: Analysis of Full, Opposite-Sex Siblings from the NLSY1979,” Intelligence (working paper, 2007).

  Possible flaws with Summers’s methodology: The difference in standard deviations may not be as great as 20 percent and the bell curves of intelligence may not follow a normal distribution—especially when you go so far into the tail.

  The Naegele rule: Janelle Durham, “Calculating Due Dates and the Impact of Mistaken Estimates of Gestational Age,” Jan. 2002, http://www.transitionto parenthood.com/ttp/birthed/duedatespaper.htm.

  Other methods of predicting due dates: R. Mittendorf et al., “The Length of Uncomplicated Human Gestation,” 341 N. Engl. J. Med. 461 (1999). For more on probabilities and predicting pregnancy terms, see W. Casscells et al., “Interpretation by Physicians of Clinical Laboratory Results,” 299 N. Engl. J. Med. 999 (1978); David M. Eddy and Jacquis Casher, “Usefully Interpreting the Triple Screen Assay to Detect Birth Defects,” working paper, Dept. of Statistics and of Biostatistics and Medical Informatics, Aug. 3, 2001; “Probabilistic Reasoning in Clinical Medicine: Problems and Opportunities,” in Judgment Under Uncertainty: Heuristics and Biases (D. Kahneman et al., eds., 1982); Gerd Gigerenzer, “Ecological Intelligence: An Adaptation for Frequencies,” in The Evolution of Mind (D. D. Cummins and C. Allen, eds., 1998), pp. 9–29; David J. Weiss, “You’re Not the Only One Who Is Confused About Probability…,” http://instructional1.calstatela.edu/dweiss/Psy302/Confusion.htm.

  Predicting contribution to the competitive bottom line: Alan Schwarz, “Game Theory Posits Measure of Baseball Players’ Value,” N.Y. Times, Nov. 7, 2004.