by Nate Silver
. . . and How He Beat the Odds
I met Bill James for the first time on a panel at the New Yorker Festival in October 2009. There was a fancy party afterward and he looked out of place among the fashionistas, wearing an exceedingly colorful sweater and a pair of scuff-marked clogs that looked like they were two sizes too big. While everyone else at the party was chasing down Susan Sarandon, we parked ourselves at the bar and chatted for a while.47
James’s responsibilities with the Red Sox are varied—and kept close to the vest. (He wasn’t able to go into much detail about them on the record.) But after having spent a quarter century writing on baseball as an outside agitator, he’s mellowed into late adulthood. The sport looks a little different to him now that he is on the inside; he is quicker to recognize the mental aspects of the game.
“There are a lot of things I wrote in the eighties that weren’t right,” he told me. “The big change was my having children. I know it’s a cliché, but once you have children you start to understand that everyone is somebody’s baby. It is an insiders-outsiders thing. You grow up and these people are characters on TV or video games or baseball cards—you don’t really think about the fact that these guys are humans and doing the best they can.”
I was struck by how similar many of James’s comments were to those of Beane and Sanders, even though they traditionally have approached the sport from very different angles. Indeed, if you put the transcripts of my conversations with James, Beane, and Sanders into a lineup, you would scarcely be able to pick them out (except that James’s would be much funnier). He’s come to recognize the value that the Red Sox scouts provide to the club and believes that it’s parallel to his own mission. In baseball, success is measured in a very particular way—by W’s and L’s—so it’s easy to keep everyone on the right track. If more information is making your predictions worse, you’ll be out of a job—not entitled to a lifetime appointment to The McLaughlin Group.
“On a certain level the way that I see baseball and the way the scouts see baseball is pretty similar,” James continued. “Maybe it’s one of those things where if you go far enough to the right and far enough to the left on the political spectrum you find people saying the exact same thing. What scouts are trying to see is really the same thing that I am trying to see.”
James was assisting with the Red Sox draft process in 2004 when they took Pedroia with the sixty-fifth pick. He had written a favorable report on Pedroia but recommended that they draft someone else. Nevertheless, he was pleased with the selection, and has enjoyed watching Pedroia make his critics look foolish.
There were a few moments early in his career, however, when even some of Pedroia’s biggest fans were starting to have their doubts. Pedroia was first called up to the big leagues in August 2006, playing in thirty-one games but compiling just a .198 batting average and just six extra-base hits. Nobody was too concerned about this; the Red Sox, unusually for them, were well out of playoff contention in the final weeks of the season, and New England’s attention was already turning to the Celtics and the Patriots. But the next year, entrusted with the full-time second base job, he started out slowly as well, his batting average just .172 after the first month of the season.
A team like the Cubs, who until recently were notorious for their haphazard decision-making process, might have cut Pedroia at this point. For many clubs, every action is met by an equal and opposite overreaction. The Red Sox, on the other hand, are disciplined by their more systematic approach. And when the Red Sox looked at Pedroia at that point in the season, James told me, they actually saw a lot to like. Pedroia was making plenty of contact with the baseball—it just hadn’t been falling for hits. The numbers, most likely, would start to trend his way.
“We all have moments of losing confidence in the data,” James told me. “You probably know this, but if you look back at the previous year, when Dustin hit .180 or something, if you go back and look at his swing-and-miss percentage, it was maybe about 8 percent, maybe 9 percent. It was the same during that period in the spring when he was struggling. It was always logically apparent—when you swing as hard as he does, there’s no way in the world that you can make that much contact and keep hitting .180.”
The Red Sox hadn’t taken their decision to draft Pedroia lightly. They were still observing him do the same things that had made him successful at every other amateur and professional level. If they were going to bench Pedroia, the decision would have to be just as carefully considered as the one to promote him in the first place. They did not let the data dictate their decision without placing it in a larger context.
Their only concern, James told me, was whether Pedroia would start to doubt himself. And another player might have—but not Pedroia, who suffers neither fools nor critics.
“Fortunately, Dustin is really cocky, because if he was the kind of person who was intimidated—if he had listened to those people—it would have ruined him. He didn’t listen to people. He continued to dig in and swing from his heels and eventually things turned around for him.”
Pedroia has what John Sanders calls a “major league memory”—which is to say a short one. He isn’t troubled by a slump, because he is damned sure that he’s playing the game the right way, and in the long run, that’s what matters. Indeed, he has very little tolerance for anything that distracts him from doing his job. This doesn’t make him the most generous human being, but it is exactly what he needs in order to play second base for the Boston Red Sox, and that’s the only thing that Pedroia cares about.
“Our weaknesses and our strengths are always very intimately connected,” James said. “Pedroia made strengths out of things that would be weaknesses for other players.”
The Real Lessons of Moneyball
“As Michael Lewis said, the debate is over,” Billy Beane declared when we were discussing Moneyball. For a time, Moneyball was very threatening to people in the game; it seemed to imply that their jobs and livelihoods were at stake. But the reckoning never came—scouts were never replaced by computers. In fact, the demand to know what the future holds for different types of baseball players—whether couched in terms of scouting reports or statistical systems like PECOTA—still greatly exceeds the supply. Millions of dollars—and the outcome of future World Series—are put at stake each time a team decides which player to draft, whom to trade for, how much they should pay for a free agent. Teams are increasingly using every tool at their disposal to make these decisions. The information revolution has lived up to its billing in baseball, even though it has been a letdown in so many other fields, because of the sport’s unique combination of rapidly developing technology, well-aligned incentives, tough competition, and rich data.
This isn’t necessarily making life easier for Beane, who expressed concern that the other teams have copied the A’s best tricks. Very few teams, for instance, now fail to understand the importance of OBP or neglect the role played by defense—and what hasn’t changed is that those teams still have a lot more money than the A’s.
In the most competitive industries, like sports, the best forecasters must constantly innovate. It’s easy to adopt a goal of “exploit market inefficiencies.” But that doesn’t really give you a plan for how to find them and then determine whether they represent fresh dawns or false leads. It’s hard to have an idea that nobody else has thought of. It’s even harder to have a good idea—and when you do, it will soon be duplicated.
That is why this book shies away from promoting quick-fix solutions that imply you can just go about your business in a slightly different way and outpredict the competition. Good innovators typically think very big and they think very small. New ideas are sometimes found in the most granular details of a problem where few others bother to look. And they are sometimes found when you are doing your most abstract and philosophical thinking, considering why the world is the way that it is and whether there might be an alternative to the dominant paradigm. Rarely can they be found in the temperate latit
udes between these two spaces, where we spend 99 percent of our lives. The categorizations and approximations we make in the normal course of our lives are usually good enough to get by, but sometimes we let information that might give us a competitive advantage slip through the cracks.
The key is to develop tools and habits so that you are more often looking for ideas and information in the right places—and in honing the skills required to harness them into W’s and L’s once you’ve found them.
It’s hard work. But baseball will remain an unusually fertile proving ground for innovators. There hasn’t been a really groundbreaking forecasting system since PECOTA’s debut ten years ago. But someone will come along and take advantage of Pitch f/x data in a smart way, or will figure out how to fuse quantitative and qualitative evaluations of player performance. All this will happen, and sooner rather than later—possibly in the time that this book is at the printer.
“The people who are coming into the game, the creativity, the intelligence—it’s unparalleled right now,” Beane told me. “In ten years if I applied for this job I wouldn’t even get an interview.”
Moneyball is dead; long live Moneyball.
4
FOR YEARS YOU’VE BEEN TELLING US THAT RAIN IS GREEN
On Tuesday, August 23, 2005, an Air Force reconnaissance plane picked up signs of a disturbance over the Bahamas.1 There were “several small vortices,” it reported, spirals of wind rotating in a counterclockwise motion from east to west—away from the expanse of the Atlantic and toward the United States. This disruption in wind patterns was hard to detect from clouds or from satellite data, but cargo ships were beginning to recognize it. The National Hurricane Center thought there was enough evidence to characterize the disturbance as a tropical cyclone, labeling it Tropical Depression Twelve. It was a “tricky” storm that might develop into something more serious or might just as easily dissipate; about half of all tropical depressions in the Atlantic Basin eventually become hurricanes.2
The depression strengthened quickly, however, and by Wednesday afternoon one of the Hurricane Center’s computer models was already predicting a double landfall in the United States—a first one over southern Florida and a second that might “[take] the cyclone to New Orleans.”3 The storm had gathered enough strength to become a hurricane and it was given a name, Katrina.4
Katrina’s first landfall—it passed just north of Miami and then zoomed through the Florida Everglades a few hours later as a Category 1 hurricane—had not been prolonged enough to threaten many lives. But it had also not been long enough to take much energy out of the storm. Instead, Katrina was gaining strength in the warm waters of the Gulf of Mexico. In the wee hours of Saturday morning the forecast really took a turn for the worse: Katrina had become a Category 3 hurricane, on its way to being a Category 5. And its forecast track had gradually been moving westward, away from the Florida Panhandle and toward Mississippi and Louisiana. The computer models were now in agreement: the storm seemed bound for New Orleans.5
“I think I had five congressional hearings after Katrina.” said Max Mayfield, who was director of the National Hurricane Center at the time the storm hit, when I asked him to recall when he first recognized the full magnitude of the threat. “One of them asked me when I first became concerned with New Orleans. I said ‘Sixty years ago.’”
A direct strike of a major hurricane on New Orleans had long been every weather forecaster’s worst nightmare. The city presented a perfect set of circumstances that might contribute to the death and destruction there. On the one hand there was its geography: New Orleans does not border the Gulf of Mexico as much as sink into it. Much of the population lived below sea level and was counting on protection from an outmoded system of levees and a set of natural barriers that had literally been washing away to sea.6 On the other hand there was its culture. New Orleans does many things well, but there are two things that it proudly refuses to do. New Orleans does not move quickly, and New Orleans does not place much faith in authority. If it did those things, New Orleans would not really be New Orleans. It would also have been much better prepared to deal with Katrina, since those are the exact two things you need to do when a hurricane threatens to strike.
The National Hurricane Center nailed its forecast of Katrina; it anticipated a potential hit on the city almost five days before the levees were breached, and concluded that some version of the nightmare scenario was probable more than forty-eight hours away. Twenty or thirty years ago, this much advance warning would almost certainly not have been possible, and fewer people would have been evacuated. The Hurricane Center’s forecast, and the steady advances made in weather forecasting over the past few decades, undoubtedly saved many lives.
Not everyone listened to the forecast, however. About 80,000 New Orleanians7—almost a fifth of the city’s population at the time—failed to evacuate the city, and 1,600 of them died. Surveys of the survivors found that about two-thirds of them did not think the storm would be as bad as it was.8 Others had been confused by a bungled evacuation order; the city’s mayor, Ray Nagin, waited almost twenty-four hours to call for a mandatory evacuation, despite pleas from Mayfield and from other public officials. Still other residents—impoverished, elderly, or disconnected from the news—could not have fled even if they had wanted to.
Weather forecasting is one of the success stories in this book, a case of man and machine joining forces to understand and sometimes anticipate the complexities of nature. That we can sometimes predict nature’s course, however, does not mean we can alter it. Nor does a forecast do much good if there is no one willing to listen to it. The story of Katrina is one of human ingenuity and human error.
The Weather of Supercomputers
The supercomputer labs at the National Center for Atmospheric Research (NCAR) in Boulder, Colorado, literally produce their own weather. They are hot: the 77 trillion calculations that the IBM Bluefire supercomputer makes every second generate a substantial amount of radiant energy. They are windy: all that heat must be cooled, lest the nation’s ability to forecast its weather be placed into jeopardy, and so a series of high-pressure fans blast oxygen on the computers at all times. And they are noisy: the fans are loud enough that hearing protection is standard operating equipment.
The Bluefire is divided into eleven cabinets, each about eight feet tall and two feet wide with a bright green racing stripe running down the side. From the back, they look about how you might expect a supercomputer to look: a mass of crossed cables and blinking blue lights feeding into the machine’s brain stem. From the front, they are about the size and shape of a portable toilet, complete with what appears to be a door with a silver handle.
“They look a little bit like Porta-Potties,” I tell Dr. Richard Loft, the director of technology development for NCAR, who oversees the supercomputer lab.
Those in the meteorology business are used to being the butt of jokes. Larry David, in the show Curb Your Enthusiasm, posits that meterologists sometimes predict rain when there won’t be just so they can get a head start on everyone else at the golf course.9 Political commercials use weather metaphors as a basis to attack their opponents,10 usually to suggest that they are always flip-flopping on the issues. Most people assume that weather forecasters just aren’t very good at what they do.
Indeed, it was tempting to look at the rows of whirring computers and wonder if this was all an exercise in futility: All this to forecast the weather? And they still can’t tell us whether it’s going to rain tomorrow?
Loft did not look amused. Improved computing power has not really improved earthquake or economic forecasts in any obvious way. But meteorology is a field in which there has been considerable, even remarkable, progress. The power of Loft’s supercomputers is a big part of the reason why.
A Very Brief History of Weather Forecasting
“Allow me to deviate from the normal flight plan,” Loft said back in his office. He proved to have a sense of humor after all—quirky and offbea
t, like a more self-aware version of Dwight Schrute from The Office.* From the very beginnings of history, Loft explained, man has tried to predict his environment. “You go back to Chaco Canyon or Stonehenge and people realized they could predict the shortest day of the year and the longest day of the year. That the moon moved in predictable ways. But there are things an ancient man couldn’t predict. Ambush from some kind of animal. A flash flood or a thunderstorm.”
Today we might take it for granted that we can predict where a hurricane will hit days in advance, but meteorology was very late to develop into a successful science. For centuries, progress was halting. The Babylonians, who were advanced astronomers, produced weather prognostications that have been preserved on stone tablets for more than 6,000 years.11 Ultimately, however, they deferred to their rain god Ningirsu. Aristotle wrote a treatise on meteorology12 and had a few solid intuitions, but all in all it was one of his feebler attempts. It’s only been in the past fifty years or so, as computer power has improved, that any real progress has been made.
You might not think of the weather report as an exercise in metaphysics, but the very idea of predicting the weather evokes age-old debates about predestination and free will. “Is everything written, or do we write it ourselves?” Loft asked. “This has been a basic problem for human beings. And there really were two lines of thought.
“One comes through Saint Augustine and Calvinism,” he continued, describing people who believed in predestination. Under this philosophy, humans might have the ability to predict the course they would follow. But there was nothing they could do to alter it. Everything was carried out in accordance with God’s plan. “This is against the Jesuits and Thomas Aquinas who said we actually have free will. This question is about whether the world is predictable or unpredictable.”
