by Nate Silver
Sanders’s background—going from perhaps the best amateur athlete in the state of Nebraska to a footnote in the Baseball Encyclopedia—gives him unique insight into the psyches of young players. I caught up with him by cell phone one morning while he was driving from North Carolina to Georgia to watch the Braves’ top affiliate play.
At the 2003 Winter Meetings in New Orleans, I would have pegged Sanders as one of the “jocks.” He devoted his life to the sport after his (brief) playing career ended. But Sanders has never seen the game through the jocks-versus-nerds prism.
“I love to evaluate,” he told me. “I’ve always enjoyed statistical proofs even way back in the day when we did it with calculators and adding machines.” Sanders relayed an anecdote: “One of the scouts said, ‘Well, let’s face it, guys, what’s the first thing we do when we go to the ballpark? We go to the press room, we get the stats. We get the stats! What’s wrong with that? That’s what you do.’ ”
Statistics, indeed, have been a part of the fabric of baseball since the very beginning. The first newspaper box score, which included five categories of statistics for each player—runs, hits, putouts, assists, and errors—was published by Henry Chadwick in 1859,38 twelve years before the first professional league was established, in 1871. Many of the Moneyball-era debates concerned not whether statistics should be used, but which ones should be taken into account. On-base percentage (OBP), for instance, as analysts like James had been pointing out for years, is more highly correlated with scoring runs (and winning games) than batting average, a finding which long went underappreciated by traditionalists within the industry.39
This type of debate was usually fought on the statheads’ turf. That OBP conveys more useful information than batting average, or that a pitcher’s ERA is a fairer indicator of his performance than his win-loss record are scientific facts, as much as the earth’s revolution around the sun; the statheads were unambiguously right about them. But being on the winning side of these arguments may have led the stathead community to be complacent or dismissive about other points on which there is much more ambiguity.
The further you get away from the majors—the more you are trying to predict a player’s performance instead of measure it—the less useful statistics are. Statistics at the more advanced minor-league levels, like Double-A and Triple-A, have been shown to be almost as predictive as major-league numbers. But statistics at the lower minor-league levels are less reliable, and the numbers for college or high school players have very little predictive power.
The scouts’ traditional alternative to statistics are the Five Tools: hitting for power, hitting for average, speed, arm strength, and defensive range. It is a list that has drawn, and deserves, much criticism. Plate discipline—which consists of drawing walks and avoiding strikeouts—is not represented. And the invocation of the Five Tools sometimes conveys the impression that they are all equally important, when in fact hitting for power is far more important than arm strength at all positions save for shortstop and catcher.
There is also reason to think that the Five Tools alone won’t tell us very much. As a player works his way up the minor-league ladder, the tools should increasingly be reflected in the player’s statistics—or he probably isn’t bound to advance any higher. Some of the categories, in fact, essentially are statistics: “hitting for average” is expressed by a player’s batting average; “hitting for power” by doubles and home runs. If a scout tells you that a certain player grades out at a seventy on the eighty-point scouting scale in the hitting-for-power category, but he’s struggling to hit ten home runs per year for the Altoona Curve, how much faith would you really put in the scouting report?
Sanders, the industry veteran, is skeptical of the emphasis placed on the Five Tools. “The impact toolbox is obvious to anyone. Runs fast, throws hard, all that stuff. Scouts can walk in and see those immediately,” he told me. “I think the question is—are those skills used effectively to [make for] winning ballplayers? Are those tools converted into usable skills? Bat speed—we can see that fairly quickly. But if the person has bat speed but he doesn’t trust it—if he’s always jumping out at pitches—that’s not usable.”
Sanders’s focus is less on physical tools and more on usable, game-ready skills. The extent to which one can be translated to the other depends on what he calls a player’s mental toolbox. The mental tools are often slower to develop than the physical ones; Sanders’s wife is a special-needs educator and pointed him toward research suggesting that most of us are still in a state of mental adolescence until about the age of twenty-four.40 Before that age, Sanders will cut a player some slack if he sees signs that their mental tools are developing. After that, he needs to see performance. Interestingly, twenty-four is right about the age when a player is usually in Double-A and his performance starts to become more predictable from his statistics.
Sanders has no formal definition of what a player’s mental toolbox should include, but over the course of our conversation, I identified five different intellectual and psychological abilities that he believes help to predict success at the major-league level.
Preparedness and Work Ethic Baseball is unlike almost all other professional sports in that games are played six or seven times a week. A baseball player can’t get “amped up” for game day as a football or basketball player might; he has to be ready to perform at a professional level every day. This means he must have a certain amount of discipline. Sanders likes to get to the ballpark early because he contends he can better detect this attribute in a player’s pregame rituals than during the game itself. Pedroia, for instance, was clearly more focused than his teammates during infield practice that September night at Fenway Park. He had his routine, and he wasn’t putting up with any distractions—and certainly not some reporter he’d never heard of trying to interview him.
Concentration and Focus Although related to preparedness, this category specifically concerns the manner in which a player conducts himself during the course of the game. Baseball is a reflex sport. A hitter has about three tenths of a second to decide whether to swing at a pitch;41 an infielder has to react to a sharply hit grounder as soon as it comes off the bat. “If a player is not energized, I don’t know what we can do with them,” Sanders says. “I want a shortstop, a middle infielder, to have pitch-by-pitch focus that’s off the charts.”
Competitiveness and Self-Confidence While it may seem like a given that any professional athlete would be a natural-born competitor, baseball players must overcome self-doubt and other psychological obstacles in the early stages of their careers. One moment, they were the king of the hill in high school; the next, they are riding busses between Kannapolis and Greensboro, reading about their failures on the Internet each time they go into a slump. When Sanders sees a talented player underachieving, he wonders, “Is there a desire to succeed to the degree that there’s a failure mechanism kicking in? Is there a fear of failure? Is the desire to succeed significant enough to overcome the fear of failure?”
Stress Management and Humility In baseball, even the best hitters fail a majority of the time, and every player will enter a slump at certain points during the season. The ability to cope with this failure requires a short memory and a certain sense of humor. One of Sanders’s favorite scouting tactics is to observe how a player reacts after a rough or unlucky play. “I like to see a hitter, when he flails at a pitch, when he takes a big swing and to the fans it looks ridiculous, I like to look down and see a smile on his face. And then the next time—bam—four hundred feet!” These skills will only become more critical once a player reaches the majors and has to deal with scrutiny from fans and the media.
Adaptiveness and Learning Ability How successfully is the player able to process new information during a game? Listen to advice from his coaches? How does he adapt when his life situation changes? What if he’s traded—or asked to play a new position? The path between amateur ball and the major leagues is rarely linear even for the most talen
ted prospects—and so a great player can’t be too rigid in his mental approach. “Players who are successful at this game are people who, when they walk down the hall of the building and there’s a turn, they make a round turn. They don’t make a sharp turn,” Sanders observes. “It’s a controlled intensity.”
These same habits, of course, are important in many human endeavors. Some of them even have applications for forecasters, especially the one Sanders calls adaptiveness: How do you react to new information when it comes your way? Being too hot under the collar and overreacting to a change in the circumstances or being too cool to change your mind when the evidence dictates otherwise will lead to bad predictions.
Few professions, however, are as competitive as baseball. Among the thousands of professional baseball players, and the hundreds of thousands of amateurs, only 750 are able to play in the major leagues at any given time, and only a few dozen of those will be All-Stars. Sanders’s job is to search for those exceptional individuals who defy the odds. He has to work nearly as hard at his job as the players do, and he is still out on the road almost every day in his late sixties.
But Sanders provides the Dodgers with the most valuable kind of information—the kind of information that other people don’t have.
Information Is the Name of the Game
Billy Beane, the protagonist of Moneyball, sees relentless information gathering as the secret to good scouting.
“What defines a good scout? Finding out information that other people can’t,” he told me. “Getting to know the kid. Getting to know the family. There’s just some things that you have to find out in person.”
Beane should know. Much of the success of the A’s was a result of the team’s statistical aptitude. But their scouting of amateur players was just as critical to their achievements. Most of the team’s stars during the early 2000s period that Moneyball chronicled—Miguel Tejada, Jason Giambi, Barry Zito, Tim Hudson, Eric Chavez—had been signed and developed by the club.
Beane told me the A’s scouting budget is now much higher than it has ever been. Moreover, he said it was the A’s fascination with statistical analysis that led them to increase it. As we’ve seen, baseball players do not become free agents until after six full seasons, which is usually not until they’re at least thirty. As Bill James’s analysis of the aging curve revealed, this often leads clubs to overspend on free agents—after all, their best years are usually behind them. But there is a flip side to this: before a player is thirty, he can provide tremendous value to his club. Moreover, baseball’s economics are structured such that younger players can often be had for pennies on the dollar.42
If a baseball team is viewed, as with any other business, from a standpoint of profits and losses, almost all the value is created by the scouting and development process. If a team’s forecasting system is exceptionally good, perhaps it can pay $10 million a year for a player whose real value is $12 million. But if its scouting is really good, it might be paying the same player just $400,000. That is how you compete in a small market like Oakland.
So the A’s have no lack of respect for the role of scouting; quite the contrary. Nor, Beane made clear, do they shy away from looking at a player’s mental makeup when deciding which ones to bring into their organization.
The organization still very much believes in rigorous analysis. The rigor and discipline is applied, however, in the way the organization processes the information it collects, and not in declaring certain types of information off-limits.
“The proportion of objective versus subjective analysis is weighted more in some organizations than in others,” he explained. “From our standpoint in Oakland, we’re sort of forced into making objective decisions versus gut-feel decisions. If we in Oakland happen to be right on a gut-feel decision one time, my guess is it would be random. And we’re not in a position to be making random decisions and hope we get lucky. If we’re playing blackjack, and the dealer’s showing a four and we have a six, hitting on the sixteen just doesn’t make sense for us.”
The key to making a good forecast, as we observed in chapter 2, is not in limiting yourself to quantitative information. Rather, it’s having a good process for weighing the information appropriately. This is the essence of Beane’s philosophy: collect as much information as possible, but then be as rigorous and disciplined as possible when analyzing it.
The litmus test for whether you are a competent forecaster is if more information makes your predictions better. If you’re screwing it up, you have some bad habits and attitudes, like Phil Tetlock’s political pundits did. If Prospect A is hitting .300 with twenty home runs and works at a soup kitchen during his off days, and Prospect B is hitting .300 with twenty home runs but hits up nightclubs and snorts coke during his free time, there is probably no way to quantify this distinction. But you’d sure as hell want to take it into account.
Many times, in fact, it is possible to translate qualitative information into quantitative information.* Scouts actually do rate players using a hard numerical scale, which ranges from 20 to 80 in each category. There’s no reason you couldn’t place that into a statistical model alongside a player’s batting average43 and see where it adds value; some teams, like the Cardinals, already attempt versions of this.
Indeed, the line between stats and scouting, and qualitative and quantitative information, has become very blurry in the baseball industry. Take, for example, the introduction of Pitch f/x, a system of three-dimensional cameras that have now been installed at every major-league stadium. Pitch f/x can measure not just how fast a pitch travels—that has been possible for years with radar guns—but how much it moves, horizontally and vertically, before reaching the plate. We can now say statistically, for instance, that Zack Greinke, a young pitcher with the Milwaukee Brewers who won the 2009 Cy Young Award as his league’s best pitcher, has baseball’s best slider,44 or that Mariano Rivera’s cut fastball is really as good as reputed.45 Traditionally, these things were considered to be in the domain of scouting; now they’re another variable that can be placed into a projection system.
We’re not far from a point where we might have a complete three-dimensional recording of everything that takes place on a baseball field. We’ll soon be able to measure exactly how good a jump Jacoby Ellsbury gets on a fly ball hit over his head. We’ll know exactly how fast Ichiro Suzuki rounds the bases, or exactly how quickly Yadier Molina gets the ball down to second base when he’s trying to throw out an opposing base-stealer.
This new technology will not kill scouting any more than Moneyball did, but it may change its emphasis toward the things that are even harder to quantify and where the information is more exclusive, like a player’s mental tools. Smart scouts like Sanders are already ahead of the curve.
Why Pedroia Was Predicted to Fail . . .
But why were the scouts so wrong about Dustin Pedroia?
All the scouts were in agreement on the basic facts about him. Everyone knew that Pedroia was a very good hitter for average, that he had a smart approach at the plate, and that his mental toolbox was “off the charts.” Everyone knew that he had a long swing; that his defense was steady but unspectacular; that his foot speed was no better than average; that he was short and did not have a terrific physique.
It was an idiosyncratic profile for a young player, however, and a lot of scouts didn’t know what to make of it. “When you draw up a player, scouts have a feel for what they want to see,” Sanders told me. “Prototypical standards. Dustin went against the grain in some of those areas, starting with his size.”
When we can’t fit a square peg into a round hole, we’ll usually blame the peg—when sometimes it’s the rigidity of our thinking that accounts for our failure to accommodate it. Our first instinct is to place information into categories—usually a relatively small number of categories since they’ll be easier to keep track of. (Think of how the Census Bureau classifies people from hundreds of ethnic groups into just six racial categories or how thous
ands of artists are placed into a taxonomy of a few musical genres.)
This might work well enough most of the time. But when we have trouble categorizing something, we’ll often overlook it or misjudge it. This is one of the reasons that Beane avoids what he calls “gut-feel” decisions. If he relies too heavily on his first impressions, he’ll let potentially valuable prospects slip through the cracks—and he can’t afford that with a payroll like Oakland’s.
A system like PECOTA, which searches through thousands of players to find the ones with similar profiles, has a more rigorous way to categorize players. It could place Pedroia’s skills more within their proper context.
PECOTA’s search found some favorable precedents. Pedroia’s short stature, for example, may actually have been an advantage given the rest of his skills. In baseball, the strike zone is defined as running from a player’s shoulders to his knees. The smaller the athlete, the smaller the target the pitcher gets to throw at. A player like Pedroia with a good batting eye can take especial advantage of this.
Meanwhile, being lower to the ground can be an asset to a second baseman’s defense. It’s a position that relies on agility, having catlike reflexes to ground balls that come hot off the bat. Many of the best second basemen in baseball history have been very short. Of the seventeen in the Hall of Fame, only two—Nap Lajoie and Ryne Sandberg—were over six feet tall.46 Joe Morgan, perhaps the greatest second baseman of all time, was just five seven.
Scouts are very good at what they do, but this is a case where they had categorized a player too quickly and too prejudicially. Pedroia’s diminutive stature was in some ways a strength.
Still, there were no guarantees: PECOTA had not seen Pedroia’s success as certain, just that the odds were in his favor. The scouts saw the odds as weighted against him. What made the difference is that the Red Sox believed in Dustin Pedroia. And fortunately for the Red Sox, Pedroia believed in Pedroia, too.
