FIGURE 21
The bottom line on this chart shows the surplus value. The thing to notice is that this curve is sloping upward throughout the first round. What this means is that the early picks are actually worth less than the later picks. But remember, the Chart says that early picks are worth a lot more than later picks! Figure 22 shows both curves on the same chart and measured in comparable units, with the vertical axis representing value relative to the first pick, which is given a value of 1.
FIGURE 22
If this market were efficient, then the two curves would be identical. The draft-pick value curve would be an accurate forecast of the surplus that would accrue to the team from using that pick; i.e., the first pick would have the highest surplus, the second pick the second-highest surplus, etc. That is hardly the case. The trade market curve (and the Chart) says you can trade the first pick for five early second-round picks, but we are finding that each of those second-round picks yields more surplus to the team than the first-round pick they are together traded for! In all my years of studying market efficiency, this is the most blatant violation I have ever seen.
We made another interesting discovery about the market for picks. Sometimes teams will trade a pick in this year’s draft for a pick next year. What is the exchange rate for such trades? Even a casual look at the data reveals that a simple rule of thumb is used for such trades: a pick in a given round this year fetches a pick one round earlier the following year. Give up a third-round pick this year and you can get a second-round pick next year. (Detailed analyses confirm that trades closely follow this rule.) This rule of thumb does not sound unreasonable on the surface, but we found that it implies that teams are discounting the future at 136% per year! Talk about being present-biased! You can borrow at better rates from a loan shark. Not surprisingly, smart teams have figured this out and are happy to give up a pick this year to get a higher-round pick the following year.‡
So our research yielded two simple pieces of advice to teams. First, trade down. Trade away high first-round picks for additional picks later in the draft, especially second-round picks. Second, be a draft-pick banker. Lend picks this year for better picks next year.
Before discussing the significance of our findings, especially the advice to trade down, it is important to rule out a few potential explanations that will occur to many readers, especially those who think like economists.
Can teams make so much money from jersey sales bearing the player’s name that they can still find it profitable to draft a high-profile player, even if he does not become a star? No. The teams share all sales of team jerseys and other official NFL products equally.
Can drafting a high-profile player sell enough tickets to make it worthwhile, even if he does not become a star? No. First of all, most NFL teams have waiting lists to buy season tickets. But more to the point, no one comes to watch a bad player even if he is famous. To thoroughly investigate this possibility, we redid our analysis using only offensive linemen, the largely anonymous behemoths who try to protect the quarterback from mountain-size defensive players who want to tackle him. Although only the most dedicated fans would be able to name many of the players on the offensive line for their favorite team, our analysis looks the same, so “star appeal” cannot be the missing factor that explains the anomaly.
Could the chance of getting a real superstar make the gamble worthwhile? No. We did one simple analysis to show this. The primary implication of our analysis is that teams with early picks should trade down, that is, trade their early pick for multiple later picks. To test the validity of this strategy, we evaluated every two-for-one trade that would be possible using the Chart as a guideline. For example, the Chart indicates that the team with the first pick could trade it for picks seven and eight, four and twelve, two and fifty, and so forth. For each of these potential hypothetical trades, we looked to see how the team did by using two measures of player performance: games started and years elected as an all-star. We found that trading down yielded a large increase in games started with no sacrifice in the number of all-star seasons.
How could the decision-makers in the league get this so wrong? Why didn’t market forces drive the price of draft picks toward the surplus value they provide to the team? The answer provides a good illustration of the limits to arbitrage concept that was so important to understanding financial markets. Suppose a team reads and understands our paper, what could they do? If they are a good team that is usually near the top of the standings, there is not much they can do to take advantage of the market inefficiency, aside from being willing to lend this year’s picks for better ones the following year. Since there is no way to sell a high draft pick short, there is no arbitrage opportunity for a smart team, much less for outside investors. The best one can hope to do is to buy a bad team, and at least for a while, improve their drafting strategy by trading down.
Before we even had our first draft of this paper, we had some interest from one of the NFL teams, and by now we have now worked informally with three teams (one at a time, of course). The first interaction we had was with Daniel Snyder, the owner of the Washington Redskins. Mr. Snyder had been invited by the entrepreneurship club at the Booth School of Business to give a talk, and one of the organizers asked me to moderate a discussion for the audience. I agreed, knowing I would have some time to talk to Snyder one-on-one during lunch.
Dan Snyder is a self-made man. He dropped out of college to start a company that chartered jets to sell cheap spring break vacation trips to university students. He later went into the direct mail advertising business and had the good fortune or wisdom to sell the company in 2000, at the peak of the market. He used the money from that sale, plus a lot of debt, to buy the Redskins, his favorite team when he was a kid. (Unsurprisingly, many consider the name of the team to be a slur, but Snyder defends keeping it.) He had only been an owner for a brief period when we met.
I told Mr. Snyder about the project with Cade and he immediately said he was going to send “his guys” to see us right away, even though they were in the midst of the season. He said, “We want to be the best at everything.” Apparently when Mr. Snyder wants something he gets it. That Monday I got a call from his chief operating officer, who wanted to talk to Cade and me ASAP. We met Friday of that week with two of his associates and had a mutually beneficial discussion. We gave them the basic lessons of our analysis, and they were able to confirm some institutional details for us.
After the season ended, we had further discussions with Snyder’s staff. By then, we were pretty sure they had mastered our two takeaways: trade down and trade picks this year for better picks next year. Cade and I watched the draft on television that year with special interest that turned into deep disappointment. The team did exactly the opposite of what we had suggested! They moved up in the draft, and then traded away a high draft pick next year to get a lesser one this year. When we asked our contacts what happened we got a short answer. “Mr. Snyder wanted to win now.”
This was a good forecast of Snyder‘s future decisions. In 2012 the Redskins had the sixth pick in the draft, meaning they had been the sixth worst team in 2011, and they were desperate for a high-quality quarterback. There were two highly rated quarterbacks available that year, Andrew Luck and Robert Griffen III, who is known as RG3 for short. Indianapolis had the first pick and had announced their intention to take Luck. The Redskins wanted RG3. The second pick belonged to the St. Louis Rams, who already had a young quarterback they liked, so the Redskins made a deal with the Rams. They moved up four spots from the sixth pick to the second one, and in addition to giving up that sixth pick they gave the Rams their first- and second-round picks for the following year, 2013, and their first-round pick in 2014. This was an astonishing price to pay to move up just four spots.
How did things work out? In the first year, RG3 did his best to make the trade look smart, and us egghead professors look dumb. He was an effective player who was exciting to watch and the team was winning, making the playoff
s, suggesting that the trade might have a chance of working out if RG3 became a superstar. But late in the season he was injured and sat out a game. When he came back to play again, possibly too early, he aggravated the injury and needed surgery. The following year, he did not return to the top form he had showed as a rookie, and the Redskins had a terrible season, so bad that the 2014 first-round pick the Redskins had given the Rams turned out to be the second pick in that draft, so giving up that pick turned out to be very expensive. (Recall that it was a number two pick that the Redskins had originally traded up to get.) The 2014 season was also a disappointing one for RG3. In hindsight, another player named Russell Wilson, who was not picked until the third round, appears to be better and less injury-prone than RG3. During his three years in the NFL, Wilson has led his team to the Super Bowl twice, winning once.
Of course, one should not judge a trade using hindsight, and the Redskins were certainly unlucky that Griffen suffered from injuries. But that is part of the point. When you give up a bunch of top picks to select one player, you are putting all your eggs in his basket, and football players, like eggs, can be fragile.§
Our relationship with the Redskins did not last very long, but we soon found that another team (whose identity shall remain confidential) was interested in talking to us about draft strategy. In our dealings with that team we learned that there would often be debates among the team’s leadership about draft strategy. Some staff members who were comfortable with analytic thinking bought into our analysis and argued for trading down and lending. Others, such as the owner or one of the coaches, would often fall in love with some player and insist on trading up to get their guy. Furthermore, on the few occasions where the team did trade down in the first round, getting a later first-round pick plus an additional second-round pick, the extra pick would not last long. The extra pick had the feel of “house money” and was usually traded away quickly to grab another “sure thing.”
The failure of teams to draft optimally is a good example of a situation where a principal agent problem would be more accurately labeled a dumb principal problem. When an economist says of a team trading up, “That is just an agency problem,” they mean that the general manager or the coach is worried about his job and needs to win now or get fired. Of course, it is perfectly rational for coaches and general managers to be worried about losing their jobs—they do often get fired. But I think blaming their bad decision-making on traditional agency problems is a mischaracterization. In many of these situations, and not only in sports, the owner is at least as responsible for causing the problem as the employees. It is often the case that general managers trade up because the owner wants to win now. This is similar to the example discussed in chapter 20 about the CEO who wanted his employees to take on twenty-three risky projects but was only going to get three because his employees were worried about the CEO firing them if the project did not pan out. It was up to the CEO to solve this problem.
The same applies to coaching decisions. In American football, each play is choreographed and there are dozens of specific, strategic decisions that coaches get to make, unlike in European football (soccer), which because of its more fluid nature only offers a small number of set plays, such as corner kicks. Some of the specific decision-making opportunities in the NFL can and have been analyzed. One specific decision is whether to “go for it” on fourth down. A team has four plays, called downs, in which it tries to gain 10 yards or score. If it does not, the other team gets the ball. If a team has failed to gain 10 yards on its first three plays, it has the option of trying to pick up the remainder of the necessary 10 yards (called “going for it”), attempting a field goal, or punting the ball down the field and giving possession to the other team. David Romer, an economist from Berkeley, studied this problem and found that teams don’t go for it enough.
Romer’s analysis has been replicated and extended with much more data by a football analytics expert named Brian Burke, and in 2013 the New York Times used his model to help create an application that computes the optimal strategy in any fourth-down situation: punt, go for it, or kick a field goal. Fans can follow the “New York Times 4th Down Bot” in real time and see what the math says a team should be doing. So what effect has this research plus a free app had on the behavior of football coaches? Essentially none. Since Romer wrote his paper, the frequency of going for it on fourth down has marginally gone down, meaning that teams have gotten dumber! (Similarly, there has been no noticeable change in teams’ draft strategy since our paper came out.)
Nate Silver, the ex–sports analytics junkie who became famous for his political forecasts and for the excellent book The Signal and the Noise, estimates that bad fourth-down decisions cost a football team an average of half a win per season. The Times analysts estimate it to be closer to two-thirds of a win per year. That may not seem like a lot, but the season is only sixteen games. A team can win an extra game every other year just by making the smart decision two or three times a game, one they can even check online if they need help.¶
Of course, coaches are Humans. They tend to do things the way they have always been done, because those decisions will not be second-guessed by the boss. As Keynes noted, following the conventional wisdom keeps you from getting fired. A smart owner (who reads economics journals or hires someone to do that) would urge his staff to follow the strategy that maximizes the chance of winning, and tell them that it is going against the odds that will get you fired. But there are not very many of those owners. So owning a billion-dollar football team does not mean you are in Gary Becker’s 10% club, and it certainly does not mean that you will be able to hire people who are in the club or get them to make optimal decisions.
Where should this leave us regarding the validity of the Becker conjecture—that the 10% of people who can do probabilities will end up in the jobs where such skills matter? At some level we might expect this conjecture to be true. All NFL players are really good at football; all copyeditors are good at spelling and grammar; all option traders can at least find the button on their calculators that can compute the Black–Scholes formula, and so forth. A competitive labor market does do a pretty good job of channeling people into jobs that suit them. But ironically, this logic may become less compelling as we move up the managerial ladder. All economists are at least pretty good at economics, but many who are chosen to be department chair fail miserably at that job. This is the famous Peter Principle: people keep getting promoted until they reach their level of incompetence.
The job of being a football coach, department chair, or CEO is multidimensional. For football coaches, being able to manage and motivate a group of young, rich giants over a long season is probably more important than being able to figure out whether to go for it on fourth down. The same goes for many senior managers and CEOs, many of whom were notoriously poor students. Even the ones that were good students have undoubtedly forgotten most of whatever they learned when they took a class in statistics.
One way to salvage the Becker conjecture is to argue that CEOs, coaches, and other managers who are hired because they have a broad range of skills, which may not include analytical reasoning, could simply hire geeks who would deserve to be members of Becker’s 10% to crunch the numbers for them. But my hunch is that as the importance of a decision grows, the tendency to rely on quantitative analyses done by others tends to shrink. When the championship or the future of the company is on the line, managers tend to rely on their gut instincts.
Cade and I have now moved on to a third team that has an owner who aspires to get into Becker’s elite club, but the more we learn about how professional teams work, the more we understand how difficult it is to get everyone in the organization to adopt strategies that maximize profits and games won, especially if those strategies violate conventional wisdom. It is clear that a necessary condition is to have clear buy-in from the top, starting with the owner, but then that owner has to convince everyone who works for him that they are really going to be reward
ed for taking smart but unconventional chances, even (especially!) when they fail. Few teams have achieved this winning formula, as evidenced by the lack of progress on fourth down and draft day decision-making. Clearly, in order to understand how teams or any other organizations make decisions—and therefore how to improve them—we need to be fully aware that they are owned and managed by Humans.
________________
* Sadly, Gary Becker died in 2014 while this book was being written. He was one of the most imaginative economists I have ever met. I am sorry that he is not around to tell me what he thinks of this book. I am sure I would have learned something from his comments, even if I disagreed. The cliché “he is a gentleman and a scholar” was an apt description of Gary.
† These statistics use the simple metric of “games started” to determine who is better. We use this simple metric because it can be measured for players at any position. However, these results and others I will mention are similar even if we use more fine-grained performance measures, such as yards gained for a wide receiver or running back.
‡ A really smart team will trade a second-round pick this year for a first-round pick next year, and then trade that first-round pick for multiple second-round picks the following year, possibly converting one into a first-round pick in the subsequent year, and so forth.
§ Postscript: The Redskins had a late-season game in 2014 against the St. Louis Rams, the team that received all those picks Washington relinquished to acquire their dream player. At the beginning of the game, the Rams’ coach sent out all the players they had chosen with those bonus picks to serve as team captains for the coin toss that began the game. The Rams won the game 24-0 and RG3 was sitting on the bench due to poor play. We will see whether Mr. Snyder learns to be patient.
¶ Footnote for NFL fans only: I think Silver’s estimate may be conservative. It neglects the fact that if you know you are going to go for it on fourth down, it changes the plays you can call on third down. If a team has a third down and five yards to go they almost always try a pass, but if they know they will go for it on fourth and two yards to go, they can try a running play more often on third down, which of course will also improve their chances when they do decide to pass, since they will be less predictable.
Misbehaving: The Making of Behavioral Economics Page 31
