by Nate Silver
If you have strong analytical skills that might be applicable in a number of disciplines, it is very much worth considering the strength of the competition. It is often possible to make a profit by being pretty good at prediction in fields where the competition succumbs to poor incentives, bad habits, or blind adherence to tradition—or because you have better data or technology than they do. It is much harder to be very good in fields where everyone else is getting the basics right—and you may be fooling yourself if you think you have much of an edge.
In general, society does need to make the extra effort at prediction, even though it may entail a lot of hard work with little immediate reward—or we need to be more aware that the approximations we make come with trade-offs. But if you’re approaching prediction as more of a business proposition, you’re usually better off finding someplace where you can be the big fish in a small pond.
The Economics of the Poker Bubble
The Pareto Principle of Prediction implies that the worst forecasters—those who aren’t getting even the first 20 percent right—are much worse than the best forecasters are good. Put another way, average forecasters are closer to the top than to the bottom of the pool. I’m sure that I’d lose a ton of money if I played poker against Dwan. But I’d gladly play him if, as part of the deal, I were also guaranteed a match for the same stakes against some random person I picked off the street, against whom I’d expect to make back my losses and then some.
We can test this hypothesis empirically by examining the statistical records of poker players. I evaluated the data from an online poker site, which consisted of a random sampling of no-limit hold ’em players over a period in 2008 and 2009. These statistics told me how much money the players won or lost per hand, relative to the stakes they were playing.17
Because near-term wins and losses are very much subject to luck, I applied a statistical procedure18 to estimate what the players’ true long-term profitability was. I then ordered the players by their skill level and broke them down into ten equal-size quadrants. The top quadrant—consisting of the top 10 percent of the player pool*—corresponds to the best player at a typical ten-person table.19 The bottom 10 percent, meanwhile, are the biggest fish.
Figure 10-8a represents my estimate of how skilled the players in each quadrant really are, measured as money won or lost per one hundred hands in a no-limit hold ’em game with $5/$10 blinds. The figures include both money won and lost to the other players and that lost to the casino, which either takes a small percentage of each pot (known as the rake) or charges an hourly fee for dealing the game.20
FIGURE 10-8A: ESTIMATED MONEY WON OR LOST PER 100 HANDS IN A $5/$10 NO-LIMIT HOLD ’EM GAME
I estimate that the very best player at the table in one of these games is averaging a profit of about $110 per one hundred hands played over the long run. That’s a nice wage in an online casino, where hands are dealt very quickly and you could get almost that many hands during an hour or two.* It’s less attractive in a traditional casino, where it might take four hours to play the same number of hands, and translates to wage of $25 or $30 per hour.
The key insight, however, is that the worst players at the table are losing money much faster than even the best ones are making it. For instance, I estimate that the worst player in the game—the biggest fish—was losing at a rate of more than $400 per one hundred hands. This player is so poor that he would literally be better off folding every hand, which would cost him only $150 per one hundred hands instead.
Here you see the statistical echo of the 80/20 rule: there’s a much larger difference between the very worst players and the average ones than between the average ones and the best. The better players are doing just a few things differently from one another, while those at the lower end of the curve are getting even the basics wrong, diverging wildly from optimal strategy.
In the classic poker movie Rounders,21 Matt Damon’s character advises us that if you can’t spot the sucker in your first half hour at the table, then you must be the sucker. I don’t think this is quite true: it may be that the game doesn’t have any suckers. It is emphatically the case, however, that if you can’t spot one or two bad players in the game, you probably shouldn’t be playing in it. In poker, the line between success and failure is very thin and the presence of a single fish can make the difference.
In the game I just described, the one fish was feeding a lot of hungry mouths. His presence was worth about $40 per 100 hands to the other players. That subsidy was enough that about half of them were making money, even after the house’s cut. Poker abides by a “trickle up” theory of wealth: the bottom 10 percent of players are losing money quickly enough to support a relatively large middle class of break-even players.
But what happens when the fish—the sucker—busts out, as someone losing money at this rate is bound to do? Several of the marginally winning players turn into marginally losing ones (figure 10-8b). In fact, we now estimate that only the very best player at the table is still making money over the long run, and then less than he did before.
FIGURE 10-8B: ESTIMATED MONEY WON OR LOST PER 100 HANDS IN A $5/$10 NO-LIMIT HOLD ’EM GAME AFTER FISH BUSTS OUT
What’s more, the subtraction of the fish from the table can have a cascading effect on the other players. The one who was formerly the next-to-worst player is now the sucker, and will be losing money at an even faster rate than before. So he may bust out too, in turn making the remaining players’ task yet more challenging. The entire equilibrium of the poker ecosystem can be thrown out of balance.
How, in fact, do poker games sustain themselves if the worst players are a constant threat to go broke? Sometimes there are fishy players with bottomless pockets: PokerKingBlog.com has alleged that Guy Laliberté, the CEO of Cirque du Soleil, lost as much as $17 million in online poker games in 2008,22 where he sought to compete in the toughest high-stakes games against opponents like Dwan. Whatever the number, Laliberté is a billionaire who was playing the game for the intellectual challenge and to him this was almost nothing, the equivalent of the average American losing a few hundred bucks at blackjack.
Much more commonly, the answer is that there is not just one fishy player who loses money in perpetuity but a steady stream of them who take their turn in the barrel, losing a few hundred or a few thousand dollars and then quitting. At a brick-and-mortar casino like the Bellagio, these players might wander in from the craps table, or from one of its nightclubs, or after going on a winning streak in a tournament or a smaller-stakes game.
In the online poker environment of my experience, the fish population was more irregular and depended on the regulatory environment in different countries, the amount of advertising that the poker sites were doing, and perhaps even the time of year.23 During the poker boom years, however, the player pool was expanding so rapidly that there was always a wealth of fishes.
That was about to change.
The Poker Bubble Bursts
In October 2006 the outgoing Republican Congress, hoping to make headway with “values voters” before the midterm elections24 but stymied on more pressing issues, passed a somewhat ambiguous law known as the Unlawful Internet Gambling Enforcement Act (UIGEA). The UIGEA, strictly speaking, didn’t make online poker illegal. What it did, rather, was to target the third-party companies that facilitated the movement of money into and out of the poker sites. Sure, you could play poker, the law said, in effect—but you couldn’t have any chips. Meanwhile, the Department of Justice began targeting companies that were offering online gambling to Americans. David Carruthers, the CEO of an offshore site known as BetOnSports PLC, was arrested on a layover in Dallas while changing planes on a trip from the United Kingdom to Costa Rica. Other prosecutions soon followed.
All this scared the hell out of many online poker players—as well as many of the proprietors of the games. Party Poker, then the largest online poker site, locked Americans out of the games two weeks after the UIGEA passed; its stock crashed by 65 perc
ent in twenty-four hours.25 Other companies stayed in business, developing workarounds to the new law, but it had become harder to get your money in and riskier to take it back out.
I had made most of my money from Party Poker, which advertised aggressively and was known for having the fishiest players. During the two-week grace period after Party Poker made its announcement but kept the games open to Americans, the games there were fishier than ever, sometimes taking on a Lord of the Flies mentality. I had some of my winningest poker days during this period.
Once Party Poker shut Americans out, however, and I shifted my play to tougher sites like PokerStars, I found that I wasn’t winning anymore. In fact, I was losing—a lot: about $75,000 during the last few months of 2006, most of it in one horrible evening. I played through the first several months of 2007 and continued to lose—another $60,000 or so. At that point, no longer confident that I could beat the games, I cashed out the rest of my money and quit.
My conclusion at the time was that the composition of the player pool had changed dramatically. Many of the professional players, reliant on the game for income, had soldiered on and kept playing, but most of the amateurs withdrew their funds or went broke. The fragile ecology of the poker economy was turned upside down—without those weak players to prop the game up, the water level had risen, and some of the sharks turned into suckers.26
Meanwhile, even before the new law passed, my play had begun to deteriorate, or at least cease to improve. I had hit a wall, playing uncreative and uninspired poker. When I did play, I combined the most dangerous trait of the professional player—the sense that I was entitled to win money—with the bad habits of the amateur, playing late into the evening, sometimes after having been out with friends.
In retrospect, things worked out pretty fortunately for me. The extra time I had on my hands—and my increased interest in the political process following the passage of the UIGEA—eventually led to the development of FiveThirtyEight. And while it wasn’t fun to lose a third of my winnings, it was better than losing all of them. Some players who continued in the game were not so lucky. In 2011, the “Black Friday” indictments filed by the Department of Justice shut down many of the online poker sites for good,27 some of which proved to be insolvent and did not let players cash out their bankrolls.
I’ve sometimes wondered what would have happened if I’d played on. Poker is so volatile that it’s possible for a theoretically winning player to have a losing streak that persists for months, or even for a full year. The flip side of this is that it’s possible for a losing player to go on a long winning streak before he realizes that he isn’t much good.
Luck Versus Skill in Poker
Luck and skill are often portrayed as polar opposites. But the relationship is a little more complicated than that.
Few of us would doubt, for instance, that major-league baseball players are highly skilled professionals. It just isn’t easy to hit a baseball thrown at ninety-eight miles per hour with a piece of ash, and some human beings are a wee bit more refined in their talent for this than others. But there is also a lot of luck in baseball—you can hit the ball as hard as hell and still line out to the second baseman. It takes a lot of time for these skill differences to become obvious; even a couple of months’ worth of data is not really enough. In figure 10-9, I’ve plotted the batting averages achieved by American League players in April 2011 on one axis, and the batting averages for the same players in May 2011 on the other one.28 There seems to be no correlation between the two. (A player named Brendan Ryan, for instance, hit .184 in April but .384 in May.) And yet, we know from looking at statistics over the longer term—what baseball players do over whole seasons or over the course of their careers—that hitting ability differs substantially from player to player.29
FIGURE 10-9: BATTING AVERAGE FOR AMERICAN LEAGUE PLAYERS, APRIL AND MAY 2011
Poker is very much like baseball in this respect. It involves tremendous luck and tremendous skill. The opposite of poker would be something like tic-tac-toe (figure 10-10). There is no element of luck in the game, but there isn’t much skill either. A precocious second grader could do just as well as Bill Gates at it.
FIGURE 10-10: SKILL VERSUS LUCK MATRIX
Low luck
High luck
Low skill
Tic-Tac-Toe
Roulette
High skill
Chess
Poker
Still, it can take a long time for poker players to figure out how good they really are. The luck component is especially strong in limit hold ’em, the game that I specialized in. Correct strategy in this game implies that you will scrap and fight for many pots, and staying in so many hands to the end means that a lot depends on the luck of the deal. A very good limit hold ’em player, in a game where the betting increments are $100 and $200, might make $200 for every one hundred hands played. However, the volatility in his results—as measured by a statistic called standard deviation—is likely to be about sixteen times that, or about $3,200 for every one hundred hands.30
What this means is that even after literally tens of thousands of hands are played, a good player might wind up behind or a bad one might wind up ahead. In figure 10-11, I’ve modeled the potential profits and losses for a player with the statistics I just described. The bands in the chart show the plausible range of wins and losses for the player, enough to cover 95 percent of all possible cases. After he plays 60,000 hands—about as many as he’d get in if he played forty hours a week in a casino every week for a full year—the player could plausibly have made $275,000 or have lost $35,000. In essence, this player could go to work every day for a year and still lose money. This is why it is sometimes said that poker is a hard way to make an easy living.
Of course, if this player really did have some way to know that he was a long-term winner, he’d have reason to persevere through his losses. In reality, there’s no sure way for him to know that. The proper way for the player to estimate his odds of being a winner, instead, is to apply Bayesian statistics,31 where he revises his belief about how good he really is, on the basis of both his results and his prior expectations.
If the player is being honest with himself, he should take quite a skeptical attitude toward his own success, even if he is winning at first. The player’s prior belief should be informed by the fact that the average poker player by definition loses money, since the house takes some money out of the game in the form of the rake while the rest is passed around between the players.32 The Bayesian method described in the book The Mathematics of Poker, for instance, would suggest that a player who had made $30,000 in his first 10,000 hands at a $100/$200 limit hold ’em game was nevertheless more likely than not to be a long-term loser.
FIGURE 10-11: PLAUSIBLE WIN RATES FOR SKILLED LIMIT HOLD ’EM PLAYER, $100/$200 GAME
Our Poker Delusions
Most players, as you might gather, are not quite this honest with themselves. I certainly wasn’t when I was living in the poker bubble. Instead, they start out with the assumption that they are winning players—until they have the truth beaten into them.
“Poker is all about people who think they’re favorites when they’re not,” Dwan told me. “People can have some pretty deluded views on poker.”
Another player, Darse Billings, who developed a computer program that competed successfully33 against some of the world’s best limit hold ’em players,* put it even more bluntly.
“There is no other game that I know of where humans are so smug, and think that they just play like wizards, and then play so badly,” he told me. “Basically it’s because they don’t know anything, and they think they must be God-like, and the truth is that they aren’t. If computer programs feed on human hubris, then in poker they will eat like kings.”
This quality, of course, is not unique to poker. As we will see in chapter 11, much of the same critique can be applied to traders on Wall Street, who often think they can beat market benchmarks like the S&P 500
when they usually cannot. More broadly, overconfidence is a huge problem in any field in which prediction is involved.
Poker is not a game like roulette, where results are determined purely by luck and nobody would make money if they took an infinite number of spins on the wheel. Nor are poker players very much like roulette players; they are probably much more like investors, in fact. According to one study of online poker players, 52 percent have at least a bachelor’s degree34—about twice the rate in the U.S. population as a whole, and four times the rate among those who purchase lottery tickets.35 Most poker players are smart enough to know that some players really do make money over the long term—and this is what can get them in trouble.
Why We Tilt
Tommy Angelo pursued a poker dream before it was cool. In 1990, at the age of thirty-two, he quit his job as a drummer and pianist for a country rock band to play poker full-time.36
“I was hooked on it,” Angelo told me when I spoke with him in 2012. “I loved the idea of being a professional poker player when I first heard the words. The whole idea was so glorious, of not having a job. It’s like you’re beating society, making all your money on your wits alone. I couldn’t imagine anything more appealing.”
But Angelo, like most poker players, had his ups and downs—not just in his results but also in the quality of his play. When he was playing his best, he was very good. But he wasn’t always playing his best—very often, he was on tilt.
“I was a great tilter,” Angelo reflected in his book, Elements of Poker, referring to a state of overaggressive play brought on by a loss of perspective.37 “I knew all the different kinds. I could do steaming tilt, simmering tilt, too loose tilt, too tight tilt, too aggressive tilt, too passive tilt, playing too high tilt, playing too long tilt, playing too tired tilt, entitlement tilt, annoyed tilt, injustice tilt, frustration tilt, sloppy tilt, revenge tilt, underfunded tilt, overfunded tilt, shame tilt, distracted tilt, scared tilt, envy tilt, this-is-the-worst-pizza-I’ve-ever-had tilt, I-just-got-showed-a-bluff tilt, and of course, the classics: I-gotta-get-even tilt, and I-only-have-so-much-time-to-lose-this-money tilt, also known as demolition tilt.”