Skill versus Luck in Poker – An Economic Analysis

by robekulick

In this paper, Freakonomics author and University of Chicago economics Professor Steve Levitt and his co-author Thomas Miles use econometrics to assess that age old question, to what extent is poker a game of skill. Now anyone who’s ever played a casual, amateur game of poker on a regular basis knows that some players are better than others and that quite often the difference in outcomes in amateur games often reflects good hand-selection, patience, and not bluffing too often.

However, a much more difficult question, is to what extent does skill affect outcomes at large events with many professional players like the World Series of Poker? Well, Levitt and Miles find substantial evidence that the most skilled poker players dramatically outperform other players:

The abstract reads:

"In determining the legality of online poker – a multibillion dollar industry -- courts have relied 
heavily on the issue of whether or not poker is a game of skill.  Using newly available data, we 
analyze that question by examining the performance in the 2010 World Series of Poker of a 
group of poker players identified as being highly skilled prior to the start of the events.   Those 
players identified a priori as being highly skilled achieved an average return on investment of 
over 30 percent, compared to a -15 percent for all other players.  This large gap in returns is 
strong evidence in support of the idea that poker is a game of skill."

Levitt and Miles explain further:

"Our empirical findings suggest a substantial role for skill in poker over the time horizon 
examined.  The 720 players identified a priori as being high-skilled generate an average ROI of 
30.5 percent in the 2010 WSOP, reaping an average profit of over $1,200 per player per event.
In contrast, all other players obtain an average ROI of -15.6 percent, implying a per event loss of 
over $400.  The observed differences in ROIs are highly statistically significant and far larger in 
magnitude than those observed in financial markets where fees charged by the money managers 
viewed as being most talented can run as high as three percent of assets under management and 
thirty percent of annual returns."

The paper explains that highly-skilled players, as defined by Levitt and Miles, represent 12.1% of the entries in WSOP events (note this paper looks not just at the “main event” but all of the 57 events associated with the WSOP).

Another very interesting point made in the paper is this one:

"... An exhaustive pairwise comparison of high skilled and low skilled players entered in each 
tournament in the WSOP finds that the high skilled player wins 54.9 percent of the match ups.  
For purposes of comparison, we calculated the regular season win rates for professional sports 
teams that made the playoffs in the previous season – making the playoffs last year is akin to 
being a highly skilled player entering the WSOP.  Since the year 2007, teams that made the 
playoffs the previous season win 55.7 percent of their games in Major League Baseball against 
teams that failed to make the playoffs in the previous year.  Thus, in some crude sense, the 
predictability of outcomes for pairs of players in a poker tournament is similar to that between 
teams in Major League Baseball.  To the extent that baseball would unquestionably be judged a 
game of skill, the same conclusion might reasonably be applied to poker in light of the data."

So a few lessons here:

(1) At the top-level of poker, this data suggests that is very much a game of skill.

(2) Success at the WSOP is very skewed towards the high end of the distribution, so it is probably a bad idea to play in it unless you are a very skilled player. Since a lot of these players play online as well, this also provides some evidence as to why most people lose when playing online poker.

One interesting remaining question I have is are there significant differences in skill among the very top players? This would require a different sort of analysis with data on player performance over time, but it would be very interesting to know if poker is basically a situation where a skilled group simply takes money from an unskilled group or whether the distribution is more like a pyramid with money moving constantly up the pyramid.