Sorry Talking Heads, You Know Nothing About What Matters in the NFL Playoffs
March 6, 2014 1 Comment
For years, sports commentators who spew evidence-free clichés about the keys to athletic victory have monopolized our airwaves. But recently a technique some of them view as akin to witchcraft, but that’s more commonly known as “statistical analysis,” has begun to bring an end to their reign of terror.
The latest volley in this ongoing battle comes from a new study by Joshua Pitts of Kennesaw State University. Pitts analyzed all 445 NFL playoff games from 1966 through 2012. Among the factors he examined were the number of previous playoff games and playoff wins of quarterbacks and head coaches, the playoff experience of all players relative to their opponents, statistical measures of offensive and defensive quality in terms of both regular season passing and rushing, whether a team was playing at home, and the degree to which a team entered a playoff game on a winning streak. The outcomes Pitts examined included whether a team won their playoff game, and their points scored, points allowed, and margin of victory or defeat.
The most noteworthy of Pitts’ findings is that there’s little evidence playoff experience matters.
Perhaps a surprising result is that neither the previous playoff experience of a quarterback/head coach nor the number of previous playoff wins for a quarterback/head coach has a significant impact on any of the measured outcomes in this study after holding current team quality constant.
All the various measures of previous playoff experience included in this study, including measures of quarterback, head coach, and team playoff experience, are rarely statistically significant determinants of the outcomes measured in this study. Furthermore, even when these measures do prove statistically significant, the magnitude of the impact on the dependent variables tends to be extremely small.
So much for that talking point. At least there’s always the “Playoff success starts with running the football and stopping the run” cliché. Oh wait…
The relative productivity of a team’s passing offense and defense in the regular season is consistently a statistically significant determinant of all of the outcomes measured in this study. However, the relative productivity of a team’s rushing defense is not a statistically significant determinant of any of the measured outcomes. Furthermore, the relative productivity of a team’s rushing offense is only statistically significant at the 10% level in a single specification presented in column 8 of Table 4. This provides overwhelming evidence, as suggested by Arkes (2011), that the key to victory in the NFL postseason is controlling the passing game.
Pitts also found almost no evidence that being on a winning streak (“They’re peaking at the right time!”) had an impact on the outcome of playoff games.
But the news for institutional clichés wasn’t all bad. It turns out the defense does make you marginally more likely to win a championship.
For every 1.4% increase in relative offensive productivity, a team is about 1% more likely to win a postseason game. Similarly, for every 1.1% increase in relative defensive productivity, a team is about 1% more likely to win in the postseason. In general, the marginal effect of defensive productivity is a little larger than the marginal effect of offensive productivity which provides slight support in favor of proponents of ‘‘defense wins championships.’’
Another noteworthy finding concerned home field advantage:
The home team is about 18–29% more likely to win, depending on the specification of the model and holding other factors constant. The results shown in Table 4 predict a 6–10 point advantage for the home team in a matchup between two identical opponents, essentially implying that home-field advantage is worth at least a touchdown in the postseason.
Bookies generally give a team about three points for home field advantage, so this implies that in the playoffs people may be underestimating that edge.
The fact that the way pro football is played can change in just a few years means these results should be taken with a grain of salt. Perhaps there’s been something important in the last five or ten years but it’s been masked by the prior 30 years of data. Alternatively, it’s possible that some of things that do appear to make a difference have already ceased to matter or will cease to matter in the next few years. But overall the study provides some compelling evidence that the clichés you hear about winning in the postseason are often disconnected from reality.
Finally, because no NFL playoff post is complete without a discussion of Tim Tebow, Pitts also used his models to predict the greatest playoff upsets of all time. And wouldn’t you know it, the Broncos Tebow-led overtime win against the Steelers in 2011 ranked in the top 10 in Pitts’ model based on expected win percentage and his model based on expected points margin. (The Vikings 1987 Divisional round win over the 49ers and the Patriots win over the Rams in the 2001 Super Bowl were ranked first, respectively.) In other words, to the surprise of nobody, Tebow’s playoff victory was a statistical anomaly (i.e. fluke.)
Pitts, J. (2014). Determinants of Success in the National Football League’s Postseason: How Important Is Previous Playoff Experience? Journal of Sports Economics DOI: 10.1177/1527002514525409