Examining the Value of Coach's Challenges

Discussion of technique and strategy.

Examining the Value of Coach's Challenges

by FenderGuy69 » Thu Oct 10, 2013 7:07 pm

Interesting work from the guys over at Harvard Sports Analysis Collective (http://harvardsportsanalysis.org/) and Advanced NFL Stats on "Examining the Value of Coach's Challenges".

http://www.advancednflstats.com/2013/10 ... enges.html


This histogram shows the distribution of leverage on the 140 plays that coaches challenged during the 2012 season. The average leverage was 0.07 percentage points of WP. Over 90% of challenges occurred on plays whose leverage was less than or equal to 0.15 WP. Past 0.15 WP, the distribution becomes very sparse – these plays had huge swings in WP, but they were also very rare events.

The most interesting takeaway from this graphic is how many challenges have almost zero leverage. Over 15% of challenges last season had no leverage - in other words, the team challenging the play should be indifferent between the challenge succeeding and failing. On these plays, the coach spent a challenge and risked losing a time out for no potential benefit, which would be highly illogical. With such an odd result, I went back to look at these play descriptions, and found three notable patterns.

First, about half of these challenges occurred in game states where the score differential was over 14, and it is hard for non-scoring plays to significantly change win probability in those cases. Of the remaining zero-leverage plays, about half were challenges that would bring the offense from “close to their opponent’s end zone” to “a bit closer to their opponent’s end zone,” which doesn’t really help the team’s WP very much (or at all).

Neither of these reasons makes the decisions to challenge those plays better, but the last reason I found might take some decisions off the hook. Many of these zero leverage plays involved keeping a highly efficient offense on the field (or getting it off the field, depending on who challenged the play) by challenging the result of a third down play. For example, in the infamous matchup between the Packers and the Seahawks last season, Green Bay challenged a third down measurement to keep Aaron Rogers and Co. in the game. In cases like this one, the WP model may not fully account for the specific in-game strengths and weaknesses of the teams involved (since it is based on an average team). Therefore in some circumstance, challenges might appear to have no leverage because of our WP estimates.
Posts: 1911
Joined: Sun Aug 12, 2012 3:47 pm

Re: Examining the Value of Coach's Challenges

by depopulationINC » Wed Oct 16, 2013 7:17 pm

Interesting. The last point makes a lot of sense though....IMO most NFL challenges are more to save the down than anything. CFL is fairly converse on this matter IMO though, where a lot of challenges involve turnovers, and the second most seem to be on catch / no catch...a little bit more critical when you are talking 3 downs...you really only have 1 more chance at to move the chains if it is first down.
User avatar
Posts: 9194
Joined: Mon Oct 18, 2010 10:21 pm

Users browsing this forum: No registered users and 1 guest