Super Bowl: Bears, Colts, and Tony Corrente

So, the news is that the Chicago Bears-Indianapolis Colts game will be called by a specially composed all-star referee crew led by referee Tony Corrente. Two reasons the Super Bowl will not match up directly to Tony's regular season games: (1) there is no home team and (2) Tony will lead an all-star crew, not everyone else on his regular season crew.

Putting aside these major warning signs, let's play what-if and pretend Tony were calling the game with his usually regular-season crew (or perhaps if you think Tony can dictate his vision on the rest of the crew?):

Using regular-season DVOA, the Bears and Colts are pretty evenly matched (Bears 5% better DVOA), which was rare for Tony this year. In games where the teams were within 10% DVOA, the favored team went 1-1. Tony did 5% better for favorites than you'd ordinarily expect in the regular season, so that's a good sign for the Bears.

By season-wins, the Bears had one greater win. In the regular season, the team with one-greater win went 1-0.

As far as penalties called, Tony had the fifth-highest number of penalties per game (4th-highest when you adjust for strength of schedule).

NFL Referees by penalties called, adjusted by schedule

It's easy to figure out which referee had the most and least total accepted penalties per game, but what about adjusting it to the strength of schedule? It only takes crunching the expected accepted penalties per game for every team in the referee's games and recalculating.

As it turns out, the results are relatively the same but Peter Morelli edges out Walt Coleman for the referee who called the net least accepted penalties in 2006 (they were tied based on adjusted numbers). Congratulations, Peter!

Find the data for all 17 referees by checking the comments to this post. Adjusted means accepted penalties compared to the referee's scheduled teams and raw means unadjusted to the schedule.

Top adjusted penalty-callers:
Ron Winter (+1.8, raw 13.9)
Gene Steratore (+1.7, raw 13.6)
Larry Nemmers (+1.3, raw 13.3)

Least adjusted penalty-callers:
Peter Morelli (-2.6, raw 9.333)
Walt Coleman (-2.3, raw 9.333)
Bill Leavy (-2.0, raw 9.9)

NFL Referees: Some better for favorites than others but no home consistency

There is a consistent correlation for NFL referees and how well favorites do in their games, but no consistent correlation related to the home teams.

Here's a short explanation of my analysis (I'll put more information in the comments to this post so check there for details):

Check whether there is a correlation between how well home teams or favorite teams did for a referee crew in the first and second halves of the NFL season. If crews have an influence, then you'd see some consistency between the first-half results and the second-half. But if it's random, then there'd by no correlation.

In doing this analysis, I filtered out the strength of schedule by using a system to estimate the expected number of wins for the home and favorite teams in each half of the season.

The result?
A positive correlation of 0.1729 for referees and how well favorites did in the first and second halves of the season! So it seems some referees are better for favorites and some not so good. Why? Perhaps because a group of referees call the game consistently so the favorites do well, but others call the game a different way so the traditionally strong teams might not do as well. For example, if 80% of the referees call holding a certain way, then the favorites will do consistently well with them but not in games called by the other 20% of the referees.

Which referees were good for favorites?
Walt Anderson (+20%, 1H +10% and 2H +32%)
Terry McAulay (+17%, 1H +7% and 2H +27%)
Scott Green (+17%, 1H +32% and 2H +2%)

Which referees were bad for favorites?
Walt Coleman (-26%, 1H -16%, 2H -37%)
Larry Nemmers (-13%, 1H +10%, 2H -37%)
Peter Morelli (-12%, 1H -2%, 2H -23%)

How does Ed Hochuli do? Pretty much in the middle of the pack: +1%, 1H -5%, 2H +6%.

But no such correlation regarding how well the home teams do. The correlation was negative, actually at -0.1551.

Post your comments, questions, and criticisms of this analysis!

NFL Referees Conference Championships

Here is the updated report, with initial statistics based on season-long analysis and adjustments for strength of schedule. By the way, my guess is that Ed Hochuli is a prime candidate to be the ref for the Super Bowl. But there is a big warning -- in the playoffs, the NFL mixes up some of the crew so just the stats for the lead referee in the regular season will not reflect each of the crew members in the actual game. So maybe these stats don't have as much relevance, unless somehow the lead ref imposes his vision on the others:

New Orleans Saints at Chicago Bears: Terry McAulay. Pretty middle of the pack, with home teams winning 56% of Terry's games this season. A bit high on the average yards per penalty (8.5 yards for 4th out of 17 referees). Terry is amazingly good for favorites. Using DVOA, home favorites went 7-0 over the season and visiting favorites went 7-2, so good news for Chicago. Also, home teams with more wins went 6-1 in the regular season, another strong sign for Chicago. The last eight games of the season were won by the team with better DVOA. Postscript: As it turns out, the home team Chicago Bears beat the New Orleans Saints by a wide margin.

New England Patriots at Indianapolis Colts: Bill Carollo. Over the course of the season, visiting teams did great in Bill Carollo's games -- winning 53% of the time (on its own very impressive, even better considering the strength of schedule predicted visiting teams winning only 45% of the time!) Bill called more penalties on the visiting team in only 33% of the games, the least of the 17 referees. Bill calls less penalties than usual (13th) and one of the lowest average yards per penalty (7.8 yards, 16th out of the refs). All sounds great for NE so far but wait, Bill is great for underdogs too. Underdogs won 50% of the games, when you'd expect them to win around 62% of the time. The team with the larger DVOA is NE, so the underdogs are the Colts this time around. Bill's three games where the visiting team had slightly better DVOA were Colts (home underdog) beat the Eagles 45-21, Cardinals (home underdog) lost to the Rams 14-16, and the Steelers (home underdog) beat the Saints 38-31. Postscript: this game went down to the wire before the home team Indianapolis Colts beat the New England Patriots. It went against Bill's trend by calling more penalties on the visiting team. The average yards per penalty was 7.9 yards, close to Bill's average (even though the other refs were not the ones Bill usually works with).

NFL Playoff Referee Commentary

Trying to compile some of the commentary on the NFL referees in the playoffs. Add a comment with a link to other articles.

Seahawks-Bears: on a key fourth and one, the Seahawks did not convert the first down after the center snapped the ball late. Holmgren said in theory the referee could have called a procedure penalty and he wished the referee had done so to give them another chance.
http://sports.yahoo.com/nfl/news?slug=ap-seahawks-hasselbeck&prov=ap&type=lgns

Kyle Rota of seahawks.net believes the referees did a good job other than one pass interference call with 9:14 left in the third quarter that should not have been called.

NFL Playoff Referee Report: Conference Semi-finals

Indianapolis Colts-Baltimore Ravens: Bill Leavy. Bill might be a good sign for Baltimore because home teams do well in Bill's games. Bill was the third best for home teams out of 17 refs this season (won 60% when the the teams involved suggest only 45%). Also, home teams that were overall stronger went 5-1 in Bill's games (using footballoutsiders.com DVOA as a yardstick) or went 4-1 in Bill's games (using W-L as a yardstick). Bill calls some of the least number of penalties of the refs (15th out of 17). So much for being a good sign for Baltimore after the Ravens lose the field goal-fest. Bill called only 9 penalties, which is very low for a game, so at least that characteristic held up.

Philadelphia Eagles-New Orleans Saints: Mike Carey. Good news for the home team New Orleans Saints. Mike was the best for home teams, who won 67% of his games. He called the second-highest percentage of the penalties against the visitors (59%). Also surprising is that his games had around 6 points less than you'd expect for the teams involved (and had less total points in 13 of his 15 games). The rare high-scoring game is still possible, though, such as the 42-30 Seattle win over the Giants in week 3. But the visiting team (Philadelphia) has a better W-L record and had a better DVOA during the season. Home teams with a lower DVOA went 5-4 over the season. So the game might go either way after all. Postscript: it wound up being good for the home team, with a victory and a key false start penalty against the visiting team in the 4th quarter. Mike called twice as many penalties against the visiting Eagles so that followed Mike's season-long trend. The number of points scored, though, bucked the trend, because it was pretty high-scoring.

Seattle Seahawks-Chicago Bears: Gerry Austin. Gerry had the highest average yards per penalty so lots of high-yardage penalties were called so his crew is not afraid to throw the flag for long penalties. For example, he is 8th (out of 17) in the number of penalties but jumps up to 3rd in total penalty yards, ranking at the very top in average yards per penalty (9.2 yards, second place is at 8.8). Gerry is also tops in the percent of penalties called against the visiting team and the number and yardage of penalties against the visiting team. So lots of penalties against visiting teams. But that doesn't translate to tons of home team wins. Home teams won 53% of the time, right on the league average. Home teams won 75% of Gerry's games over the first half of the season but only 29% over the second half so Gerry's results were very skewed first for then against home teams. The strength of the teams wasn't that different -- you'd expect 57% home wins over the first half and 51% home wins over the second half. Chicago's DVOA is a big 37% more than Seattle's and home teams with DVOA advantages went 6-2 in Gerry's games. But the huge DVOA advantage is an especially bad sign for Seattle -- home teams with 30-40% DVOA advantage went 21-2 this season for all refs. Home teams with 4 more regular-season wins went 9-3 this season for all refs. Postscript: well, the home team won in overtime so it was a very close game. And the most distinguishing detail about Gerry -- a large average yards per penalty -- didn't happen in the game, which had an average of only 8 yards per penalty and a majority of the penalties were against the home team, not the visitors.

New England Patriots-San Diego Chargers: Tony Corrente. Home teams slightly worse than average in Tony's games, winning 47% as opposed to the 53% league average. But Tony is pretty close to what you'd expect when you factor in the strength of schedule (visiting teams had a better DVOA 10 out of 15 games and more season-wins 8 times, equal 3 times, and less wins only 4 times). By DVOA, the teams are very evenly matched, which is rare for Tony's games this season. By season-wins, the home team SD has 2 more wins. When home teams had 3+ more wins in Tony's games, they went 3-1. For all refs, when the home team has a less than 10% DVOA advantage, home teams went 27-15 (64% rate). For all refs, when the home team had 2 more wins, they only went 4-4 (50%). Postscript: wow, Tony called lots of significant penalties, including two personal fouls that affected the game. But overall, not a large number of penalties. As far as the score, a very close game.

NFL Refs: Bias May Exist Regarding Home Teams but not for Favorites

I am now working on studying whether there are NFL referee tendencies either in favor of or against home teams or favorites. Are some referees better for home teams (or for favorites)?

The first question is how to come up with a model for predicting which team is the favorite. Please post comments to offer suggestions.

I checked how often the home team actually won to how many times you'd expect the home to win considering the teams involved. To estimate the expected-wins, I used the difference in season DVOA (using footballoutsiders.com statistics) and a homemade conversion of the difference and the estimate win rate.

I checked whether there was a correlation by comparing a random sampling of 6 out of the 17 referees and compared the first and second halves of the season.

The result: there may be a bias for or against the home teams by referee. But there might be no bias for or against favorites by referee. Specifically, there was a correlation of 0.177 about whether a ref favors the home team and a correlation figure of -0.162 about whether a ref tends to like the favorite or not. It was critical to factor in the strength of the visiting and home teams -- if you ignored that, there was no correlation. In fact, there were wild fluctuations about the expected win-rate of home teams for a ref in the first and second halves of the season (Houcholi had 63% in the first half and 40% in the second half while Winter had 49% in the first half and 63% in the second half...) This suggests you actually do want Mike Carey as your referee if you are the home team!

Wild Card Weekend: Referee Report

Kansas City Chiefs at Indianapolis Colts: Jeff Triplette. Moderately good news for the Colts -- there are few upsets when Jeff has refereed games this season. Home teams have done close to the league average by winning 53% of the games, going 8-7. The big insight is when you compare the season record of the teams (excluding the game that Jeff covered). When the home team had 3 or more wins than the visiting team (such as KC-Indy today), the home team went 6-0. When the two teams were even or differed by one win, the home team went 1-2. When the visiting team had three or more wins greater than the home team, the visiting team went 5-1. In games that Jeff covered this year, Indianapolis as a home team clobbered Houston 43-24 while KC as visitors got beaten by San Diego 20-9. Jeff's games were only slightly higher-scoring than you'd expect (41.7 total points compared to 40.6 for the teams involved). Postscript: true to form the much-favored home team indeed won, but the total points scored was less than expected.

Dallas Cowboys at Seattle Seahawks: Walt Anderson. Bad news for the Seahawks -- home teams have the worst winning percentage for games with Walt than for any other referee this season. Home teams have only won 33% of the games (5-10) where Walt Anderson was the ref in the 2006 season. To help you visualize how bad that is, imagine Washington or Arizona, teams that went 5-11 this season. Otherwise, Walt is fairly middle of the pack, calling slightly less penalties per game than average (12th out of 17), but the average yards per penalty is larger than usual (6th out of 17). More bad news for the Seahawks -- it wasn't a case where the home teams happened to be worse in Walt's regular season games. The visiting teams had an average net point difference of -2.1 but the home teams were better, with a net point difference of -0.3. To put it another way, the visiting teams averaged 7.3 wins on the season but the home teams averaged 7.7 wins on the season. Here's another way to analyze it: in 8 of the games, the home team ended up with more wins in the season than the visiting team. The home team went only 4-4 in those games. In 7 of the games, the visiting team had more wins over the entire season and the home team went 0-7 in those games.

Also, Walt's games are higher-scoring than you'd expect: 45.7 total points rather than 40.5 you'd expect per game. Visiting teams scored 25.1 (you'd expect only 19.8) and home teams scored 20.5 (you'd expect 20.9 based on the teams' offenses and defenses). Postscript: Seahawks buck the trend and win even though they were the home team in a Walt Anderson game! Of course, the final score was extremely close... also, the game was not particular higher-scoring so not exactly as expected.

New York Jets at New England Patriots: Bill Vinovich. Bill started off with an incredible streak of home team wins, but the visiting teams won his last three games to end up with home teams winning 67% of his games (tied for tops of NFL refs this year). When home teams had a better record (excluding Bill's game), the home team went 5-2. Not good for the Jets. Bill's games are lower scoring that you'd expect for the teams involved, 35.4 total points (5.8 less than for the teams involved and lower than you'd expect in 11 of his 15 games). Considering the average of the Jets and Patriots' points scored and allowed yields an expectation of just 38.5 total points and you might say the game might have less than 35 total points when you factor in The Vinovich Effect.

Additional analysis: the net points for the favorite teams have less correlation to the expected results in Bill's games than for other referees, and this trait holds up over the two halves of the 2006 season. This could be good for the Jets, who are considered the underdog. But don't crack open the champagne yet, when the home team is significantly favored, the home team has gone 4-1 in Bill's games. In other stats, Bill is pretty middle of the pack in terms of penalties called (9th of 17 refs), penalty yards (10th), and percent penalties on the visiting team (10th). Postscript: NE kept up the trend of home teams winning with Bill Vinovich, but the total points scored was much higher than expected.

New York Giants-Philadelphia Eagles: Peter Morelli. Relatively good news for the Giants. Home teams only won 40% of Peter's games (the league average is 53%), so that's good for the Giants. When home teams had 2 or more victories than the visitors, the home teams went 3-5 in Peter's games (3-4 when home teams had less victories). Today, the Eagles have 2 more victories than the Giants. In general, Peter calls few penalties per game -- 16th (out of 17) in the number of penalties, 17th in penalty yards, 17th in average yards per penalty (7.6 yards per penalty). For the stat-fiends, the correlation between net team wins and scoring margin is only 0.11 for Peter's games while the correlation between net DVOA and scoring margin is a relatively low 0.344. Peter's games are a bit higher scoring than you'd expect for the teams involved (45.2 total points, more than the 40.9 for the teams involved or the 41.3 league average). Postscript: The Eagles bucked the trend and won even though a home team in a Peter Morelli game. The total points scored was not higher so it did not go as expected.

Vinovich Better For Underdogs?

Here's a hypothesis: The rate a home team wins might be unrelated to the referee, but some refs are better for favorites than others.

I've tried to see which ref characteristics hold up over the course of a season by comparing the first half of 2006 with the second half. If they don't hold up, it seems less likely it was related to the refs as opposed to an outside factor. (I won't get into the disclaimers to this simple test.)

How about the rate that home teams win? Probably not related to the ref -- only a correlation of -0.058 between first-half home win rates and second-half home win rates by referee.

How about whether some refs are better for favorites? Yes, probably some relation to the ref. Let me get into the stats then some theories why this might be true.

The stats: using a sampling of 5 of the 17 refs, how well the net points scored matched up to how big a team was a favorite had some correlation to which referee it was. This is true whether the yardstick of determining which team is a favorite is by the team's final record (excluding the game in question) (0.186 correlation) or using net DVOA from footballoutsiders.com (0.229 correlation).

Who is good for favorites? Jeff Triplette: 0.699 correlation by final record and 0.723 correlation by net DVOA.

Which had results that didn't match up as well to expected team strengths? Bill Vinovich: 0.104 by final record and 0.349 by net DVOA.

The theory: Hard to know why this would be true, but here's a theory -- some refs call the games differently from others and that makes the final result less predictable according to how the teams ordinarily do. For example, if 90% of the refs call the game one way, but 10% of the refs call it another way, then the results for those 10% of the refs will not match up well to the results from the rest of the season. Let's say 90% of the refs almost never call holding but 10% of the refs call it by the book. Results for the strict refs will be skewed to favor teams that know how to obey the rules. Results will be different from how well those teams do over the course of the season because for most of the season, following the rules strictly wouldn't matter. You can see how could make a game less predictable if a 5% of the refs call defensive holding strictly but the others don't.

So, with some refs, the favorites do better than with other refs. I will analyze many other statistics such as over/under (total points scored) to see which characteristics really look like it's related to the ref, as proven over the course of the 2006 NFL season. Post some comments if you have questions or ideas.

NFL Referee Statistics and Data 2006 Season (Final)

The full 2006 NFL season statistics! This includes all regular season games.

Sorted by referee (alphabetical, last name), interesting figures include:
TtlPen = average total number of accepted penalties per game
Ttlyd = average total yards of accepted penalties per game
Hwin = percentage that home team wins the game
We base our statistics on penalties accepted, so we ignore penalties called that teams decline. We do this because our stats are based on the NFL web site's game summaries, which tabulate accepted penalties only

Please post a comment if you find this helpful, or if you have requests or suggestions. Enjoy the games!

We are working on some interesting statistics to find out the correlation among some of these factors -- and whether the trends held up throughout the season (as opposed to being a fleeting correlation)!

I wish I knew how to get rid of this annoying space between this line and the table. Ugh, my limited blogging skills are killing me...

Ref Ttlpen Ttlyd AvgPen V%pen V%yd Vdiff pen Vdiff yds Vpen Vyd Hpen Hyd Hwin Vmore Anderson, Walt 11 93 8.4 50% 50% -0.1 0 5.5 46.5 5.5 46.5 33% 47% Austin, Gerry 12.2 113 9.2 61% 58% 2.6 17.4 7.4 65.1 4.8 47.7 53% 80% Boger, Jerome 13.1 103 7.9 49% 50% -0.3 -0.8 6.4 51.3 6.7 52.1 47% 40% Carey, Mike 11.8 97 8.2 59% 54% 2.2 7.1 7 51.9 4.8 44.9 67% 80% Carollo, Bill 10.9 85 7.8 47% 49% -0.7 -0.9 5.1 42 5.8 42.9 47% 33% Coleman, Walt 9.3 74 8 46% 44% -0.7 -9 4.3 32.7 5 41.7 60% 43% Corrente, Tony 13.1 106 8.1 53% 50% 0.7 -0.5 6.9 52.7 6.2 53.3 47% 63% Green, Scott 10.7 94 8.8 55% 58% 1.1 14.7 5.9 54.3 4.8 39.6 47% 67% Hochuli, Ed 12.9 114 8.8 48% 50% -0.5 0.1 6.2 57.2 6.7 57.1 53% 37% Leavy, Bill 9.9 78 7.9 51% 52% 0.1 3.3 5 40.6 4.9 37.3 60% 50% McAulay, Terry 12.2 104 8.5 49% 52% -0.2 3.3 6 53.7 6.2 50.4 56% 47% Morelli, Peter 9.3 71 7.6 53% 56% 0.5 8.1 4.9 39.7 4.4 31.6 40% 47% Nemmers, Larry 13.3 107 8.1 54% 51% 1 2.1 7.1 54.5 6.1 52.4 60% 57% Steratore, Gene 13.6 108 7.9 49% 48% -0.3 -4.5 6.7 51.7 6.9 56.2 60% 43% Triplette, Jeff 13 109 8.4 53% 52% 0.9 4 6.9 56.7 6.1 52.7 53% 60% Vinovich, Bill 12.2 103 8.5 50% 51% -0.1 1.1 6.1 52.1 6.1 51.1 67% 53% Winter, Ron 13.9 113 8.2 53% 51% 0.8 2.6 7.3 58 6.5 55.4 53% 70% TOTAL 11.9 98.4 8.3 52% 51% 0.4 2.8 6.2 50.6 5.7 47.8 53% 54%

Boise St and Oklahoma Ending: Clock Management

On a completely different topic, I'm sure you caught the end of the Boise State and Oklahoma game. There was one key play that I was thinking about as it was happening -- the score is tied with 1:16 left in the fourth quarter and Boise State has the ball at their 25 yard line. They throw an interception and Oklahoma is returning it for a TD with just 1:02 left. While I was watching it live, I thought to myself -- if I'm the Boise State QB, I would let the guy score rather than tackling him at the 1-yard line. And if I'm the CB, I'd down it at the 1 yard line after killing as much time as possible by pausing at the 2-yard line.

Why? Because I would rather have the ball with 60 seconds left and the chance to run time off the clock (remember, in college football the clock starts running after a change of possession) to leave no time left for Boise State to score again and to force them to burn their time outs.

Boise State had only 2 time outs left. So you have two scenarios:

A. Burn the clock and kick a game-winning FG with no time left to go up by 3. If you think you will successfully kick the FG 95% of the time, then this option wins 97% of the time.

B. Score and go up by 7 points, kick off, give Boise State a chance to tie the game with 65 seconds. Footballcommentary.com has a model for estimating the chances of scoring a TD with 60 seconds and 2 time outs left: from your own 10 yard line it's 15%, from your own 30 yard line it's 23%. If Boise State starts around the 20 yard line, then they have around a 19% chance of tying the game. This means they have around a 9% chance of winning the game in OT.

So, stepping out at the 1-yard line is the better choice. Stepping out wins 97% of the time; scoring a TD only wins 91% of the time. They essentially tripled Boise State's chances of winning by scoring rather than stepping out at the one yard-line.
Even if you just took a knee for 3 downs and then kicked a FG, I would rather be up 3 with virtually on time left than give Boise State a 20% chance to score a tying TD...