NFL Referees 2009 And How The Underdogs Did With Them

One way to look at how normally a referee crew calls a game is to see whether home teams did extremely well when they were mega-favorites, somewhat well when they were slight favorites, poorly when they were slight-underdogs, and really badly when they were mega-underdogs. If the distribution of results has little correlation to the betting line, then it might be an indication that the referee crew calls games differently than most, so that the results are not that predictable. We looked at this in a recent posting.

A different way to look at how normally a referee crew calls a game is to see how well the favorites did, either as a raw won-loss percentage or against the spread. This approach ignores whether the home team performed easily or barely won (or beat the spread). It also ignores trying to find a correlation between the performance and by how much the team was a favorite.

Top and bottom referees for favorites' won-loss record in the 2009 regular season:
1. John Parry and Jeff Triplette (87%)
3. Don Carey and Bill Leavy (80%)
...
13. Carl Cheffers, Pete Morelli, Alberto Riveron, Gene Steratore (60%)
17. Jerome Boger (53%)
[The overall average was 69.5%, by the way.]

And the top and bottom for favorites against the spread:
1. Don Carey (80%)
2. John Parry (77%)
3. Bill Leavy, Walt Coleman, Ron Winter (60%)
...
13. Scott Green (38%)
14. Mike Carey and Terry McAulay (37%)
16. Ed Hochuli and Tony Corrente (33%)
[The overall average was 49.4%]

Scott Green is the referee for Super Bowl XLIV, the 2010 Super Bowl between the Colts and the Saints. How does he stack up (putting aside how he has an all-star crew rather than his regular crew)?

Favorites went 69% as a won-loss record, so he was 8th out of 17, or slightly good for favorites. This is right on the overall average of 69% for the year for all refs.

Favorites against the spread went 38%, so he was 13th out of 17th. Not good for favorites. This was way below the overall average of 49% for the year for all refs.

So a basic way to look at it would be to say the favorites (the Colts) have a 69% chance of winning and a 38% or 49% chance against the spread, depending on whether you focus on the season-long trend or Scott Green's trend.

But let's look at games where the betting line was 4-5 points. This will weed out games where there was an overwhelming favorite or one where the teams were virtually equal. For all referees, the favorites in those games went 56% as a won-loss rate and 48% against the spread. Not that favorable for the Colts.

As for Scott Green's regular season games, the results for the favorites W-L and ATS were: favored by 3 (0-1 W-L, 0-1 ATS), 3.5 (2-1 W-L, 2-1 ATS), 4.5 (1-0 W-L, 0-1 ATS), 5.5 (1-0 W-L, 1-0 ATS), 6 (3-0 W-L, 1-1-1 ATS). So when the point spread was between 3 and 6, favorites in Scott Green's games went 7-2 W-L and 4-4-1 ATS.

NFL Referees 2009 and Teams' Performance Compared To Expectations

This piece looks at which referee crews in 2009 had a strong correlation between game results and what people expected for the game.

In a perfectly predictable crew, all things being equal, the home team's margin of victory or defeat would correlate almost perfectly with the betting line that for the most part tends to give an indication of the expected result for the game. (Of course, people who set the betting line are not solely focused on the expected result -- they are also focused on what betters' expectations of the results are, but we put that minor flaw to the side in this analysis because it is difficult to account for.)

Also for a perfectly predictable crew, all things being equal, the home team should win more often when it is heavily favored by the betting line and it should lose when it is the heavy underdog according to the betting line.

Is it good to be a perfectly predictable crew? Perhaps yes -- maybe being perfectly predictable means that the crew makes calls in the way that both teams expect, so they perform up to (or down to) their abilities. Meanwhile, a crew that has unpredictable results might reflect that they have unexpected interpretations of the rules and good teams cannot perform up to their expectations.

Let's take a look at two yardsticks -- correlation of the betting line to whether the home team won and correlation of the betting line to the home team's margin of victory (or loss).

Here are the rankings. The most predictable for 2009 is listed first:
1. Bill Leavy
2. John Parry
3. Don Carey
4. Jeff Triplette
5. Walt Coleman
...
13. Carl Cheffers
14. Ron Winter
15. Alberto Riveron
16. Pete Morelli
17. Scott Green

The Super Bowl referee for the Saints-Colts game is Scott Green, who had the least predictable game results in the regular season as correlated to the betting line. (Keep in mind the Super Bowl has an all-star crew so it is not Scott's usual crew.)

How did Scott Green rack up the least predictable results in the regular season? Let's take a look at some of the results compared to the betting line in his games.

You'd think that games where the home team is heavily favored, they would most likely win the game and vice-versa. But Denver as a home team was heavily favored (9.5 points) over Kansas City in week 17 but lost by 20 points. The Giants as a home team were heavily favored (9 points) over the Panthers in week 16 and lost by 32 points. Meanwhile, the Redskins as a visiting team were pretty favored (6.5 points) over the Lions in week 3 but lost by 5 points.

Other minor upsets were the Chargers as a home team was favored (3.5 points) over the Broncos in week 6 but lost by 11 points. The Steelers as a visiting team was favored (3 points) over the Bears in week 2 but lost by 3 points.

It turns out that the betting line did not help you predict which team would win the game. The upsets were randomly distributed along the betting line in his games.

For other referees, the home team won in the games where it was the most favored and its losses were clumped toward where it was the least favored. So the results correlated strongly to the pre-game betting line.

The specifics are that Scott Green had an -0.076 correlation between the betting line and the home team's margin of victory (17th out of 17 crews) and an 0.198 correlation between the betting line and whether the home team won (15th out of 17 crews).

If this holds up for the Super Bowl, then it suggests that the result of the game will not correlate well to the betting line. Perhaps teams do not perform in line with their expectations in Scott's games as much as they do in games with other crews...

NFL Referees for Super Bowl XLIV: February 2010 Saints-Colts

Some analysis of the NFL referees for Super Bowl XLIV between the Saints and the Colts on February 7, 2010. Let's focus on the Super Bowl referee Scott Green, even though the Super Bowl has an all-star crew so Scott Green's regular season statistics (which are based on his usual crew) will not directly apply to the members of the Super Bowl crew.

Total penalties: 11th of 17 at 11.4 accepted penalties per game. Looking at the entire Super Bowl crew, none of them worked on the crews that had the 5 highest accepted penalties per game so my prediction is there will be a slightly below average number of penalties in the Super Bowl.

Total penalty yards: 8th of 17 at 99 penalty yards per game. Looking at the entire crew, it trends toward a lower number of penalty yards per game. Other than Scott Green, they were 3rd, 11th, 12th, 14th, 15th, and 17th out of 17.

Yards per penalty: 3rd of 17 at 8.7 yards per penalty. My prediction is there could be some high-yardage penalties, although not too many penalties. Other than Scott Green, the crew was 1st, 5th, 7th, 11th, 13th, and 15th out of 17.

Total points scored: 3rd of 17 at 48.1 yards per game. Hard to tell if it would be high-scoring because Scott Green suggests it would be high-scoring at 3rd out of 17, but the crew is spread out having worked on crews that were 1st, 7th, 9th, 11th, 12th, and 14th out of 17 during the regular season in total points scored.

Points scored compared with the over-under: 3rd of 17 at +5.3 per game. Think of this as the adjusted total points scored, accounting for what people expected to be scored in the game. This is similar to total points scored but if you looked only at the referees and nothing else (a dangerous approach), I would tend to bet on the over for the game. The crew worked during the regular season on crews that were 1st, 7th, 8th, 11th, 12th, and 15th in the regular season.

Over/under percentage win with the over: 4th of 17 at 63%. Again, I would tend to bet on the over for this game if all you looked at was the referee crew. The working crew for the regular season was on the 1st, 6th, 7th, 11th, 14th, and 17th crews so it was quite evenly-distributed.

Home team win rate: 5th of 17 at 63%. But this probably is irrelevant because the Super Bowl does not really have a true home team. The other crew is widely distributed, having worked on the 1st, 3rd, 8th, 11th, 14th, and 15th crews in the regular season for home team win rate.

Super Bowl Referees for Super Bowl XLIV (Saints-Colts)

We come to the Super Bowl and two difficulties in analyzing the referee crews.

First, it is an all-star mixture of officials who worked on different crews in the regular season. So we can't rely on the regular season data because they did not all work together and it is difficult to separate out what part of a crew's statistics depends on each crew member.

The breakdown is:
Referee: Scott Green
Umpire: Undrey Wash (Cheffers's crew)
Head Linesman: John McGrath (Corrente's crew)
Line Judge: Jeff Seeman (Steratore's crew)
Side Judge: Greg Meyer (Don Carey's crew)
Back Judge: Greg Steed (McAulay's crew)
Field Judge: Rob Vernatchi (Morelli's crew)

So, do we analyze this as if Scott Green will impose his will on the crew and check his statistics only? None of the crew members have worked together before, so it is a mixture of seven different crews. Should we average out those seven crews' regular season statistics?

The second difference is that there is no true home team in the Super Bowl. I think we throw out the statistics we have about how each crew treated the home team and visiting team in the regular season games. However, maybe the statistics about how they treated favorites and underdogs still is useful. For example, if underdogs did especially well with a referee in the regular season, maybe that referee calls games a little differently than most referees so that it is harder to predict the result, which would also be true in a Super Bowl.

Let's dig in to the statistics for Scott Green's regular season, even though he has different crew members in the Super Bowl...