On the one hand, the inter-year correlation for even strength save percentage is no stronger for goalies remaining with the same team when compared to the value for goalies that changed teams.
This suggests that team effects are negligible.
On the other hand, there is a statistically significant correlation between the even strength save percentage of starters and backups. Moreover, the magnitude of the correlation is moderate when viewed in light of the fact that even strength save percentage exhibits low reliability over the course of a single season.
This suggests that team effects are important.
DO GOALIES WIN GAMES OR JUST ALONG FOR THE RIDE
A good team can win with an average goalie.
In the short run, goalies can and do steal games. In the long run, it’s the team that wins or loses, and the goalie is in most cases just along for the ride.
TEAM EFFECTS ON GOALTENDING
Team effects must always be considered when evaluating goalies. Some of the key considerations are goal support, amount of shots faced, and difficulty of shots faced.
Other variables are slim chance games (described in the last post), as well as cheap wins and tough losses, statistics that measure how often the win or loss was unfairly credited to the goaltender, given his actual performance.
TEAM STRENGTH & SV%
NHL save percentages are quite dependent on the strength of the team in front of the goaltender
For the best men’s teams, there is no relationship between save percentage and shots against. This is similar to the NHL, where there are a number of different teams on a fairly level playing field.
The top teams are very close to each other in save percentage. There isn’t a single team that has a huge advantage in terms of goaltending
When the competition is tight, then the shots/save percentage relationship largely disappears.
That makes it even more important to take team strength into consideration when evaluating goalies
LOOKING FOR OUTLIERS
there is very little shot quality effect at even strength.
Goalie results are usually quite similar by team, and only the top goalies are able to maintain a clear separation in performance compared to their backups year after year.
This result on its own is pretty good evidence that there are differences between teams,
Shot quality is primarily dependent on skill rather than style.
SHOT QUALITY FANTASY
there is likely some sort of change in a team’s ability to effect shot quality from season to season. Just that, on the whole, it is tiny.
on average, the difference in shot quality from any two teams selected at random … the expected difference will be 2 goals on the season.
In short: The universe thinks that the NHL ‘Shot Quality’ metrics are a complete crock of shit
shot quality can only account for a fraction of the observed team-to-team variation in shooting and save percentage
WINS THRESHOLD
The two most important team factors that affect a goalie’s ability to win are his goal support and the number of shots he has to face.
We can calculate what I’ll call the “win threshold” for the goalies on each team by taking (shots against – goals for) / shots against. This gives us the save percentage that would result in the team ending up with an equal number of goals for and goals against over the course of the season. If the goalie’s save percentage is above that number, the team is likely to win more than they lose, while anything below the threshold means that the team should end up sub-.500 (or sub-.550 in the shootout era).
TEAM EFFECTS ON GOALTENDING
Over most of the NHL’s history, the teams that scored the most goals and prevented the most shots have also tended to have the goalies with the best numbers. It is only over the last two decades that we have seen increasing independence between team results and goalie results.
The evidence to me suggests that shot quality effects are mostly determined by differences in skill rather than differences in style of play, and therefore shot quality effects are going to be largest in an unbalanced league with large differences in skill between the top and bottom teams. That is supported by the data above, since in today’s salary-capped league we don’t see the same degree of goalie/team stat dependence, whereas in leagues that were more unbalanced because of factors like expansion, territorial rights or management competence, the goalie and team numbers are far more intertwined.
SHOT DISTRIBUTION BY GAME STATE
Although there are players with more extreme save percentages I usually consider anything above .920 excellent and anything under .900 poor. That’s only two goals every one hundred shots. There’s an awful lot of things other than “goaltender skill” that can eat that difference.
I would assume that goaltenders have very little influence on these distributions and that they’re mostly a function of team skill.
A .005 shift in save percentage at the extremes is both not much and quite a bit.
On the one hand, this is a small difference. On the other hand this is 10% of the difference between an excellent goaltender and a bad one. Such is the nature of goaltender evaluation where the difference between poor and excellent is so small.
TEAM EFFECTS & EVSV%
The correlations are scarcely distinguishable, which implies that team effects aren’t important at even strength.
Therefore, it would appear that the EV SV% of individual goaltenders is affected to some degree by team effects.
Because the evidence is mixed, I’m not entirely confident that there is a significant team effect.
SHOT DISTANCE ALLOWED AS A TEAM TALENT
People calculate shot quality differently, but the simplest way to assess its existence is to look at shot distance.
Shot distance – and by extension, long-run shot quality allowed – accounts for 5% of save percentage. Worst-case, I suppose it could be 10%, but it’s very small.
A big chunk of our 5-10% of save percentage due to shot distance is scorer bias – the true talent portion is not nothing, but it doesn’t appear to be a major driver of save percentage overall.
CAN PLAYERS IMPACT SV%
The most compelling argument that players can and do impact save % is that we see it happening all the time and it is fully accepted among the hockey analytics community. It is known as score effects
The difference in the Bruins save percentage between leading and trailing is 1.33%. This is the difference between a .923 save percentage goalie and a .910 save percentage goalie which is the difference between an elite goalie and a below average goalie. That is not insignificant. Is this the goalies fault or does it have something to do with the players in front of him? The latter seems most likely.
So, now if a team can play a style that boosts the team save percentage when they are protecting a lead, why is it so inconceivable that a player could see the same impact in his on-ice save percentage if that player plays that style of hockey all the time?
It is Difficult to Detect because Individual Players Don’t Have a lot of Control of Outcomes
The average player’s individual ability to influence of what happens on the ice is actually fairly small as there are also 9 other skaters and 2 goalies on the ice with him. At best you can say the average player has a ~10% impact on outcomes while he is on the ice. That isn’t much.
The “team” aspect in hockey is more significant than any other sport and any particular players statistics are largely driven by the quality of his team mates. Even more than teammates, style of play can be a significant factor in a players statistics.
THOUGHTS ON SV% AND SHOT QUALITY
Like it or not, a high save percentage is the ultimate goal for a netminder. Their job is to prevent shots on goal turning into goals against.
Save percentage in many ways is analogous to goal differentials for a player. It correlates to wins. In small sample it is hugely impacted by variance and near-random, making it difficult to use.
Save percentage is flawed, as it needs large samples to begin to represent talent levels. This does not mean it is a team statistic, although it does mean that one should be wary when subdividing the highly variance influenced data.
TEAM PLAY VS GOALIE STATS
(June 2015)
Using the rolling ten game averages for both Fenwick Against per 60 and even-strength save percentage we can see that there was definitely not a positive correlation between the two variables
At even-strength there appears to be no correlation between a goaltender’s ability to do his job and his team’s ability to suppress shots.
Perhaps it is time to curb the belief that the team in front of the goaltender is to credit/blame for a goaltender’s successes/failures.
there is no correlation between the play in front of a goaltender and the goaltender’s ability to play well. A goaltender can be judged on his own merit without dealing with the noise that comes from team related variables.
PLAYERS CAN AFFECT SV%
to claim that players have zero talent to influence their goalies save percentage is an extraordinary claim that shouldn’t be made lightly.
save percentage varies based on score due to how players play based on whether they are protecting a lead or playing catch up. If score effects impact save percentage there should be no doubt that different players with different roles can do so as well
The ability of players to impact a goalies save percentage is real. It is difficult to reliably detect over small sample sizes and over larger sample sizes often gets washed out by other factors such as roster changes, coaching changes, changes in role, etc. and in fact there may not be a lot of players that exhibit this talent to a significant degree but it doesn’t mean it doesn’t exist. It most certainly does
SHOT QUALITY & SV%
While the team with possession tries to maximize the probability of a goal being scored from each shot, the defensive side battles to minimize it.
Just as the scoring teams trying to maximize shot quality factors are trying to increase expected shooting percentage, the defensive team is trying to minimize these factors and lower the expected shooting percentage, in other words: raise expected save percentage.
the observed results do not act any different than what we would expect if they were random.
When you look at a player who has a particular impact on relative team save percentage, you cannot simply assume that means they are effectively causing that impact and will continue to perform like that.
SHOT QUALITY & SV% REVISITED
My theory for this is aligned with what we see in score effects. With score effects, when defending a lead (and presumably playing more defensive hockey) teams see their save percentage rise. My theory for individual players is when players are assigned defensive roles it is likely that their style of play will result in a boost to their teams save percentage.
There is strong evidence in support of the theory that players have an ability positively influence save percentage and that Sv%Rel and/or Sv%RelTM are measures of that ability.
TEAM EFFECTS ON SV%
Low_Sv% doesn’t matter because we already know that Low_Sv% means almost nothing. The correlation with Mid_Sv% is nonexistent, so we shouldn’t even bother with it either. We now have to deal with High_Sv%.
How do we account for the team effects? I think the first step is thinking about what makes up a player’s High_Sv% (or to be fair Sv% in any zone). We know a player’s own skill matters, some luck, and as we covered here team effects. So it’s kind of like this:
Observed High_Sv%= Talent + Luck + Team Effects
At the end of the day, it seems like we were mistaking a little noise for signal in a goalie’s High_Sv%. The SD of Talent is really smaller and we just have to regress a little more.
Comments