INTRODUCTION TO PLUS-MINUS
One major problem with on-ice goals plus-minus, an official NHL stat since 1967, was that a player could create a high rate of scoring chances but due to bad puck luck or hot opposing goaltending, he and his linemates might not be rewarded with an equally high rate of goals.
On the other hand, a different player and his linemates might create an average rate of scoring chances, but be rewarded with a high rate of goals.
Similar problems existed at the defensive end of the ice — and an NHL season just isn’t long enough for those rates to even out. It’s important to be aware of these issues when using on-ice goals plus-minus to rate an individual player, and most folks are, indeed, well aware of them by now.
RESEARCH ON PLUS-MINUS
(2002)
Plus-minus is not a one-dimensional statistic. It reflects the overall skill of the player. So if we can transform the raw form of plus-minus into a more informative number, we will have a very useful statistic indeed.
In order to properly examine plus-minus, we must transform it into Adjusted Plus-Minus (APM). APM is a necessary tool when examining plus-minus statistics.
Adjusted plus-minus is calculated as follows:APM = PM – EPM
APM should be examined on a per-minute basis, not only as a total. Again, this reflects the idea that players should be evaluated based on opportunity.
MAKING SENSE OF +/-
(July 2008)
This statistic does not measure defence per se. It is an attempt to measure the overall quality of a player.
It is one measure where good defence may come out, if a player appears much higher in the +/- ratings than he should be from offence alone.
The biggest problem with +/- ratings is comparing between different teams.
In order to attempt to compare players on different teams, it is necessary to establish a baseline for a given team to compare individual player with.
The theory behind this technique is to compare +/- ratings for players when they are on the ice with that of their teammates when they are off the ice.
In an attempt to find good defensive players statistically, +/- ratings are a start. This can be significantly improved by comparing them to a baseline for a given team (so that comparison between different teams is possible). Comparing a player’s +/- when he is on the ice against his team’s +/- when he is off the ice is a very good way to do this.
ADJUSTING RATINGS FOR +/-
(July 2008)
One of the flaws of the on/off ice adjustment is that it is a rate measurement and not a counting measurement.
Nobody cares who scored the most points per minute, unless they also scored the most total points.
One way to fix this is to treat +/- ratings as a counting stat and not a rate stat
WHAT CAN WE LEARN FROM ADJUSTED +/-
(Aug 2008)
It is clear that +/- ratings are heavily influenced by the team in which a player plays. A player on a really good team will almost certainly have a higher +/- than a better player on a poor team.
There are several potential adjustments one can make. I think the most reasonable first step is to measure the difference that a player makes when he is on the ice against when he is not on the ice.
INDIVIDUAL GAA
(Aug 2008)
If we want to rank defensive values of players, it might be a more useful starting point to look at the number of goals scored when a player is on the ice. It is possible to give each player a goals against average in the same manner that goaltenders get them.
Individual players have a much larger range in their GAAs. This is because, unlike goalies, many players only play in certain situations that will tend to help or hurt their numbers and goalies play the entire game.
Usually bad teams have ineffective checking lines and put their most talented offensive stars out against the top line in a game (particularly if they are losing and want to attempt to comeback). Thus, we have a list of some of the best offensive players on some of the worst teams in the league. In fact, given the chance, I would happily trade the players with the 10 best individual GAAs for those with the 10 worst.
Generally, the league leaders are players on strong defensive teams who play against weak opposition (there are a few exceptions).
INDIVIDUAL GOALS FOR AVERAGE
Clearly, offence is less team dependant than defence. More teams are represented on this list than on a defensive list. It only takes one player to escape from his check to score a goal. On defence, all players must responsibly take their checks to prevent a goal. This is why it is harder to study defensive values sabermetrically than offensive ones. Defence is by its nature a team effort, while offence is more individualistic.
scoring is often initiated by forwards and that defencemen play with far too many forwards to be at either extreme.
The players with low GFAs are usually players with little value as NHL players. Looking for players with the lowest GFAs in the league is one of the surest ways to locate players who should not be in the NHL.
ADJUSTED GFA
By subtracting the off ice GFA from a player’s on ice one we should get a better look at the offence a player brings independent of his team
We are subtracting the goal scoring rate when a player is off ice from the player is on the ice. This is intended to see the effect the player has independent of his team. Does it?
The bottom list is more shuffled than the top is, which is expected because there is more differentiation between the best players in the NHL than the worst (in general the worst players in the league are roughly interchangeable with one another).
In order to test the effect of the on/off ice adjustment to +/- that I have looked at in several posts this summer, I decided to look at a simpler system. Goals For Averages (GFA) is this simpler system. Defensive contributions to +/- are removed. Since defence is harder to measure than offence this makes a simpler system.
ADJUSTED GAA
(Aug 2008)
A better test is to apply this adjustment to the individual goals against averages since defence is a much more team dependent skill.
Clear that this list does not do a good job of identifying players who are defensive stars. It identifies players who played against low quality opposition and did an acceptable job
The problem is that it is clear that quality of opposition is much more important than team adjustments, so this list is not in any meaningful final state.
The adjustment worked better at the worst end. This is because these are players who do play against tougher opposition and are not protected from it by their team matching lines.
NEW METHOD TO LOOK AT RATINGS
(Aug 2008)
One way to look at the problem, which to my knowledge nobody has done, is to calculate a player’s goals for average and goals against average, while simultaneously calculating the time averaged goals for and against averages of the players on the ice who play against the player. In this way, it is possible to remove the issue of quality of opposition.
ADJUSTED PLUS-MINUS
Adjusted +/- Rating tells us how much a player increases (or decreases) the likelihood that a goal will be scored in the next 10 seconds if they were paired with an average lineup playing against an average lineup.
This methodology takes account of the value of all the plays that a player was on the ice for and then adjusts their rating based upon who they were on the ice with.
We think this is a better way to assess the overall performance of a player because it deals not just with goals but with all of the events associated with a player and the likelihood that those events lead to a goal.
THE USELESS STAT & RATING PLAYERS
(Nov 2008)
The problem with +/- is not that it is an inherently bad thing to measure, but that it doesn’t measure an individuals performance but rather the performance of the individual, all his linemates, the goalie, and his opponents.
Stats such as on ice vs off ice comparisons help. But these stats, while significantly better than +/-, are still flawed.
But even this doesn’t really solve the problem.
The problem still exists because Chris Draper is having his on ice stats compared to off ice stats that are racked up by guys like Zetterberg, Datsyuk, Holmstrom, etc.
What I have done is taken into account who a player has played with and against and tried to factor out any benefits they might have by playing with better than average players or against worst than average players or penalties they may get by playing with worse than average players or against better than average players. Instead of looking at how his team plays when he is off the ice vs when he is on the ice, I look at how his teammates play when they are playing with him vs how they play without him. The result is a player will get a good rating when he consistently makes his team mates better when they play with him vs when they aren’t playing with him.
If we can somehow isolate a players performance then +/- is, at least conceptually, a good stat. What I have done is instead of looking at how a player compares to his team as a whole, I compare individual players to each other when they are on the ice together vs when they are on the ice apart from each other.
PROBLEMS WITH +/-
(Feb 2009)
Plus/minus at the junior level is extremely important in predicting who will do well in the NHL. Not only for the potential superstars (or obviously goalies) but for everyone else, it, combined with points per game, is the gold standard
A significant factor in a player’s Plus/Minus rating is the team he is playing on.
More specifically than the quality of his team, the success of a player is influenced by the quality of the other skaters the coach chooses to play him with and by the skill level of the specific opposing players that he lines up against.
· Time On Ice is not taken into account.
· Plus/Minus doesn’t differentiate offensive contribution from defensive contribution.
· Important facets of a player’s game are not included in Plus/Minus (no context)
· The scale is out of whack.
Breaking it down into Plus and Minus components, it is apparent that extremely low Minus ratings are hard to sustain.
ADJUSTING FOR TEAM STRENGTH
(April 2009)
The most fundamental problem with Plus/Minus is obvious: it is highly affected by team strength. Ergo, if we want to fix it, we could simply define an “adjusted +/-“ that is Plus/Minus minus some team factor.
The impact of a team’s goaltending can and should be factored out of a player’s adjusted +/-. At this point, I will start calling our new stat RPM, because we have moved beyond the traditional definition of adjusted +/-.
RELATIVE PLUS-MINUS
One of the weaknesses of traditional +/- is that it tends to favor players on good teams while penalizing players on bad teams.
We can make a small improvement on +/- by subtracting the +/- when a player is off the ice from it. That is, if a player was +1 goal per 60 minutes on the ice and his team was even when he was off, he ends up appearing the same as a guy who was even on the ice while his teammates were -1 per 60 minutes. It’s not perfect, but it does make an adjustment for how good a player’s teammates were. This statistic has several names – relative +/-, On-Ice/Off-Ice +/-, or simply “Rating”, as I’ve called it on the stats page.
ADJUSTING +/- FOR INDIVIDUAL CONTRIBUTION
(2011)
In this paper we develop an adjusted plus-minus statistic that attempts to isolate a player’s individual contribution.
The APM statistic is considered an improvement of the traditional plus-minus statistic, which is highly dependent on the strength of a player’s team, and also the strength of the opponents he faces.
NHL ADJUSTED PLUS-MINUS
The basic goal of the APM statistics, like the other player performance metrics mentioned above, is to measure a player’s individual contribution to his team.
for this model, the results are interpreted as an estimate of the goals per 60 minutes that a player contributed to his team, independent of all the other players in the league. In other words, it estimates the goals per 60 minutes contributed by a player, independent of (or adjusted for) the strength of his teammates and opponents.
HOW USEFUL IS RAW +/-
A lot of bad +/- is driven by a low on-ice shooting percentage and a low on-ice save percentage – PDO. And PDO has zero year-to-year correlation, so it’s difficult to infer much skill from a bad +/-.
And clearly a lot of decision-makers in the NHL do get that (otherwise they’d cut their +/- trailers) but I’m willing to bet that guys with poor +/- are still undervalued.
+/- IS A STUPID STAT
Plus/Minus is often maligned by fancy stats types as not descriptive – it doesn’t take into account the quality of competition or teammates. It doesn’t acknowledge whether your team’s goaltending is great, horrible, or somewhere in between. There’s yet another reason why it’s dumb – it describes game states that are not entirely relevant: short handed and empty net goals.
Conclusion: Plus/Minus is stupid.
CORSI PLUS-MINUS
The biggest problem being that goals are a very infrequent event, especially when you are limiting them only to even strength or 5v5. Small sample size makes plus/minus a bad statistic unless over a long period of time (years).
Plus/minus has very low repeatability, which demonstrates that players have very little control over it. There are many factors that contribute to plus/minus:
WHY +/- IS WORST STAT
1. Shot based metrics are typically better than goals
2. Plus/Minus arbitrarily chooses to exclude some goals and not others
3. Plus/Minus is a counting statistic
Plus/minus doesn’t even say who outscored the most because of its arbitrary methodology, and can even give you false positives — a guy who has a positive plus/minus rating might actually have a negative goal differential. Those who have the worst plus/minus, may not even the worst plus/minus players relative to their ice time. Two players can have the same rating and be vastly different in their outscoring performance.
If you want to use goals, do not use plus/minus. Use goal% (team’s share of on-ice goals) or turn a goal differential into a rate relative to ice time instead. Also, separate out different situations, like the power play, penalty kill, and even strength, from each other.
Anything plus/minus tries to do, there is something else that does it better.
RAPM
SABERMETRICS PHILOSOPHY ON +/-
(July 2008)
I would argue that these regression based models are not significantly better than those that I am presenting.
They are more mathematically complex and may be satisfying to those who enjoy “exact solutions” (in as much as anything that comes from flawed hockey statistics can be exact), but it is hard to draw any more information from them than can be gathered from the more simple models I am presenting. Any model should be as simple as necessary to produce the results we are looking for (but no simpler).
One must keep in mind that the input parameters (the statistics) are flawed.
Most events (for example goals) are only weakly correlated with all the players on the ice.
When most goals are scored, some players on the ice were not directly involved. There will be a huge amount of statistical noise inherent in any numbers which can create significant error in even the most “exact” calculations.
REGRESSION BASED +/-
http://hockeyanalytics.com/2010/11/a-regression-based-adjusted-plus-minus-statistic-for-nhl-players/
an adjusted plus-minus statistic for NHL players that is independent of both teammates and opponents (addressing the largest of many deficiencies of the plus-minus stat).
used data from the NHL’s shift reports in a weighted least squares regression to estimate an NHL player’s effect on his team’s success in scoring and preventing goals at even strength. Both offensive and defensive components of adjusted plus-minus are given, estimates in terms of goals per 60 minutes and goals per season are given, and estimates are given for forwards and defensemen.
ADJUSTED +/- USING RIDGE REGRESSION
(May 2018)
REGULARIZED ADJUSTED +/-
The goal of this type of analysis (APM/RAPM) is to isolate a given player’s contribution while on the ice independent of all factors that we can account for. Put simply, this allows us to better measure the individual performance of a given player in an environment where many factors can impact their raw results.
Skaters are not the only thing we can measure using a regression technique – we can also measure team performance. In this case, the setup is the same, but instead of using skater offensive and defensive variables we use team offensive and defensive variables.
VENUE ADJUSTED RAPM (3 ARTICLES)
(Nov 2020)
Regularized Adjusted Plus-Minus (RAPM) model allows me to provide a point estimate of a player’s isolated offensive, defensive, and net impact on expected goals.
The end goal of this process will be to provide one point estimate of a skater’s isolated offensive, defensive, and net impact using reported shot distance, one point estimate of a player’s isolated offensive, defensive, and net impact using adjusted shot distance, and then display them side-by-side in order to provide an estimate of how adjusting for scorekeeper bias may affect a player or team’s isolated impact according to one of these models.
(Nov 2020)
Adjusted Distance = (Reported Distance)-(Rink Mean Reported Distance)
RAPM model, I will have “a point estimate of a player’s isolated offensive, defensive, and net impact on expected goals.”
This is the essence of top down hockey analysis: using on-ice results to isolate a player’s impact.
A player’s RAPM impact on expected goals against is one of the most important metrics for the defensive component goals above replacement, and the other components in goals above replacement correlate very closely with RAPM, so providing a point estimate of a player’s RAPM before and after venue adjustments will show how much a venue adjustment may impact certain components of their goals above replacement.
(Nov 2020)
But the adjustment is not so significant that it flips the world upside down and tells us that Minnesota’s goaltenders are good and Dallas’s goaltenders are bad
Since scorekeeper bias goes both ways, shouldn’t most players wind up with a similar net impact before and after venue adjustments?
Most offensive adjustments are essentially balanced out by defensive adjustment
Note: the defensive side of RAPM is far more repeatable for defensemen than offense is.
So, it’s fair to say that scorekeeper bias doesn’t play a big role in a player’s net impact on expected goal differential, and if we were only looking at this metric, scorekeeper bias would be a non-issue.
Venue adjustments can make significant changes to a player’s offensive impact, but their defensive impact will generally see changes of a similar magnitude in the opposite direction, so their net impact will be very similar.
To whatever degree scorekeeper bias leads us to overrate a team’s defense, it leads us to are subsequently underrate their goaltending (and vice-versa).
USING RAPM TO EVALUATE PLAYERS
(Dec 2020)
I’ve used regression to obtain point estimates of an NHL player’s individual impact on the following six components:
· Even Strength Offense
· Even Strength Defense
· Power Play Offense
· Penalty Kill Defense
· On-Ice Penalty Differential
· Individual Shooting
The regression isolates a player’s impact by accounting for various external factors that surround them.
This regression, however, is not a typical linear regression; rather, it is a weighted ridge regression, where the length of each shift is used as a weight, and the “ridge” serves to shrink coefficients down to zero
With regularization, we implement a “penalty” to the dataset which minimizes the mean-squared error by shrinking coefficients towards zero.
One year of RAPM was somewhat unreliable, which is why they generally incorporate prior knowledge into their regressions.
DETERMINING UNCERTAINY ASSOCIATED WITH RAPM
(Feb 2021)
Even at the 68.1% confidence interval, for most players who played 500 minutes, we can’t make a confident statement that their play driving was above or below average. If we want to be 90% or 95% confident in our statements, these numbers plummet quickly.
Not only is there a good deal of uncertainty that should be applied to how well RAPM actually measures a player’s impact in a given year, but our measurement in year 1 doesn’t manage to explain a huge percentage of the variance in our measurement in year 2.
Comments