Analytics 2 - Goals vs Shots Metrics
- tmlblueandwhite
- Dec 24, 2022
- 28 min read
Updated: Dec 27, 2022
GOALS BETTER THAN SHOTS
TEN LAWS OF HOCKEY ANALYTICS
Goals for and against are the only factors that affect winning
In any sporting event, one wins by having more credits than debits.
In a single game it is an absolute truth that the team that scores the most goals is the winner. However over the course of a season a team with a positive average goal differential only has a tendency to win more than it loses (a consequence of Law #3).
Nevertheless the predictive power of goals for (GF) and goals against (GA) is very strong. Regression analysis tells us that about 94% of winning is explained by a sophisticated model involving just GF and GA. The remaining 6% seems to just be statistical noise. All other variables simply drive GF or GA.
The sophisticated model: Winning Percentage = GF n / (GF n + GA n),where n depends on some things (but you can just assume n=2 and you are nearly there).
Goals are random events
There is a great deal of statistical proof that goals occur randomly. This does not mean that skill, strategy and execution give way to luck. It means that outcomes are uncertain, influenced by a myriad of factors including skill, strategy and execution
Winning has a nearly linear relationship to goal differential
The simple model: Winning Percentage = .500+ (GF – GA) / (g x GP),where g is the average number of total goals per game and GP is games played.
Goal differential by itself explains about 93% of winning. We get only about 1% of extra information from the more sophisticated (non-linear) model described in Law #2 (because it captures the “S” shape of the data).
Why does this matter?
This ‘linearity law’ is a huge building block for hockey analysis. It means that goals saved and goals scored have the same kind of impact. Linearity means that the game is essentially the sum of its parts. It means that individual performances are basically additive. It means that team performances can be decomposed into individual contributions.
Sample size matters
An average, modern NHL team participates in about 450 goal events, 5000 shot-on-goal (‘SOG’) events and 9000 shot-at-goal (‘SAG’) events. The statistical information from a sample is proportionate to the square root of the sample size. If there is 1 unit of information in a game outcome, then there are 2.3 units of information in goal events, 7.8 units in SOG events and 10.6 units in SAG events.
Expressed differently, there may (see Law #7) be as much statistical information in 8 games of SAG data as there is in a full season of wins/losses or 35 games of goal data.
This is also important at the individual player level. An elite goaltender might face about 2,000 shots per season. An elite shooter might take just 300 shots. So we generally know more about goaltending talent than shooting talent.
Expect mean reversion
At the team level it is pretty easy to accept mean reversion in shooting percentages. Talent and circumstances are averaged very quickly (note, however, Law #10).
Individuals, on the other hand, have their performances revert to their personal mean and circumstances. Phil Kessel and Colton Orr do not have the same mean.
This means that one source of variation in team shooting percentages is the variation in number of shots between talent levels.
Goaltending is a bit different.
At the individual goaltender level, sample sizes are much larger and we know more about true talent. At the team level we quickly observe that one player may dominate team save percentages. This means that save percentage differentials are more credible.
The caution here is that a 1% difference in save percentages (.925 vs .915) is the difference between very good and just average and that degree of discrimination in the mean requires a great deal of data to confirm.
Puck possession matters
we should take our analytic efforts to the largest mountain of data – shot data.
Rigorous analysis has shown that historic SAG data predicts future winning better than does historic winning or historic goal scoring.
This is almost certainly due larger sample sizes – the signal is more evident through the background noise of randomness – but it is also almost certainly a commentary on the quality of the underlying ‘process’ of goal scoring and prevention.
What is SAG? Otherwise known as “Corsi”, it is simply a count of the number of times the puck was directed at the net:
SAG = Shots At Goal = Goals + Saves + Missed Shots + Blocked Shots, either For or Against
SAGD = Shots At Goal Differential = SAGF – SAGASAG% = Shots At Goal Percentage = SAGF / (SAGF + SAGA)
ALL ENCOMPASSING PLAYER RATING MODEL
(Sept 2010- Part 1)
The recent trend is to stray away from goals for and against type methodologies towards shot attempts for or against methodologies (i.e. corsi analysis).
One of the key reasons for doing so is to increase sample size to reduce errors in the evaluation including errors that might be associated with nothing more than pure luck.
There is merit to wanting to achieve this goal and to some extent looking at shots attempts for/against achieve this goal but you will also find the method fail in many situations due to the fact that it treats all shots equal, which we know is not true. Some shots are simply better than others and more likely to produce goals.
Because of this I still prefer to consider a goal based approach because really, goals are what matter in hockey. If you are on the ice for more goals for than goals against, that is a good thing.
The stat that measures this that everyone knows and understands best is plus/minus but it has major drawbacks. A below average player that plays on a great team with great teammates can have a very good plus/minus while a great player that plays on a bad team with below average teammates can have a poor plus/minus.
Furthermore, a defensive specialist that goes head to head with the oppositions best offensive players may also have a poor plus/minus but still be getting the job done for the most part. The standard plus/minus stat is, in general, a very poor stat for evaluating players.
To improve on straight forward plus/minus we need to take into consideration the quality of teammates/linemates a player has along with the quality of the opposition players he plays against.
With this in mind, when I developed my all encompassing player evaluation method I choose to consider line mates performance levels when that line mate is playing with the individual being evaluated and when he is not playing with the player being evaluated.
Similarly, when evaluating strength of opponent I chose to consider each opposition players performance levels when he is not playing against the player being evaluated.
(Sept 2010 – Part 2)
In short, the system compares how many goals are scored for and against while a player is on the ice and compares it to how many goals scored for/against one should expect based on the quality of his line mates and opposition.
The first method of improvement is to utilize the additional information we have about the quality of a players line mates and opposition once we have run the model.
Initially I use the goals for and against performance of his line mates and opposition when the player being evaluated is not on the ice at the same time as his line mates and opposition.
But now that we have run the model we, at least theoretically, have a better understanding of the quality of his team mates and opposition.
I can then take the output of the first model run and use it as the input of the second model run to get new and better results. I can then continue doing this iteratively and the good news is that after every iteration the difference between the player rating from that iteration and the previous iteration trends towards zero which is a very nice result.
For the most part changes in player ratings are not significant and generally speaking the iterations tighten the results (i.e. the magnitude of the extremes are lessened) which is probably desirable and certainly something we find when we increase the size of the dataset
The second method I utilized in improving the rating system is to use a 3 season dataset as opposed to a single season.
GOAL RATES BETTER THAN CORSI/FENWICK
(May 2011)
The problem I have with many corsi/fenwick enthusiasts is that they often dismiss the influence that ability to drive or suppress shooting percentage plays in the equation.
Many dismiss it outright, others feel it has so little impact it isn’t worth considering except when considering outliers or special cases. In this article I am going to take an in depth look at the two and their influence on scoring goals on an individual level.
What we find is shooting percentage is more correlated with goal production than fenwick rate.
Shooting percentage is much more highly correlated with goal scoring rate than fenwick rate is which would seem to indicate that being able to drive shooting percentage is more important for scoring goals than taking a lot of shots.
It seems that both shooting percentage and fenwick do a reasonable job at identifying offensively talented players. That said Shooting Percentage is a better indicator of offensive talent than Fenwick For rates.
An organization that focuses on puck control dominates the corsi for statistic so I guess what that shows is that corsi/fenwick probably is a good measure of puck control. But, as we have seen, fenwick (i.e. puck control) doesn’t automatically translate into goals scored.
Shooting percentage matters a lot in scoring goals, but for the staunch corsi supporters they will argue that corsi is more persistent from season to season and thus is a better predictor of future performance. So which is the better predictor of future performance?
When dealing with a single season of data the correlation with GF20 is much better for fenwick rate than for fenwick shooting percentage. The gap closes when using 2 seasons as the predictor of a single season and is almost gone when using 2 seasons to predict the following 2 seasons.
It seems that the benefit of using corsi over shooting percentage diminishes to near zero when we have multiple seasons of data and though I haven’t tested it shooting percentage probably has an edge in player evaluation with 3 years of data.
Of course, you would never want to use shooting percentage as a predictor of future goal scoring rate when you could simply use past goal scoring rate as the predictor.
Past goal scoring rate has the same ‘small sample size’ limitations as shooting percentage (both use goals scored as it sample size limitation) but scoring rate combines the prediction benefits of shooting percentage and fenwick rate.
When looking at single seasons both GF20 and FenF20 perform similarly at predicting next seasons GF20 with fenwick shooting percentage well behind but when we have 2 years of data as the starting point, GF20 is the clear leader.
This means, when we have at least a full seasons worth of data (or approximately 500 minutes ice time), goal scoring rates are as good or better than corsi rates as a predictor of future performance and beyond a years worth of data the benefits increase. When dealing with less than a full season of data, corsi/fenwick may still be the preferred stat when evaluating offensive performance.
What about defensively?
Defensively, fenwick against rate is very poorly correlated with future goals against rate and it gets worse, to the point of complete uselessness, when we consider more seasons. Past goals against rate is a far better predictor of future goals against rate.
For forwards at least, goals against rates are by far the better indicator of defensive ability.
In summary, it should be clear that we cannot simply ignore the impact of a players ability to drive or suppress shooting percentage in the individual player performance evaluation and so long as you have a full year of data (or > 500 or more minutes ice time) the preferred stat for individual player performance evaluation should be goal scoring rate. Corsi/fenwick likely only provide a benefit to individual performance evaluation when dealing with less than a full year of data.
When we look at a list of players sorted by 4-year on-ice shooting percentages we clearly see the best offensive players rise to the top of the list and more defensive minded players fall to the bottom.
It is a definite skill level and the top players can have an on-ice shooting percentage 60-80% higher than the players with the worst shooting percentage. It’s definitely a skill, but you just need large enough sample sizes (certainly greater than a full season) to recognize it at any kind of confidence level because goals are relatively infrequent events
PERSISTENCE AND PREDICTABILITY
(June 2011)
The great advantage of corsi/fenwick has over goals as an evaluator of talent is the greater sample size associated with it. The greater the sample size the more confidence we can have in any results we conclude from it and the less chance that ‘luck’ messes things up.
Year over year shooting percentage fluctuates a lot, but that doesn’t necessarily mean that it isn’t a talent or doesn’t have persistence, it could mean that the sample size of one year is too small.
The four year shooting percentage leader board seems to identify all the top offensive players so it can’t be completely random. So what happens if we increase the sample size?
There isn’t a lot of persistence year over year but for 2 years over 2 years we are starting to see some persistence. Still not to the level of corsi/fenwick, but certainly not non-existant either, and the greater correlation with scoring goals makes fenwick shooting percentage on par with fenwick as a predictor of future goal scoring performance when we have 2 seasons of data as I pointed out in my last post.
But as I pointed out in my previous post, you would probably never use shooting percentage as a predictor because you may as well use goal rate instead which has the same sample size limitations as shooting percentage but also factors in fenwick rate.
Year over year correlation of GF20 (goals for per 20 minutes) is approximately 0.45 depending on years used and the 2 year vs 2 year correlation is 0.619 so GF20 has persistence and has a 100% correlation with itself making it as reliable (or more) a predictor of future goal scoring rates as fenwick rate with just one year of data and a better predictor when using 2 years of data.
The conclusion is, when dealing with less than a years worth of data, fenwick/corsi is probably the better metric to identify talent and predict future performance, but anything greater than a year goals for rate is the better metric and for one years worth of data they are about on par with each other.
Note: This is only true for forwards. The same observations are not true about defensemen where we see very little persistence or predictability in any of these metricts, I presume because the majority of them don’t drive offense to any significant degree
HOCKEY ANALYTICS AHA MOMENT
The only reason shot analytics work is if over a large enough sample the quality of shots averages out such that the average quality of shot for one team is more or less equal to the average quality of shot for another team.
Now the problem is, when hockey starts to incentivize shots rather than goals I am not certain that that premise will hold up.
There are lots of time a player could shoot the puck, but chooses not to because it is not a good scoring chance. If we start rewarding players on the basis of shot totals and that player starts shooting in those bad scoring chance situations the premise by which shot analytics is based on falls apart.
Hockey at its core is, and always will be, about scoring goals. The fact that shot differentials correlate highly with winning is an interesting observation, and maybe even a useful one, but to change the focus of the game to shot differentials from goals differentials is not likely a strategy that will work in the long run.
Positive shot differentials is a result of good play and not because a team chose shot differentials as their goal and achieved it. The reality is, to generate positive shot differential you need to:
· When you have control of the puck you generate an offensive opportunity from that puck possession more frequently and you give up control of the puck less frequently.
· When you do not have control of the puck you force the opposing team to give up the puck more frequently and generate an offensive opportunity less frequently.
· You gain possession more frequently than the opposition be that through winning face offs or winning the puck battles after shot attempts.
If you can win the puck battles, give away the puck less frequently and force the opposition to turnover the puck more you should win the shot differential contest.
I suspect shot differential is highly correlated with winning because good teams do those three things better than their opposition and not choose to shoot more often than their opposition.
Realizing that that shot differentials is highly correlated with winning is not the ‘aha’ moment in the sense that all of hockey should change focus to out shooting over out scoring at the cost of shot quality because that won’t work. The focus always has to be how to generate more shots from good scoring plays, not just generating more shots.
POSSESSION STATS PREDICTIVE TOOL
Since the 2008 playoffs, the team with the better 5v5close Fenwick% has a 53-35-2 record (there were 2 cases where teams had identical fenwick% to 1 decimal place). That actually makes it sound like 5v5close Fenwick% is predictive overall, not just in cases where one team is significantly better than another.
Of course, if we look at goals we find that the team with the better 5v5close goal% has a 54-34-1 record. In other words, 5v5close possession stats did no better at predicting playoff outcomes than 5v5close goal stats. It is easy to throw out stats that support a point of view, but it is far more important to look at the complete picture. That is what analytics is about.
We know that goal percentage correlates with winning far better than corsi percentage. This is an indisputable fact. It is actually quite a bit better. The sole reason we use corsi is that goals are infrequent events and thus not necessarily indicative of true talent due to small sample size issues. This is a fair argument and one that I accept. In situations where you have small sample sizes definitely use corsi as your predictive metric (but understand its limitations).
The question that needs to be answered is what constitutes a small sample size and more importantly what sample size do we need such that goals become as good or better of a predictor of future events than corsi.
I have pegged this crossing point at about 1 seasons worth of data, maybe a bit more if looking at individual players who may not be getting 20 minutes of ice time a game (my guess is around >750 minutes of ice time is where I’d start to get more comfortable using goal data than corsi data). I am certain not everyone agrees but I haven’t see a lot of analyses attempting to find this “crossing point”.
Let’s take another look at how well 5v5close Fenwick% and Goal% predict playoff outcomes again but lets look by season rather than overall.
In full seasons not affected by lockouts we find that GF% was generally the better predictor (only 2008 did GF% under perform FF%) but in last years lockout shortened season FF% significantly outperformed GF%. Was this a coincidence or is it evidence that 48 games is not a large enough sample size to rely on GF% more than CF% but 82 games probably is?
Puck possession is just one aspect of the game and puck possession analytics has largely been oversold when it comes to how useful it is as a predictor. Conversely goal based analytics has been largely given a bad rap which I find a little unfortunate.
WEIGHTED SHOTS DIFFERENTIAL (TANGO)
(November 30, 2014)
What predicts future goals best is past goals.
The reason is that shots-that-were-goals contains a lot of information about the talent of the team. Shots-that-were-saved contains SOME information, but not as much as goals. Wide shots have some information, and Blocked shots have the least.
With Blocked Shots, you can even argue that it is NEUTRAL, since it tells you a lot about the shooting team (put themselves in a position to shoot) as it does about the defending team (executed a block, but also put themselves to allow a shot). So, I figured that Shots-that-were-blocked was close to neutral, but in favor of the shooting team.
Basically, it really comes down to this as to how to weight the various shots:
· 1.0 Shots that were Goals
· 0.2 Shots that were Not Goals
As a reminder, this was my prior:
· 1.0 Shots that were Goals
· 0.3 Shots that were Saved
· 0.2 Shots that were Wide
· 0.1 Shots that were Blocked
A metric that EQUALLY weights all shots is not a good metric (I’m looking right at you Corsi and Fenwick). It first ignores our prior that says that shots-that-are-goals contains more information than non-goal-shots. Secondly, it’s not supported by empirical research.
Now, I know what you are going to say: how come all-shots correlate so much better than only-goals? That’s easy. The best way to increase correlation is to increase the number of trials. It’s really that simple. 100 shots is not as good a forecaster as 500 shots, which is not as good as 2000 shots. So, if you have 10 non-goal shots and 1 goal-shot, then naturally, the 10 non-goal shots will correlate better with future goals.
And indeed, this is consistent with the above results! Since we weight each non-goal shot at 0.2 and each goal at 1.0, and if you have 2 EV goals and 20 EV non-goals, then guess what. The 2 EV goals count as “2 trials”, while the 20 EV goals count as “4 trials”. So, naturally, the 20 EV non-goals will correlate better than the 2 EV goals. But, that still doesn’t mean you can weight both the same. Not at all.
The weights are the weights. And right now, you need to weight goals at “1” and non-goals at “0.2”.
Welcome to our new metric, the Weighted Shots Differential:
wSH = Goals + 0.2*(Shots + Missed Shots + Blocked Shots)
LESSONS FROM BILL JAMES
(December 2014)
Since goals contain (much much) more information than non-goal shots, then clearly, we can’t just go ahead and add goals to non-goals in an unweighted manner. Well, you CAN if all you care about is “possession time”. That’s (probably) a good way to do it. But, more important than possession time is QUALITY of possession time.
Since no two things can possibly be exactly equal, then you have to figure out in which direction you have to move something to make them come close to being equal. It should be obvious that a goal and non-goal shot aren’t EXACTLY equal.
So, if you had to guess which of the two you would weight more than the other, which would it be? Would you weight the goal more or the non-goal shot more? Right, it’s obvious, the goal has to get more weight. Once you accept that, the search is on.
So, the search is on for goals and non-goal-shots. And then within the subset of non-goal-shots, can we weight them differently? As a case for further advancement, we’d want to know how far from the net the non-goal shots were. Heck, even for the goals, we’d want to know that.
CORSI VS TANGO
(Pro Tango)
Goals is just shots multiplied by shooting percentage (SH%). The consensus among the hockey research community is that their studies show SH% is not a skill that carries over from year to year. And, therefore, goals can’t matter once you have shots.
“One of the first things that (many) people did was to run a correlation of Corsi v Tango, come up with an r=.98 or some number close to 1 and then conclude: “see, it adds almost nothing”. If only that were true.
It’s just another case of jumping to conclusions from a high or low correlation coefficient.
We’re already well into the .9s, when it comes to understanding hockey. Any improvements are going to be marginal, at least if you measure them by correlation. And so, it follows that *of course* Tango and Corsi are going to correlate highly.
Tom illustrates the point by noting that, even though Tango and Corsi appear to be highly correlated to each other, Tango improves a related correlation from .44 to .50. There must be some significant differences there.
Goals matter, not just shots.
That’s an important finding! You can’t dismiss it just because the predictions don’t improve that much. If you do, you’re missing the point completely.
Why does the conventional wisdom dispute the relevance of goals? Because the consensus is that shooting percentage is random – just like clutch hitting in baseball is random.
Why do they think that? Because the year-to-year team correlation for shooting percentage is very low.
I think it’s the “low number means no effect” fallacy.
LOOKING AT WEIGHTED SHOTS
(Feb 2015)
Weighted Shots, first proposed by Tom Tango, attempted to tackle this problem by changing the value given to each Corsi component (goals, saved shots, misses and blocks).
Tango’s weights, which he determined using a multiple linear regression that predicted a half-season’s Goal Differential using the shot attempt data from the other half, weighted goals 5 times higher than non-goal shot attempts (goals = 1, saved shots/misses/blocks = 0.2).
Tango’s work showed that weighting each event equally didn’t maximize Corsi’s predictive power, and that by changing the weights assigned to each shot attempt we’re able to make a better estimate of a team’s underlying talent.
And while Tango’s original analysis looked at pure/unadjusted Corsi and Weighted Shots, when we apply the score and venue adjustment methodology outlined by Micah Blake McCurdy, we see that Score Adjusted Weighted Shots (or SAwSH) are also a better predictor of future GF% than Score Adjusted Corsi at the team level
The natural extension to all this is applying the idea of weighted shots to individual players – Individual SAwSH
Forwards:
As we’d expect, a forward’s individual metrics appear to be a better predictor of future success than those his teammates’ rack up while he’s on the ice.
It’s comforting to see that goals are again much more important than non-goal shot attempts, with an individual goal being worth anywhere from 6 to 15 times more than other shot attempts.
Saved shots, misses and blocks are all in the same ballparks, although it’s interesting to note that misses receive a higher weight than either saved shots or blocks (this may be a bit of recording bias seeping in, where all saved shots are captured but only dangerous misses written down by the scorer).
What’s also notable is that assists, primary and secondary, are key predictors of future offensive success. In particular, first assists are worth nearly as much as an individual goal for a forward, indicating that playmaking and scoring ability are both valuable talents for forwards.
Defensemen – Offense
For defencemen, we see a different story as individual goal scoring is less critical than general shot generation (this is to be expected, as defencemen have less control over their own or their teammates’ shooting percentages). Once again, we see that primary assists are an important indicator, while secondary assists seem to show a lot more variance for defencemen, and don’t receive nearly the same weight they do for forwards.
Defensemen – Defense
What’s most interesting about the defensive weights is that goals against are much more predictive of future goals against for forwards than for defencemen.
While this must seem counterintuitive at first (since we’d expect defencemen to have the largest impact in their own zone), it’s easier to understand if you think about a forward who completely abandons their defensive responsibility.
A forward who has no interest in playing defence whatsoever will likely see more rush shots against when he’s on the ice (which have been shown to be more dangerous) or a greater number of odd-man situations in his own end.
Defencemen, on the other hand, don’t really have the luxury of selecting whether to backcheck or not, and are dependent on the 3 players up front to ensure that they’re not stuck defending down a man. It’s this dependence that’s likely resulting in the weights we observe – defencemen just don’t have the same level of control over results that forwards do.
What’s also good to see in the defensive weights is that blocked shots are weighted less than all other shot attempts (and in particular, significantly less for defencemen).
This difference allows us to effectively address the “Why does a blocked shot count as a negative” argument that’s often made about Corsi.
While Corsi treats a shot attempt the same whether the defenceman is able to block it or not (providing little incentive to block the shot if a player were only concerned about their stats), SAwSH shows the clear benefit of blocking that shot. It’s still a negative event, as it’s indicative of the other team having the puck and being able to shoot towards your net, but it’s a less negative event than allowing the puck to go through for a miss/save/goal.
We know that SAwSH is a good predictor of in-season results because it’s designed to be – the weights we’ve chosen are those that maximise our ability to predict half a season of goal data using the metrics we gather from the other half. The question we need to ask ourselves then is whether our metric is still the best predictor year-over-year. After all, it may be that within season stats are more “sticky” and that the weights we’ve chosen don’t produce talent estimates that are stable between years.
When we measure SAwSH at the single season level it allows us to forecast SAwSH in future years just as well as (or for forwards better than) SAC
The next question that we need to answer then is whether it remains the best predictor of future goals over longer timespans.
Two things stand out when we look at the results:
First, for forwards SAwSH is consistently a better predictor of future GF% than SAC.
Second, for defenceman, both SAwSH and SAC are equally good predictors.
In a way though, this is what we’d expect for defencemen – while we did separate out individual and teammate shot attempts in our regression, in most cases defencemen take a very low percentage of the total shot attempts that occur when they’re on the ice.
So while we have the ability to isolate an individual defenceman’s efforts, they’re often washed out by the play of their teammates. And when we throw in the fact that defencemen have little control over their own or their teammates’ shooting percentages, it’s easy to see why the results align so closely with Score Adjusted Corsi.
While SAwSH appears to be a better individual metric than SAC, it’s still not perfect, and it admittedly suffers from some of the same flaws that Corsi does. Although it’s more representative of individual talent, a player’s teammates still have a strong influence on his results, particularly on the defensive end.
Fortunately, we can take a cue from the work that’s been done to refine Corsi in looking at how to address these issues – in the same way that we have Corsi Rel, we can compute a player’s SAwSH Rel (which may be particularly useful for defensive measurements), and we can also use SAwSH to calculate Quality of Teammate and Quality of Competition metrics.
Corsi is not a bad statistic by any means – it’s still a better predictor of future success than simple goal differential, and it allows us to make better player evaluations than if we simply focused on shooting percentages and intangibles as our primary talent indicators.
It does, however, leave itself open to criticism because it treats every shot attempt the same. And that’s where a major benefit in using SAwSH lies – weighting shots makes sense intuitively, as we know that players can influence both their own and their teammates’ shooting percentages, and that we should be giving credit to players who block shots rather than let them go through.
SAwSH allows us to do these things, without introducing the sample size issues that we see by filtering out events we know to be important. Using SAwSH isn’t an indictment of Corsi as a metric, but rather a vindication of the effectiveness of possession based approaches, and proof that they can form the basis for the next generation of hockey statistics.
GOAL BASED METRICS BETTER THAN SHOT METRICS
(Aug 2016)
In the sport of hockey the advancement of analytics has lead to a decline in the use of goal-based metrics, and an increased reliance on shot-based metrics.
I tested assumptions behind this trend by using statistical modeling of 10 years of NHL data to directly compare the effectiveness of goal versus shot-based metrics at predicting team success, and comparative hypothesis testing to determine how well goals and shots quantify player contributions to team success.
Goal-based models consistently outperformed their shot-based analogs.
Models of team goal differential successfully predicted winning % during the 2015-16 season, while shot differential did not. Goal-based metrics (i.e. relative plus-minus/minute of ice time) were also better than shot-based metrics (i.e. relative Corsi/minute of ice time) for evaluating individual player contributions to team winning %.
These results show that team and individual performance is not correlated with all shots, but only those shots effective enough to result in goals.
There is an increasing use of shot-based metrics, such as Corsi (net-shots directed at goal) and Fenwick (Corsi with blocked-shots excluded) values for both teams and individuals, and a subsequent decline or even abandonment of goal-based metrics, such as individual plus/minus (eg. Gramacy et al., 2013). Driving this trend may be a number of known or assumed advantages of shot-based metrics, which include the following:
· Hockey is a low scoring game.
· Shots are plenty, goals rare, so shots give a larger sample size.
· Shooting and Save percentages regress to the mean.
· Shots are a good proxy for possession.
· Teams that shoot more, win more.
Assumptions 1-4 lead to the inevitable conclusion that if team shot-differential predicts winning, players who make the largest positive contributions to that shot differential are therefore contributing the most to team success.
Along with these perceived and assumed advantages of shot-based metrics, goal-based metrics, including individual plus/minus, seem to have largely fallen out of favour for the following reasons:
· Team quality has an inordinate influence on individual plus/minus, in both directions. Plus-minus does not adequately quantify what it is intended to
· Individual plus/minus measures derived from goals for and against are also assumed to be inferior measures of individual contributions to that long-term team success.
However, and obviously, winning or losing a hockey game directly depends only on how many goals each team scores, not how many shots each takes. The vast majority (90 to 95%) of shots do not actually contribute to team success at all, while every single goal directly impacts the game result.
In essence, while using shots may increase the quantity of data compared to data based on goals, the quality of the data decreases.
Objective: What are the best metrics for predicting team success? And are individual contributions to team success best measured by goal based or shot based metrics?
The top model ranked by AIC was goal differential in all situations, followed by goal differential at 5 on 5. The top two models were composite goal-based models derived from two other statistics – goals allowed, and goals scored – which each represented the top 2 models using non-derived statistics. Each of the 4 goal-based models ranked higher than their analogous shot-based models.
Defense based models (goals allowed, shots allowed) outperformed offense based models (goals scored, shots taken).
The best goal-based model used team goal differential [winning % = 0.5517 + (0.1811*goal differential)], while the comparative shot-based model using shot differential [winning % = 0.5517 + (0.01527*shot differential)] was only the 6th ranked model
When these two models were applied to data from 2015-16 winning % predicted by the goal-based model was correlated with actual 2015-16 winning % (F1,28 = 12.29, P < 0.005), but winning % predicted by the shot-based model was not significantly correlated with actual 2015-16 winning % (F1,28 = 3.37, P = 0.077; Figure 2).
Over a 10 year span, winning % was significantly different between the 30 teams (ANOVA; F31= 3.57, P < 0.0001), but the number of shots taken (ANOVA; F31 = 0.64, P = 0.93) and shots allowed (F31 = 0.87, P = 0.67) were not. In other words, both winning and losing teams take a similar amount of shots over time, showing the lack of a relationship between winning and shot-taking.
On a game-by-game basis there was no difference in the mean winning percentage of teams outshooting their opponents (0.504 ± 0.007%) compared to those teams that were outshot.
Despite the recent trend towards shot-based metrics for evaluating team and individual success, I found that comparable goal-based metrics consistently outperformed shot-based metrics at predicting team success and individual player contributions to that team success.
The best single predictor of team success was the amount of goals a team allows, while the best overall model predicted team success using goal differential.
Of all single parameter and composite parameter models, those incorporating goals invariably outperformed those using shots. The “shots for” model was not even as highly ranked as the “faceoff wins” model.
Teams that gave the most ice-time (whether purposefully or not) to forwards with high relative goals/minute had significantly higher winning % than those that gave the most ice time to forwards with low relative goals/minute. Teams giving the most ice time to players with the best relative shots/minute did not have any higher team winning % than teams allotting ice time randomly.
Neither relative net-goals nor net-shots were useful for evaluating individual defensemen, suggesting that defenseman evaluation should be based more on metrics not related to the production or prevention of shots and goals.
Over a period of 10 seasons, both save and shooting % varied amongst teams, and correlated with winning %, which also varied. What did not vary over time were the mean shots taken and shots allowed. This forces us to conclude that teams take and allow about the same amount of shots, year after year, but what determines whether they win or not is how many of those shots result in goals.
All our tested metrics were actually significant predictors of team success, and our models simply identified which were the best of this demonstrably good bunch.
However, when shot-based metrics are used a proxy for offensive zone possession, possession is itself is being used as a proxy for goals.
Perhaps instead of abandoning goal-based metrics, the most useful approach to improving our ability to predict team and individual success may be to simply improve our goal-based metrics.
The growth of shot quantity based metrics for evaluating teams and individuals has the effect of encouraging shot quantity only, and merely hoping that shot volume will translate into more goals. Instead we should be promoting analytical methods for hockey that better reflect a game whose sole purpose is to score more goals than your opponent, regardless of how many shots are taken.
SHOTS BETTER THAN GOALS
Hockey is a game of scoring goals and preventing the other team from scoring them. But goals are relatively rare events in hockey, so relying purely on goal-based stats to predict future performance is dicey.
If we can’t rely on goals, though, we need something that’s more common and acts as a reasonably good stand-in. For many of hockey’s advanced stats, that’s puck possession. After all, you can’t score if you don’t have the puck, and you can’t be scored on if you do.
WHY SHOT STATS ARE BETTER THAN GOALS
Success at scoring and preventing goals in hockey, like every activity, is a combination of skill and luck. For some things, e.g. roulette, luck is the dominant factor. In others, like sprinting 100m, skill overwhelmingly wins the day. Hockey falls somewhere in the middle, perhaps closer to roulette than anyone would care to admit.
I’m using skill loosely here to refer to any skills that help a team score goals and prevent them, including those like grit and mental toughness that pundits love to talk about.
Goal percentage in the first half of the season is only weakly correlated to goal percentage in the second half. It’s pretty clear that putting up good scoring numbers 5-on-5 with the goalies in net in the first half of the season doesn’t mean much in the way of predicting performance in the second half.
The relationship between Corsi percentage in the first half of the season and goal percentage in the second half is far stronger.
We want to know how important goals in the first half are once you take Corsi into account, and vice-versa. The regression makes it very clear that Corsi% is a far, far better predictor of goal% in the second half than first-half goal%. Not only that, it appears that virtually all of the tiny amount of explanatory power you get from goal% comes from the fact that goals are a type of shot.
Corsi is like goal percentage, but for all types of shots, including missed shots and blocked shots. Here is the scatterplot:
The correlation is 0.36, which is statistically significant. Keep in mind that we’re looking at how shooting ratios in the first half relate to goals in the second half.
When the regression spits out a formula, the size of the coefficient tells you how big its effect is.
When both first-half goal% and Corsi% are included, the goal% coefficient is a minuscule 0.007. For the stats nerds, the standard error is 0.087 so the p-value is an astonishing 0.936. This is about as statistically insignificant as it gets.
For comparison, the coefficient for Corsi% is 0.550 (SE of 0.142, p < 0.001) which is very strongly significant.
If you have a team that breaks even on goals in the first half of the season but Corsi outshoots its opponents 60-40 then they will average about 83.3 goals scored and 66.7 allowed in the second half of the season (assuming 150 total 5-on-5 goals, which is close to the league average).
If instead you have a team that was even on shots but won the goal battle by that much then they will average 75.2 goals in the second half and concede 74.8.
Once Corsi is taken into account, goals do not at all predict future success
PAST GOALS MORE PREDICTIVE OF FUTURE GOALS
(Oct 2013)
What we see is that past goals is more predictive than past wins, in determining future wins, in ALL slices of time.
What was more interesting was that the shot percentages was more predictive than goal percentages, with respect to future wins.
Now, goals did gain more meaning relative to shots, the more data we had, but, it never got to the point that goals was more predictive than shots, with respect to wins. It did seem though (if we extrapolated) that once you have a year’s worth of data, and maybe a bit more, then it’s goals, not shots, that takes the lead, in predicting wins.
CORSI vs TANGO
(Anti-Tango)
Score-adjusted Corsi is currently our most correlative stat for predicting future goals and wins.
As of late, score-adjusted Corsi has gained traction in the analytics community, and for good reason. Score-adjusted Corsi basically takes into consideration the various game states. It is currently our most predictive stat of future goals and wins, and thus has been used more as of late than raw 5v5 corsi, and has rendered 5v5 score close Corsi (which ignores a portion of data from game states) relatively obsolete.
“Tango,” or weighted shot attempt differential, doesn’t provide enough goal data points to move the needle. The correlation between the two is ~1, which would be exactly the same. So it would be a lateral change at best, and unduly confusing at worst, to even adopt it.
SCORE ADJUSTED WEIGHTED SHOTS
One of the problems with how wSH is formulated though, is that it aimed behind the current state of Hockey Analytics.
As Micah McCurdy has illustrated, Score Adjusted metrics vastly outperform standard possession metrics, since both the location of the game and the current score state have significant impacts on how teams perform. Unless we take score and venue effects into account, even an improved metric like wSH is missing important information.
Fortunately, if we follow along with Micah’s original methodology, we can figure out the appropriate adjustments to bring these factors into wSH.
First calculate the probability of a given team recording an event based on the event type, score state and game location (home/away):
Then, we can take the probabilities, along with Tango’s wSH weights (1 for goals, 0.2 for shots/misses/blocks) and combine them to calculate weighted adjustment factors for a Score Adjusted Weighted Shots metric (SAwSH).
SAwSH does a better job of predicting out of sample Goals For % than Score Adjusted Corsi.
This makes sense of course, since SAwSH includes goal scoring/goaltending data where SAC doesn’t.
The difference between SAC and SAwSH is also interesting to note: we seem to be able to explain ~5% more of the variance in out of sample GF% by using wSH rather than Corsi, illustrating the fact that shooting percentage and save percentage do matter at the team level.
While they’re obviously not as important as possession (after all, we still do fairly well using only SAC), there’s clearly a benefit to including them our analyses.
Yorumlar