Shots 4 - Skill vs Luck
- tmlblueandwhite
- Dec 25, 2022
- 6 min read
Updated: Jan 26, 2023
SKILL VS LUCK
· Injuries,
· shooting and save percentages (which is called PDO when added together),
· their power play and penalty killing percentages (which is called Special Teams Index when added together),
· their record in one-goal games (regulation time only),
· Their record in overtime and shoot-out games.
OUTPERFORMING xGF IS A SIGN OF LUCK, NOT SKILL
outperforming expected goals is more likely a sign of luck than a sign of skill.
EVEN STRENGTH SH%
Teams do indeed differ in their underlying shooting percentage at EV. Nonetheless, this variation is only very slightly larger than what would be predicted by chance alone. The underlying differences appear to be minimal.
SH% AND FALSE PERCEPTIONS
(July 2009)
The pattern of EV shooting% is very nearly identical to that expected by chance alone,
A top six player will probably get about 130 EV shots in a season. Less than that if he is playing on a weak team, more than that if he is playing on a territorially dominant team. And a 10% EVshooting% ability is about what you’d expect from a top-six-ice-time type of forward.
while we can predict the behavior of the population, we can’t predict which ones will throw more than their expected number of bullseyes.
PLAYERS CANT OUTPERFORM EXPECTED SH%
(May 2010)
I looked at whether some players exhibited an ability to score more often from a given spot on the ice than other players. This “talent” regressed 75% to the mean
A shooter is basically a shooter…
COMPONENTS OF SH%
(May 2010)
A player’s ability to get to his shooting locations regresses just 11% to the mean – in other words, a player controls where he gets opportunities from, and it is an inherent part of his play, unchanged from game-to-game:
the components of shooting percentage:
- Ability to get to a location to take a shot = 24%
- Transient ability in getting to that location = 3%
- Shooting talent (ie – ability to exceed average shooter performance) = 19%
- Transient ability to score = 54%
The bottom line is that a player doesn’t even control 50% of his shooting percentage in the long-term.
GOAL SCORING & TRUE TALENT
A player’s goal-scoring rate – on a per-game basis – is about 65% driven by his true talents – whether he’s a shooter, where he shoots from, and his ability to pick corners from a given spot on the ice over and above other players. The other 33% is shooters making their shots or whatever you want to call it.
DO GOOD TEAMS CREATE THEIR OWN LUCK
(Dec 2010)
The correlation is fairly low but a correlation exists. Maybe good teams can generate their own luck.
Regardless of what ‘skill’ we look at there does seem to be a small positive correlation between how good a team is and how good their luck is
EV SH% AND TEAM QUALITY
even strength shot ratio only accounts for roughly 30% of the variance in outscoring with respect to a single NHL season
However, because goals in the modern-day NHL are relatively rare events, a substantial proportion of the team-to-team variation in seasonal goal ratio can be attributed to luck.
For both shooting and save percentage, the adjustment is significant as luck accounts for a majority of the variation in respect of both over the course of a single season
For shot ratio, however, the adjustment is less severe as the impact of randomness is comparatively smaller. Consequently, as the sample size increases, so too does the correlation between shot ratio and goal ratio.
a team’s EV shot ratio with the score tied serves as a reasonably good indicator of how it can be expected to perform over the long run.
It seems that as the level of parity between teams has increased, even strength shooting has become even more important.
FUTURE PERFORMANCE NOT CONSISTENCY MATTERS
Consistency during the regular season is fool’s gold, especially when 70% of scoring is due to luck.
THE LUCK INVOLVED IN SCORING
This analysis gives us an estimate of 47-54% of goals are observed luck. The results agree pretty well with statistical methods that estimate that luck contributes to 38% in a team's standings and 33% to individual scoring.
TEAM EVEN STRENGTH SHOOTING TALENT
There are many more very good teams than very bad ones when it comes to even strength shooting performance.
LUCK, SKILL, SMALL SAMPLE SIZES
The average team shoots at an 8.1% clip 5-on-5, with a standard deviation of 0.48%. Going by this, a team that is good at shooting, let's say 7th or 8th best in the league, would have a true 5-on-5 shooting percentage of something like 8.42%.
On the other hand, a team that is bad at shooting, say 7th or 8th worst in the league, would be expected to score on about 7.78% of their shots.
Luck plays a huge role for all reasonable sample sizes. This is the fundamental reason why shooting stats are better than goals. Luck is less of a factor for number of shots taken than number of shots made, so they are more reliable indicators of skill
HOW MUCH OF SH% IS SKILL
How Much of Shooting Percentage Is Skill?
The correct answer, as it is for most questions, is "it depends." In this case, on sample size.
At the earliest points in the season shooting results are 90% luck and 10% skill. This is likely an underestimate, as I’ll discuss below. Over a whole season it goes to a little over 60% random chance. It takes about 140 games worth of shots for results to be 50/50.
Think of the above estimates as lower bounds on how much luck explains variation in shooting percentage or an upper bound for how much skill matters
LUCK VS SHOT QUALITY
even though shot distance and shot location/quality are persistent, shooting percentage is very luck-driven
Bottom line: shot distance/location/quality is just a tiny sliver of shooting percentage (both for and against.)
shot quality accounts for just under 10% of team shooting percentage.
IS SH% PREDICTIVE
over a season or less worth of data team shooting percentage is mostly driven by luck. It is not very sustainable, in other words
There is very little correlation between shooting percentages in one period of time and another,
Stats such as shooting percentage, save percentage and the sum of these, referred to as PDO, show very high regression to the mean.
A related but separate question is whether shooting percentage is predictive of future scoring.
All signs would seem to point to no.
shooting percentage does turn out to have both statistical and, I would argue, actual significance in predicting future scoring.
I want to emphasize that while shooting percentage appears to be a significant predictor, shooting rate is stronger.
A team with a higher rate but average percentage is predicted to score more goals than one with an average rate and higher percentage.
To me shooting percentage indicating something about shooting rates in the future is pretty strong evidence that if either it’s about shot quality and not sniping ability.
I have found that on an individual level shooting percentage is predictive of future goal scoring rates so I am not surprised to see the same at the team level.
IS LUCK MORE IMPORTANT THAN PUCK POSSESSION
Fenwick Close % is used as a measure of effective possession of the puck, and it shouldn’t be surprising that it correlates with winning: to consistently dominate other teams in Fenwick Close, your team effectively tilts the ice away from their goal, implying that they’re good at things like zone exits, forechecking, and minimizing turnovers.
Puck possession may be a reproducible skill indicative of high-quality play, as opposed to volatile skills like shooting percentage or save percentage, but is it better to be good than lucky?
PDO (Sh% + Sv%) has a far greater impact than Fenwick Close on wins and points. Specifically, a 1% increase in Fenwick Close is only good for 0.12 more wins, while a 1% increase in shooting percentage or save percentage is worth about 3 additional wins.
if you’re inclined to view PDO as a proxy for luck and Fenwick as a proxy for “good at hockey”, then apparently it is better to be lucky than good.
The only statistically significant coefficients in these models are those on Fenwick Close and save percentage. So, if you want to know which teams are likely to do well (or terribly) next season, these analyses suggest you should focus on possession and goaltender quality. Maybe it’s better to be good after all.
TRUE TALENT LEVELS FOR NHL TEAMS
How much of the difference in team shooting percentage (SH%) is luck, and how much is talent?
Either way, it doesn’t mean that SH% is meaningless. This analysis suggests that teams who have a very high SH% are demonstrating a couple of 5-on-5 tied goals worth of talent
So, instead of using Corsi alone … just add CF% and SH%! That only works in 5-on-5 tied situations – otherwise, it’s ruined by score effects.
ESTIMATING SHOOTING TALENT
Observed Sh%= Shot Quality* + Shooter Talent + Randomness
So what we need to do: Is estimate the shot quality for each player, and regress for randomness and what should be left is our best estimate of a player’s shooting talent.
Sh% includes both location measures and actual shooting talent.
Comments