ESTIMATING PROSPECT SUCCESS
One of the earliest forms of statistical analysis of prospects is The Projectinator,
The first model used to predict future prospect success was based on Desjardins’ NHLe model.
The largest issue of using this as a draft tool is that the average player leaving one league is not always in an equivalent situation to another.
PCS, and similar models, worked around this issue by looking at how players statistically similar to the prospect performed,
PCS started with the variables height, scoring, and age, as they were the strongest signals to future success.
The distribution of situational scoring is one of the main factors included in SEAL adjustments (the ‘S’ is for situational),
THE PROJECTINATOR
(March 2009)
points-per-game, adjusted for league scoring context (PPG+), in the player’s draft year. Surprsingly, this number all by itself does a better job of predicting Career than draft position does
Without any other adjustments, this single stat seems to do a better job of projecting the success of OHL forwards than NHL scouts do as a whole
(Part 1 – June 2009)
(Part 2 – June 2009)
Age: This is a very important factor in evaluating players, especially the “early birthday” players.
This effect is quite pronounced. For most players born between September 15 and December 31, a significant adjustment is required to avoid overestimating their value.
Goals per point, assists per point: Many players who are overrated based solely on their points-per-game rate have a very high assist-per-point rate.
This implies he was more reliant on his teammates to produce his point totals
Penalty minutes: Penalty minutes are a small but significant factor.
Team offense: Players who play with high-scoring teammates can have inflated scoring totals,
Height: Based on the work I’ve done so far, there does not seem to be any reason to incorporate an adjustment for a player’s height.
NHL EQUIVALENCIES
AN ARGUMENT FOR NHLe
NHLe is an equivalency formula used by some in the hockey analytics community. It’s a method of standardizing scoring across various major and junior leagues. Standardized scoring gives an idea of how players, generally younger prospects, perform at the NHL level.
The knock on using NHLe as a method to project how a player is going to perform in the NHL is that it’s rarely completely accurate.
The higher a player’s NHLe and the higher they are chosen (often the highest NHLes go first … but not always) the higher their eventual PPG in the NHL should be. That’s the main take-away.
Regardless of how you slice
It, junior/developmental league scoring is, in my mind, the single biggest predictor of future offensive performance
PREDICTING FUTURE NHL SCORING SUCCESS WITH NHLe THRESHOLDS
Over 90% of players (62 of 69 players) that went on to be impact point producers had recorded a 30+ NHLe at least once, before turning pro.
So to even make it to the NHL and produce at all, a player will very likely record a 30+ equivalency
kids who make big jumps year over year are more likely to be NHLers. Kids who only make modest gains are less likely to make the show or be impact players
We all know that scoring very high point totals as a younger junior player is far more predictive of future elite talent than scoring high point totals later in junior.
Of the IPPs that scored at least one 30+ equivalency, 90% did so before they turned 20 and 70% did so before they turned 19.
A player almost certainly needs to have reached 30+ equivalency at some point if he is expected to make the NHL and score at all, while the earlier and more frequently they’re able to the more likely the player will go on to produce significantly in the NHL.
If we up the threshold to 34+ while keeping the other parameters the same … 100% of the players made the NHL, 100% turned into at least average point-producers and 70% turned into impact point-producers.
LEAGUE EQUIVALENCIES
NHLe WRITEUP
The basic premise is to create a network of transitions that trace pathways to the NHL from various developmental leagues and aggregate the resulting cumulative translation factors.
Take all players that played in each of two leagues in adjacent seasons, and bucket their production in each league. The ratio of those per-game productions is the NHLe for that league.
INTRODUCTION TO NHLe AND THE NHLe THRESHOLDS ANALYSIS
NHLes provide a guideline for how a player new to the NHL can expect to produce in their first season in the NHL. But also, when applying the overall translations from each league to each player, season over season, they provide a clean way to standardize player scoring and rank them.
In each ventile, I’m looking at how many turned into NHLers. You can see that the top ventile is picking up ~80% NHLers, then ~70% NHLers, then ~40% NHLers, etc. So it’s doing exactly what you’d expect. At the top end, nearly every player is an NHLer and past the 10th ventile (50th percent) there’s very few NHLers.
We can see that the top 5 percent of all scores pick up on ~60% of all stars, the next ventile picks up another ~20%. The next ventile another ~10%. The model is picking up 90% of all stars in the top 15 percent of the data.
NHL EQUIVALENCY AND PROSPECT PROJECTION MODELS
(June 2021)
(June 2021)
(June 2021)
(June 2021)
This methodology is very effective in its own right, but suffers from three issues:
· The development which players undergo between year 1 and year 2 is effectively “baked in” to the model, which skews it in favor of leagues with younger players
· Using the average points per game in each league makes the data more susceptible to being skewed by extreme values and does not take sample size into account.
· This method only works for leagues that directly produce NHLers in the following season. In addition, while a handful of leagues do produce NHLers in the following season, many of them produce such a small amount that one outlier in either direction can heavily throw off the final estimate.
This process was simple: An NHLer is a player who has met the classic criteria of at least 200 NHL games played and provided a positive career WAR
final definition of a star: A skater in the top 18.5% of career WAR per game rate among all skaters who have played at least 82 NHL games.
With a universal measurement of the scoring proficiency demonstrated by prospects alongside the eventual outcomes of these prospects’ careers, I had everything I needed to build out the final model: One which leveraged scoring as prospects (as well as a few other less important variables in height, weight, and age) to predict the outcome of a player’s career at the NHL level.
As it turns out, the hardest thing for scoring rates as a prospect to predict is whether a defenseman will become an NHL star;
The rate at which a defenseman scores as a prospect is very important, and I greatly recommend against using high draft picks on defensemen who aren’t proficient scorers.
PROSPECT COHORT SUCCESS MODELS
DRAFT THEORY ON RISK AND REWARD
In simulated re-drafts, teams that utilized cohort success rates along with NHL equivalency adjusted points/game scored 23% more points than the players actually drafted by the NHL teams
Through the use of PCS% and PCS Pts/GM we can estimate both the risk, relative to draft position, and reward, relative to the player’s closest peers who have come before them.
THE PROSPECT COHORT SUCCESS MODEL
The idea behind PCS is that you can take a player and generate a list of comparable players (aka: “cohorts”). Knowing these comparable players, we can look at their success in other leagues beyond junior to estimate the likelihood of a current prospect becoming an NHLer, and what kind of player they could become.
We currently are using three main attributes which we have found to be predictive of a prospects success: age, scoring rate, and height.
What these values tell us is that we can look at a prospect of any age and in any league, and estimate their likelihood of graduating to the NHL as well as their most likely level of production as a player.
PROSPECT COHORT SUCCESS EVALUATION OF RESULTS
The underlying theory behind this system is that if you assemble a cohort of the closest comparable peers to any given player, using the variables we know to be statistically significant for draft age players in the Canadian Hockey League (age, height, points per game), that cohort peer group can be inform what type of career we can expect that prospect to achieve.
PCS has a stronger correlation with both NHL games played and points for forwards than defensemen, which seems to confirm the theory that drafting defensemen, especially earlier in the draft is inherently riskier than selecting forwards.
PROSPECT GRADUATION PROBABILITIES SYSTEM
PROSPECT GRADUATION PROBABILITES SYSTEM
Like PCS, the goal was to compare present players with past players based on a few key factors that are known to correlate with NHL success: age, stature and production.
In pGPS, similarity between players is measured by the distance in Euclidean space, where age, stature and production are the three points of imaginary shapes some distance apart. The closer the players are in Euclidean space, the more similar they are. Players with a high degree of similarity are deemed to be compatible matches. Each match is cross-referenced with the NHL’s all-time data, and a series of results are formed.
The main components of pGPS aim to look at players who are alike, based on league, height, weight and adjusted points per game, and then observe how successful that cohort is at producing NHL players. A probability is assigned to a player in the cohort by dividing the total successful NHL players of the cohort by the total number of matches.
PPer
NHL PPer PROSPECT MODEL
PPer is a regression based approach. Instead of using points per game to predict NHL success, I use the following features:
· Age
· League
· Era Adjusted Goals / Game
· Era Adjusted Assists / Game
· % of Team’s Goals / Game
· % of Team’s Assists / Game
Now that we have NHL probabilities and expected measurements, we can cluster our players to create the “Peer group” of the PPer model. Similarity scores are estimated by looking at players’ height and weight and NHL probability.
DEV
(June 2017)
The goal of DEV is to arrive at a one number rating for the expected value of a prospect at the time they are drafted. To do this, it’s required to know a given prospects chance of becoming an NHL player and how they would likely do if they were to make the NHL. Then, the product of these two metrics can be taken to arrive at our final expected value at the time of the draft.
DEV = NHLc + xPS. This is their value in terms of the point shares per season they are expected to add to the team drafting them.
DEV: With our two models, NHLc and xPS, complete we now have the components necessary to arrive at the final expected value of a prospect. By taking the product of NHLc and xPS for a prospect, we receive that prospect’s DEV. Thanks to this approach, our final DEV ranking accounts for a prospect’s chance of making the NHL as well as the significance of the impact they are likely to have if they do make the NHL.
SEAL ADJUSTED SCORING
SEAL ADJUSTED SCORING AND WHY IT MATTERS FOR PROSPECTS
This is why I would suggest not looking merely at models like PCS in prospect analysis, but also including overall player performance models. These would be things like adjusted scoring and, hopefully in the future, shot-based differentials like Corsi or Expected Goals.
Secondary assists have a lot more noise to them. They are heavily volatile year-to-year, and scorekeepers are notoriously biased and inaccurate in applying them equally between different arenas. That said, they still provide value, even if it is not as strong.
Era adjustments account for the goal rate differences between different teams, but there is still a difference in quality that separates leagues beyond their scoring rates. For this reason, I added a league adjustment with a similar method to Desjardins’ NHLEs, although looking at multiple league translations beyond NHL while holding age constant.
SEAL ADJUSTED SCORING FOR FIRST TIME ELIGIBLE PROSPECTS
S ituational
E ra
A ge
L eague
SEAL adjustments are a great way to compare the scoring rates of one prospect to another, no matter what league they occur in. While pGPS is useful at determining the rate of success that individuals have previously had within similar parameters, SEAL adjusted scoring allows for more succinct, contextual comparisons and ordered rankings from a large list of players.
コメント