Create Account

Between Two Ferns with HO: Shooting Accuracy and Range
#31

10-13-2022, 08:44 PMItsFrenchie Wrote:
10-13-2022, 02:44 PMCanadice Wrote: Yes this is also an issue.

I did some things. First I took the almost two seasons of FHM8 data (S66 and S67) we have and combined it with the last two seasons of FHM6 data (S64 and S65). I added two columns from the index data, one indicating positional group (D or F) and one indicating the engine used to produce that data. Then I went to work on some visualizations.

First we have the two engines' data overlaid for each position in a scatter plot. FHM6 is orange and FHM8 is black. I also added a linear regression line (straight line) on top. It looked like Jeffie added a spline (non-linear line) on the original graphs that is better at modelling details in the data so the lines will look a bit different. The straight lines will not be able to find as many details in the relationship between the two variables but works for my current visualizations.

[Image: unknown.png]

The image shows that there has been a drastic increase in "base" level of points for Forwards, while there seems to have been a decline in points for Defensemen. One note is that there are more players overall in FHM8 with less than 500 applied TPE which could affect the estimates and are probably not going to be the general population we want to look at at the SHL level.

I then filtered out those players and redid the graph:
[Image: unknown.png]

Now the difference between the engines becomes clearer, the defense sees a small reduction in points at the same TPE level while forwards see a large increase in points at the same TPE level.

One of the options listed in the original post was to increase the Shooting Accuracy limit for Defensemen. To visualize this, I created a boxplot for each Shooting Accuracy and grouped them per position (top is D, bottom is F) and engine (left is FHM6, right is FHM8).

[Image: unknown.png]

First looking at the FHM6 data, we can see that Forwards saw a big uptick in points produced when increasing their Shooting Accuracy from 12 to 14. In FHM8 we see that Forwards with 12 in Shooting Accuracy already produce at the same level as 14-15 in FHM6. As for Defensemen, having Shooting Accuracy at 9-12 seems to produce about the same in both engines, maybe a small decrease in the median (the line inside the colored box).

One would expect that Shooting Accuracy affects Goals more than Assists. Being able to hit the net would probably give you a better chance of scoring compared to getting an assist. One could maybe argue that it would lead to an increase in rebounds that would produce assists, but regardless I wanted to look at the two ways you can produce points.

First comes goals:
[Image: unknown.png]

Then comes assists:
[Image: unknown.png]

then comes a baby in a baby carriage...

For forwards the same trend can be seen here between the engines as we saw in points. There is an increase in goals when increasing Accuracy from 12 to 15 in FHM6 while the same cannot be said for FHM8. The increase comes instead at around 15 to 17, which can also be seen in FHM6 up to 18. Defensemen do see an increase in goals from 10 to 12 in FHM8 compared to FHM6 where it's more stable (or at least a lower slope in trend).

Assists don't really see any trend for defensemen in either engine but forwards show the same trend that we've seen in the other graphs. This trend is most likely due to confounding effects (other variables), i.e. when the player has put in TPE in Shooting Accuracy, they have most likely put in TPE in other attributes that affect the ability to produce assists.

What does this all mean? It's a really tough situation to solve. Without doing specific controlled simulations where we can control the attributes of the players, there is no real way to know the real causal relationship between an attribute and a performance metric. Observational studies where we just take measurements of the attribute values and performances will never be able to find whether or not the correlation between two variables are because of one another or if it exists purely by chance. A good example of this is that there is a positive correlation between the number of movies Nicolas Cage stars in during a year and the number of drownings in a US pool that same year. These two variables have nothing to do with one another, but there is a correlation between them nonetheless.

As Shooting Accuracy is the only attribute that has different limits for the two positional groups and is described as an offensive attribute, I think it be worth to test allowing Defensemen to get to 13 or even 14 while keeping the average number of goals per game setting the same. Hopefully the trend we saw in the boxplot of goals vs Shooting Accuracy in FHM8 can continue and we see more goals scored by Defensemen.

After completing this writing I also saw fellow stats enthusiast @Jumbobone19 looking at number of shots as a statistic so I went ahead and added a final graph of that performance metric.

[Image: unknown.png]

It looks like overall the number of shots is lower for both D and F at all levels of Shooting Accuracy. I then had to do a plot on shooting percentage where it looks like FHM sees a higher shooting percentage overall for all Forwards at the same levels of Shooting Accuracy.

[Image: unknown.png]

So the theory is not as conclusive as the original post made it sound to be with this data analysis. Although it highlights a few trends that may lead us to believe this, I think overall its still pretty close and varies from a metric to another. I think you are correct at this point in suggesting we go to a trial phase to see if the suggested changes truly have an impact on defensmen scoring goals or not.

With that being said, I am truly impressed by all this, kudos my friend! You should get a special pay for this, its amazing.

For clarity, anything we do here would be a “trial” - if we changed this and it went catastrophically wrong, we wouldn’t be insisting that we be fully locked into it for all eternity. The idea would be to bump the accuracy cap up to 13 or 14 for next season and keep a close eye on the data to see if it’s helping spread some of the increased scoring to defense instead of just having all of it come from an increase in forward scoring.

[Image: gunnarsoderberg.gif]


[Image: nbb.png]
[Image: DG0jZcS.png]
. : [Image: zS2lCMp.png] : .
Reply
#32

10-13-2022, 04:58 PMsköldpaddor Wrote:
10-13-2022, 04:26 PMRomanesEuntDomus Wrote: I don't think so but I also don't think it's needed. The attributes aren't comparable anyway and what matters in this case is the baseline of stats imho, not the attributes that led to it.

I'd just add onto this and say - I don't think we can really expect the STHS "normal" to be the normal ever again. That ship has sailed. We're never going to be able to shoehorn results from a completely different engine into the standards of one we aren't using anymore. I think what we're trying to determine now is what level of variance we should expect/accept between two versions of the same game, with the same set of attributes, and what changes to those attributes we should get used to and which ones behave in ways we weren't necessarily expecting (and if we're okay with that or if we want to try to actively work to balance it out along the way).

I'm not arguing to return to any sort of pre-FHM state, I'm arguing that it might lead to faulty conclusions if we take the last dozen seasons as the sole baseline for all our evaluations when in fact they could be an outlier.

Evan Winter
Edmonton Blizzard
Player Page - Update Page


[Image: winter-500.png]
Reply
#33
(This post was last modified: 10-13-2022, 10:22 PM by goldenglutes. Edited 1 time in total.)

10-13-2022, 02:44 PMCanadice Wrote:
10-13-2022, 01:35 PMItsFrenchie Wrote: I kind of want to add to this. Although sample sizes do affect scatter plots in how strong regression lines are, regression lines cannot judge wether or not a correlation can be established between X and Y, in this case the amount of applied TPE in shooting accuracy and other relevant attributes and point production. There is then the issue that (i'm sorry, i'm a newbie, so this may be wrong) shooting accuracy mostly affects goals and not necessarily assists, and in that sense, a scatter plot comparing total points vs. appliedTPE seems to skew the data. All this to say that more data needs to be collected, or at the very least two similar periods of time need to be compared so that equivalent sample sizes are being analyzed, there should be consideration for comparing applied TPE with goals if this is what shooting accuracy and other scoring attributes tend to influence more than overall point production, and further testing, such as correlation tests should be done.

I reiterate, my experience both with FHM and the SHL is limited, so I may be wrong on this or I may have omitted other aspects in my reflection.

Yes this is also an issue.

I did some things. First I took the almost two seasons of FHM8 data (S66 and S67) we have and combined it with the last two seasons of FHM6 data (S64 and S65). I added two columns from the index data, one indicating positional group (D or F) and one indicating the engine used to produce that data. Then I went to work on some visualizations.

First we have the two engines' data overlaid for each position in a scatter plot. FHM6 is orange and FHM8 is black. I also added a linear regression line (straight line) on top. It looked like Jeffie added a spline (non-linear line) on the original graphs that is better at modelling details in the data so the lines will look a bit different. The straight lines will not be able to find as many details in the relationship between the two variables but works for my current visualizations.

[Image: unknown.png]

The image shows that there has been a drastic increase in "base" level of points for Forwards, while there seems to have been a decline in points for Defensemen. One note is that there are more players overall in FHM8 with less than 500 applied TPE which could affect the estimates and are probably not going to be the general population we want to look at at the SHL level.

I then filtered out those players and redid the graph:
[Image: unknown.png]

Now the difference between the engines becomes clearer, the defense sees a small reduction in points at the same TPE level while forwards see a large increase in points at the same TPE level.

One of the options listed in the original post was to increase the Shooting Accuracy limit for Defensemen. To visualize this, I created a boxplot for each Shooting Accuracy and grouped them per position (top is D, bottom is F) and engine (left is FHM6, right is FHM8).

[Image: unknown.png]

First looking at the FHM6 data, we can see that Forwards saw a big uptick in points produced when increasing their Shooting Accuracy from 12 to 14. In FHM8 we see that Forwards with 12 in Shooting Accuracy already produce at the same level as 14-15 in FHM6. As for Defensemen, having Shooting Accuracy at 9-12 seems to produce about the same in both engines, maybe a small decrease in the median (the line inside the colored box).

One would expect that Shooting Accuracy affects Goals more than Assists. Being able to hit the net would probably give you a better chance of scoring compared to getting an assist. One could maybe argue that it would lead to an increase in rebounds that would produce assists, but regardless I wanted to look at the two ways you can produce points.

First comes goals:
[Image: unknown.png]

Then comes assists:
[Image: unknown.png]

then comes a baby in a baby carriage...

For forwards the same trend can be seen here between the engines as we saw in points. There is an increase in goals when increasing Accuracy from 12 to 15 in FHM6 while the same cannot be said for FHM8. The increase comes instead at around 15 to 17, which can also be seen in FHM6 up to 18. Defensemen do see an increase in goals from 10 to 12 in FHM8 compared to FHM6 where it's more stable (or at least a lower slope in trend).

Assists don't really see any trend for defensemen in either engine but forwards show the same trend that we've seen in the other graphs. This trend is most likely due to confounding effects (other variables), i.e. when the player has put in TPE in Shooting Accuracy, they have most likely put in TPE in other attributes that affect the ability to produce assists.

What does this all mean? It's a really tough situation to solve. Without doing specific controlled simulations where we can control the attributes of the players, there is no real way to know the real causal relationship between an attribute and a performance metric. Observational studies where we just take measurements of the attribute values and performances will never be able to find whether or not the correlation between two variables are because of one another or if it exists purely by chance. A good example of this is that there is a positive correlation between the number of movies Nicolas Cage stars in during a year and the number of drownings in a US pool that same year. These two variables have nothing to do with one another, but there is a correlation between them nonetheless.

As Shooting Accuracy is the only attribute that has different limits for the two positional groups and is described as an offensive attribute, I think it be worth to test allowing Defensemen to get to 13 or even 14 while keeping the average number of goals per game setting the same. Hopefully the trend we saw in the boxplot of goals vs Shooting Accuracy in FHM8 can continue and we see more goals scored by Defensemen.

After completing this writing I also saw fellow stats enthusiast @Jumbobone19 looking at number of shots as a statistic so I went ahead and added a final graph of that performance metric.

[Image: unknown.png]

It looks like overall the number of shots is lower for both D and F at all levels of Shooting Accuracy. I then had to do a plot on shooting percentage where it looks like FHM sees a higher shooting percentage overall for all Forwards at the same levels of Shooting Accuracy.

[Image: unknown.png]

I guess this is already kind of captured by the shooting % graph, but the attribute vs points/goals/assists graphs are likely biased because people with better attributes also have more ice time (and probably better offensive deployment/PP time too). Regressing against even strength goals/60 would probably be a bit more accurate, but that doesn't get around the potential offensive/defensive deployment bias.

it would also get around the issue of the shortened season for s67

great work though

[Image: glutes2.gif]
Signatures by Vulfzilla, Jepox, Jess, rum_ham, Ragnar, and myself
[Image: 9vAsr7c.png]
[Image: tkMQzhf.png] [Image: tdKmZA0.png]


Reply
#34

10-13-2022, 01:05 PMhotdog Wrote:
10-13-2022, 01:02 PMhotdog Wrote: do the same charts but x axis is Shooting Accuracy attribute (probably not much variance for D but show it for forwards)
or better yet Shooting Accuracy/Total TPE

both are bad, most shl dmen have 12 shooting accuracy so the graph won't show anything, and because of that, higher TPE players will have less shooting accuracy/total TPE

idiot

[Image: glutes2.gif]
Signatures by Vulfzilla, Jepox, Jess, rum_ham, Ragnar, and myself
[Image: 9vAsr7c.png]
[Image: tkMQzhf.png] [Image: tdKmZA0.png]


Reply
#35

10-13-2022, 10:24 PMgoldenglutes Wrote:
10-13-2022, 01:05 PMhotdog Wrote: or better yet Shooting Accuracy/Total TPE

both are bad, most shl dmen have 12 shooting accuracy so the graph won't show anything, and because of that, higher TPE players will have less shooting accuracy/total TPE

idiot
literally said show it for forwards cuz D has no variance you dingus

[Image: ekovanotter.gif]
thanks @Carpy48 and @frithjofr and @rum_ham and @Julio Tokolosh and @Briedaqueduc for the sigs
Armada Inferno norway
Reply
#36

[Image: unknown.png]

This histogram plot looks at the last 4 seasons of data from both engines. The y-axis is the number of players within each group of applied TPE seen at the bottom. Once again I have filtered out players with less than 500 applied TPE.

The vast majority of defensemen has 12 in Shooting Accuracy and the distribution of applied TPE at that attribute level is relatively normal (normally distributed).

Splitting it up into the different engines with two seasons each of player data:

FHM6
[Image: unknown.png]

FHM8
[Image: unknown.png]

We don't see any real changes in Defensemen distribution between the engines. A bit fewer defensemen overall with 12 but the distribution seems about equal in range. FHM6 seems to have more D with less applied TPE that still have 12 in Shooting Accuracy (SHA) whereas in FHM8, there is an even distribution of applied TPE with SHA 12.

[Image: 8E70VfU.png]

[Image: canadice.gif]
Thanks to @sköldpaddor, @Ragnar, @Carpy48 and @High Stick King, for the signatures







[Image: jZtKPwK.png] | [Image: RyzkmSj.png] | [Image: HKi05IH.png]

[Image: EzY5jpl.png]
Reply
#37

10-13-2022, 09:23 PMRomanesEuntDomus Wrote:
10-13-2022, 04:58 PMsköldpaddor Wrote: I'd just add onto this and say - I don't think we can really expect the STHS "normal" to be the normal ever again. That ship has sailed. We're never going to be able to shoehorn results from a completely different engine into the standards of one we aren't using anymore. I think what we're trying to determine now is what level of variance we should expect/accept between two versions of the same game, with the same set of attributes, and what changes to those attributes we should get used to and which ones behave in ways we weren't necessarily expecting (and if we're okay with that or if we want to try to actively work to balance it out along the way).

I'm not arguing to return to any sort of pre-FHM state, I'm arguing that it might lead to faulty conclusions if we take the last dozen seasons as the sole baseline for all our evaluations when in fact they could be an outlier.

I suppose that's possible, but how many seasons should we wait? I think we have to consider the FHM era and the STHS era as two completely separate things. It would be great if we had more data, but all we have right now is the seasons we've been using FHM - we can't really wait around for another ten seasons and wait for an entire generation of players to go through their whole careers before we make any changes.

Even if we were doing any kind of testing in HO (which we are not currently), for example if we simmed 20 seasons right now, that data wouldn't be terribly useful because of how players will develop and rosters will evolve over that time, so while the twelve seasons are limited, they're kind of all we have to go on right now, and we have to make decisions based on that.

[Image: gunnarsoderberg.gif]


[Image: nbb.png]
[Image: DG0jZcS.png]
. : [Image: zS2lCMp.png] : .
Reply
#38
(This post was last modified: 10-14-2022, 02:25 PM by spooked. Edited 4 times in total.)

Why not just give Dmen a higher starting shooting range so they don't need to sink a boat load of TPE into it to have the actual amount of range that someone would need to hit the net regularly from the blue line, and if you want to balance this with forwards, just give them the same amount inverted into accuracy instead.

To illustrate the issue:

Cale Makar (best D sniper we think in the NHL file), has 18 shooting range... literally NO ONE has that in my index right now. Only 2 players show as 17 atm, and many at 16. Range functions a bit weird in the engine, so not really being able to afford the intended amount of range to be an OFD without massive investment is probably a contributing issue tbh. And if we are to believed that one rating jumps are pretty impactful (14 bad, 15 okay, 16 good, etc.) not being able to get range to 18 from the blueline may actually just make it that much harder to have a viable OFD build (it would be RNG based, so that may help get MORE people reach the ceiling in more seasons)...
Reply
#39

10-14-2022, 02:21 PMspooked Wrote: Why not just give Dmen a higher starting shooting range so they don't need to sink a boat load of TPE into it to have the actual amount of range that someone would need to hit the net regularly from the blue line, and if you want to balance this with forwards, just give them the same amount inverted into accuracy instead.

To illustrate the issue:

Cale Makar (best D sniper we think in the NHL file), has 18 shooting range... literally NO ONE has that in my index right now. Only 2 players show as 17 atm, and many at 16. Range functions a bit weird in the engine, so not really being able to afford the intended amount of range to be an OFD without massive investment is probably a contributing issue tbh. And if we are to believed that one rating jumps are pretty impactful (14 bad, 15 okay, 16 good, etc.) not being able to get range to 18 from the blueline may actually just make it that much harder to have a viable OFD build (it would be RNG based, so that may help get MORE people reach the ceiling in more seasons)...

This assumes that FHM8 would sim last NHL season similar to the real NHL season and that doesn't seem to be the case. The defense do score less and get less points overall in general in FHM, it isn't a SHL specific thing, and that likely comes down to them underrating accuracy for most defensive players. I don't know how much detail I can share but they have a guide for the people who create player ratings and 12 is high but isn't considered elite in that guide, so essentially fhm ratings tell us that they don't think any existing D man is elite when it comes to shooting accuracy.

[Image: DrunkenTeddy.gif]



[Image: CsnVET2.png]  |  [Image: sXDU6JX.png]
Reply
#40
(This post was last modified: 10-14-2022, 03:03 PM by spooked. Edited 4 times in total.)

10-14-2022, 02:43 PMDrunkenTeddy Wrote:
10-14-2022, 02:21 PMspooked Wrote: Why not just give Dmen a higher starting shooting range so they don't need to sink a boat load of TPE into it to have the actual amount of range that someone would need to hit the net regularly from the blue line, and if you want to balance this with forwards, just give them the same amount inverted into accuracy instead.

To illustrate the issue:

Cale Makar (best D sniper we think in the NHL file), has 18 shooting range... literally NO ONE has that in my index right now. Only 2 players show as 17 atm, and many at 16. Range functions a bit weird in the engine, so not really being able to afford the intended amount of range to be an OFD without massive investment is probably a contributing issue tbh. And if we are to believed that one rating jumps are pretty impactful (14 bad, 15 okay, 16 good, etc.) not being able to get range to 18 from the blueline may actually just make it that much harder to have a viable OFD build (it would be RNG based, so that may help get MORE people reach the ceiling in more seasons)...

This assumes that FHM8 would sim last NHL season similar to the real NHL season and that doesn't seem to be the case. The defense do score less and get less points overall in general in FHM, it isn't a SHL specific thing, and that likely comes down to them underrating accuracy for most defensive players. I don't know how much detail I can share but they have a guide for the people who create player ratings and 12 is high but isn't considered elite in that guide, so essentially fhm ratings tell us that they don't think any existing D man is elite when it comes to shooting accuracy.

So why shouldn't we give D the actual range they should have to play from the blueline? Not sure I get why this wasn't done already. Just looking at the way range works, I have no idea how we didn't give D a starting bump in the first place when we came from STHS. It is literally based on distance from the net so D and forwards starting at the same rating on it is honestly a big oversight to me. D literally need to have a higher range rating to get their accuracy working 1:1, but we don't even give them any extra right now. Basically totally screwing D out of something they need in the position tbh. Especially with the update scale changes, forwards can get their needed range and more with nearly no TPE investment, but D won't be able to get it until well into the SHL career without giving up a lot of TPE, and yeah D cant get more accuracy to help, but like if you want to be an OFD early in your career you won't have one of the ratings you actual need to do that properly cause it costs too much to get it to the right place for your position.
Reply
#41

Out of curiousity (And slight anger at 0% shooting accuracy after bumping up to 13) I went and checked if D goals are up across the board. The current leader of goal scoring for defenseman is Dirty Boots at 8, putting him on pace for 24 goals, and he did bump to 13. Checking last season, Angus McFife XVIII led defenseman in goals with 24, and in S66 it was Jonas Kahnwald with 23. So even with the bump the pacing seems to be the same. We could bump it, but it may need to go fairly high with the top goal scorer overall pacing 90+ goals with 16 Shooting Accuracy (I'm aware defenseman scoring anywhere near 90 goals in unrealistic but it'd be nice if defenseman could get anywhere near PPG seeing as only 1 person has done it so far since we switched with no one pacing PPG this season)



[Image: hPSkjwC.jpeg]
Reply




Users browsing this thread:
1 Guest(s)




Navigation

 

Extra Menu

 

About us

The Simulation Hockey League is a free online forums based sim league where you create your own fantasy hockey player. Join today and create your player, become a GM, get drafted, sign contracts, make trades and compete against hundreds of players from around the world.