Scoring in CF Games Comps
1 Attachment(s)
Hey.
So CF competitions are all about having fun and building community and pushing yourself and all that stuff. But, the athletes, especially the top ones, really go there to compete. And even though unfair competitions could still be exciting, a fair competition is almost certainly better than the alternative. To that extent, a scoring system that accomplishes what it's supposed to accomplish (figuring out which athletes had the best performances) is better than one that doesn't do that. Now, sometimes it's easy to figure out who the top 1 or 2 athletes are. But, other times your results will vary depending on which scoring system you use. I think most of the commonly used scoring systems have shortcomings. I'll explain them. #1  The "Rank" scoring. Example: First place gets "1," second place "2," etc, for each event. The person with the lowest score across all the events is the winner. Issues: In this system, really dominant performances aren't given extra points. So, if a 500lb Deadlift is #2, then it doesn't matter at all whether #1 is 505 or 900. Likewise, really terrible performances aren't penalized enough (although, admittedly, usually a really bad performance means you're out, but it would be nice to rank middleofthepack folks correctly, too). And, if you have a ton of people all together, a couple pounds could mean the difference between 5th and 25th in an event. Variation: At my Regional, the Southeast, they set it up so that 500 goes to first, 490 second, 485 third, then deducted 5 from a few more, and then started deducting 3. This is an improvement, because it's likely that first place beat everyone by a fair amount, and by the time you get to 8th20th, times are closer together. But, that's not necessarily the case, and really bad performances that don't DNF aren't penalized enough... and a really dominant performance still isn't given enough points. #2  "Every Second Counts" Example: You have a few events, and they're all timebased, and you score based on total time. Issues: In this example, you're kinda forced into doing all timebased events. You're also forced to decide to either have all about the same time domain events, in which case you poorly test broad time domains, or you test broad time domains, in which case the longer events and the events where there is a broader discrepancy (standard deviation) in times will be weighed disproportionately high (example: A 5k Row at the elite level puts all the times very close together ... a 5k Run at the same level puts the times more spread out). Variation: You theoretically could do some math to convert something like a 1rm event to the same scoring system, but you still have the same types of problems. #3  Give Proportions off the Highest Score Example: So say the top Deadlift is 500lbs. #2 gets 400. So #1 gets 100 points, and #2 gets 80 points (400/500*100). #3 gets 360, so he gets 720 points. Issues: Well the same issue here. When the scores are more spread out in an event, the people that finish a good bit below, or a good bit above average are given a disproportionately large amount of points. Variation: You could give proportions off the average score, but the issue there is that some events will end up being weighted more than others. Ok, so that may sound a little bit minor and such, but if you were to plug in the data sets for the various Regionals and change up the scoring systems, you'd have some different athletes qualifying and some different athletes missing out. And in some of the cases the people that should have advanced, or won, didn't. Solution So here's my solution. It sounds like it makes things hard, but with some computer scripting it's a breeze. Steps #1. Get all the numbers/times and figure out whether a high score is good (i.e. Deadlifts) or Bad (i.e. 5k run) #2. Find the Max #3. Find the Min #4. Find the Standard Deviation #5. Create a new data set. Subtract the Minimum score from each score. Then, add 2*STDEV to that number. #6. Find the Max of the new Data set #7. 100 divided by the Max of the new Data set #8. Final data set: Multiply each number of the second data set by the number you just got in #7. #9. Highest total score when you take the sum of the Final Data Set for each individual, across all events, wins. So, basically, you're giving everybody a score based on the top performer, but you're accounting for standard deviation. Some possible criticisms: #1  Less exciting because sprinting to the finish ahead of someone doesn't matter much  Ok, yeah, true, but to that same extent, if someone is about to win an event and he's crushing everyone else, now he does have an incentive to kick it up a notch at the end, because he gets more points. #2  A really good performance + a mediocre performance could be scored ahead of 2 pretty solid performances  Here's an example. Someone that is taking the SAT is given an 800 Math and 500 Verbal for a 1300. Someone else got a 600 on both for a 1200. The second guy walks away with the better score. Fair? I think so. Some people might think differently, but I think they're wrong. It's not like you can give a really good performance and then completely suck on the rest of the events. And, from a practical matter, getting really good at one thing usually comes at an overall improvement being less than improving everything together, over the course of a year. Attached is an Excel explaining this. Also, it's possible that there's a smart Statistician or something that can make some alterations, or maybe even a whole new idea (although I imagine the best way to score is going to involve using Standard Deviations and scoring based off the #1 performance). Oh, and for those that dig into the formula, you may wonder why I added 2*STDev in #5. The number 2 was my guess of what was appropriate, but maybe someone has a better number there. It basically keeps the scores consistent when the data sets have different standard deviations. There may be a little more sophisticated and better way to do this, but this works well as it is. 
Re: Scoring in CF Games Comps
Quote:
For my own records and playing around I was doing: #1. Get all the numbers/times and figure out whether a high score is good (i.e. Deadlifts) or Bad (i.e. 5k run) #2. Find the Mean (ie. average) of these scores #3. Find the Standard Deviation of these scores #4. Create a new data set of standardized scores (ie. zscores) by subtracting the mean from each score and dividing by the standard deviation. Ie z = (original score  mean) / stddev #5. Now work with the columns of z scores. If the z score is negative when a positive score is good, make it 0. If the z score is positive when a negative score is good, make it 0. Ie. delete the ones that are already below average scores. #6. Create a Final Score column, which is the sum of the absolute value of the zscore columns. The one with the largest score wins. Justin 
Re: Scoring in CF Games Comps
With that method (a) won't the top scores be weighted exponentially high since it uses ZScores, and (b) in this case, won't the #1 score in each event have a different value? i.e. first place won't always be worth 100 points, or 1 point, or whatever, but may be 120, then 94, then 100, etc?
I think that might be problematic. I guess the total scores for each event will be equal, but I feel like first place should always get the same amount. I may change my mind if I think about it more. I could be very wrong on both points. You're clearly a lot more advanced on the stat stuff. 
Re: Scoring in CF Games Comps
Quote:
Justin 
Re: Scoring in CF Games Comps
Yeah you're right on (a). I forgot you're not consulting the Z chart to get a percentage but instead stopping a step before that.

Re: Scoring in CF Games Comps
I just want to point (npi) out that HQ made a post about Scoring in the CF Games this year.
They are doing the "1st place gets 1, 2nd 2, etc" system. They dismissed the "scoring in proportion to finishing" system for the reasons I listed in the first post. They dismissed fixes like the one I suggested and said it was too complicated and would ruin the flow of the Games. In the individuals the system isn't terrible (although it's certainly not ideal). But, in the Masters it's ridiculous. For the Men's Deadlift, #1 was 490, #2 was 485, and #3 was 425. That's ridiculous that #3 gets the same amount of points whether he hit 425 or 480. 
Re: Scoring in CF Games Comps
edit: nevermind

Re: Scoring in CF Games Comps
I don't think you should be too concerned about this topic  they're really just trying to find a convenient way to average the results from the events so that the person with the best average wins.
Of course their sample of events is too small to estimate something as complex as fitness, especially with cuts made. For all the talk of power curves being integrated to get fitness, it's much talk about nothing really, since when faced with the task of ranking the people based on fitness, power curves don't even get mentioned. 
Re: Scoring in CF Games Comps
Even though proper scoring wouldn't mean fitness is measured perfectly, it would be an improvement, and at any rate, it would do a better job of telling who had the best performance at the events, cumulatively.
Note that now that they are down to 24 competitors, it's much harder to gain/lose ground than when they were at 45. Tough for the people that do very well in the next couple events to catch up. 
All times are GMT 7. The time now is 07:34 PM. 
Powered by vBulletin® Version 3.6.8
Copyright ©2000  2020, Jelsoft Enterprises Ltd.
CrossFit is a registered trademark of CrossFit Inc.