View Single Post
Old 06-06-2010, 07:27 PM   #1
Justin McCallon
Member Justin McCallon is offline
 
Profile:
Join Date: Oct 2007
Location: Atlanta  GA
Posts: 551
Scoring in CF Games Comps

Hey.

So CF competitions are all about having fun and building community and pushing yourself and all that stuff. But, the athletes, especially the top ones, really go there to compete. And even though unfair competitions could still be exciting, a fair competition is almost certainly better than the alternative. To that extent, a scoring system that accomplishes what it's supposed to accomplish (figuring out which athletes had the best performances) is better than one that doesn't do that.

Now, sometimes it's easy to figure out who the top 1 or 2 athletes are. But, other times your results will vary depending on which scoring system you use.

I think most of the commonly used scoring systems have shortcomings. I'll explain them.

#1 - The "Rank" scoring.
Example: First place gets "1," second place "2," etc, for each event. The person with the lowest score across all the events is the winner.
Issues: In this system, really dominant performances aren't given extra points. So, if a 500lb Deadlift is #2, then it doesn't matter at all whether #1 is 505 or 900. Likewise, really terrible performances aren't penalized enough (although, admittedly, usually a really bad performance means you're out, but it would be nice to rank middle-of-the-pack folks correctly, too). And, if you have a ton of people all together, a couple pounds could mean the difference between 5th and 25th in an event.
Variation: At my Regional, the Southeast, they set it up so that 500 goes to first, 490 second, 485 third, then deducted 5 from a few more, and then started deducting 3. This is an improvement, because it's likely that first place beat everyone by a fair amount, and by the time you get to 8th-20th, times are closer together. But, that's not necessarily the case, and really bad performances that don't DNF aren't penalized enough... and a really dominant performance still isn't given enough points.

#2 - "Every Second Counts"
Example: You have a few events, and they're all time-based, and you score based on total time.
Issues: In this example, you're kinda forced into doing all time-based events. You're also forced to decide to either have all about the same time domain events, in which case you poorly test broad time domains, or you test broad time domains, in which case the longer events and the events where there is a broader discrepancy (standard deviation) in times will be weighed disproportionately high (example: A 5k Row at the elite level puts all the times very close together ... a 5k Run at the same level puts the times more spread out).
Variation: You theoretically could do some math to convert something like a 1rm event to the same scoring system, but you still have the same types of problems.

#3 - Give Proportions off the Highest Score
Example: So say the top Deadlift is 500lbs. #2 gets 400. So #1 gets 100 points, and #2 gets 80 points (400/500*100). #3 gets 360, so he gets 720 points.
Issues: Well the same issue here. When the scores are more spread out in an event, the people that finish a good bit below, or a good bit above average are given a disproportionately large amount of points.
Variation: You could give proportions off the average score, but the issue there is that some events will end up being weighted more than others.


Ok, so that may sound a little bit minor and such, but if you were to plug in the data sets for the various Regionals and change up the scoring systems, you'd have some different athletes qualifying and some different athletes missing out. And in some of the cases the people that should have advanced, or won, didn't.

Solution
So here's my solution. It sounds like it makes things hard, but with some computer scripting it's a breeze.
Steps
#1. Get all the numbers/times and figure out whether a high score is good (i.e. Deadlifts) or Bad (i.e. 5k run)
#2. Find the Max
#3. Find the Min
#4. Find the Standard Deviation
#5. Create a new data set. Subtract the Minimum score from each score. Then, add 2*STDEV to that number.
#6. Find the Max of the new Data set
#7. 100 divided by the Max of the new Data set
#8. Final data set: Multiply each number of the second data set by the number you just got in #7.
#9. Highest total score when you take the sum of the Final Data Set for each individual, across all events, wins.

So, basically, you're giving everybody a score based on the top performer, but you're accounting for standard deviation.

Some possible criticisms:
#1 - Less exciting because sprinting to the finish ahead of someone doesn't matter much - Ok, yeah, true, but to that same extent, if someone is about to win an event and he's crushing everyone else, now he does have an incentive to kick it up a notch at the end, because he gets more points.
#2 - A really good performance + a mediocre performance could be scored ahead of 2 pretty solid performances - Here's an example. Someone that is taking the SAT is given an 800 Math and 500 Verbal for a 1300. Someone else got a 600 on both for a 1200. The second guy walks away with the better score. Fair? I think so. Some people might think differently, but I think they're wrong. It's not like you can give a really good performance and then completely suck on the rest of the events. And, from a practical matter, getting really good at one thing usually comes at an overall improvement being less than improving everything together, over the course of a year.


Attached is an Excel explaining this.

Also, it's possible that there's a smart Statistician or something that can make some alterations, or maybe even a whole new idea (although I imagine the best way to score is going to involve using Standard Deviations and scoring based off the #1 performance).

Oh, and for those that dig into the formula, you may wonder why I added 2*STDev in #5. The number 2 was my guess of what was appropriate, but maybe someone has a better number there. It basically keeps the scores consistent when the data sets have different standard deviations. There may be a little more sophisticated and better way to do this, but this works well as it is.
Attached Files
File Type: email Scoring.xls (30.5 KB, 242 views)
  Reply With Quote