CrossFit Discussion Board  

Go Back   CrossFit Discussion Board > CrossFit Forum > Competitions
CrossFit Home Forum Site Rules CrossFit FAQ Search Today's Posts Mark Forums Read

Competitions Competitions, contests & challenges

Reply
 
Thread Tools
Old 07-22-2010, 10:43 PM   #131
Mauricio Leal
Affiliate Mauricio Leal is offline
 
Mauricio Leal's Avatar
 
Profile:
Join Date: Jun 2009
Location: Oakland  CA
Posts: 839
Re: 2010 games scoring - impossible to catch up?

Quote:
Originally Posted by Justin McCallon View Post
Edited slightly. Red is what you can edit to add people/events/times to see changes in results.

Non-arbitrary scoring that provides legitimate rankings for people and doesn't have any of the problems associated with any of the other systems.
Justin,

I understand your scoring system is superior and fixes all the flaws of the lesser systems. I may be wrong, but I just doubt it's practical enough for the Games as they stand. I suppose if CrossFit ever goes to the Olympics or is sanctioned by a larger sports governing body, strict and fair scoring will be required. As it stands today, it is just so much easier to perform at most one arithmetic operation (A/B), which almost anyone can do on a napkin, versus having to perform ~3*N operations on a fairly large data set in order to determine the standard devation. Make it a real time iPhone app and it has a chance .
__________________
CrossFit Cypher
www.crossfitcypher.com

Last edited by Mauricio Leal : 07-22-2010 at 10:48 PM.
  Reply With Quote
Old 07-23-2010, 12:06 PM   #132
Shane Skowron
Member Shane Skowron is offline
 
Profile:
Join Date: Aug 2007
Location: Queens  NY
Posts: 3,800
Re: 2010 games scoring - impossible to catch up?

Quote:
Originally Posted by Ewen Roth View Post
I'm not all against proportional+SD scoring, but why do you insist on including world record performances in your theoretical span of results? Those athletes will NOT, ever, be at the CF Games. What matters (if it actually matters; I'm kinda with Mauricio on this) is the spread for final CF Games contestants.
It's how the decathlon is scored. It's the fairest system.

Comparing the results to what the top Crossfitter can do is silly, because the top Crossfitter (or all the competitors) might be extremely weak in a particular event. How else would we know that than by comparing to world record?
  Reply With Quote
Old 07-23-2010, 12:29 PM   #133
Justin McCallon
Member Justin McCallon is offline
 
Profile:
Join Date: Oct 2007
Location: Atlanta  GA
Posts: 551
Re: 2010 games scoring - impossible to catch up?

Quote:
Originally Posted by Shane Skowron View Post
It's how the decathlon is scored. It's the fairest system.

Comparing the results to what the top Crossfitter can do is silly, because the top Crossfitter (or all the competitors) might be extremely weak in a particular event. How else would we know that than by comparing to world record?
Just to be clear since the original response was to me and maybe what I was saying was ambiguous --- my point was that rowing tends to have results that are pretty close together in relation to running.

I used the world record vs. the average crossfitter to show that the difference was sorta close, whereas the difference between the world record and an average crossfitter is pretty spread out. Perhaps it would be easier to simply agree that some events have about the same median time but that there's much greater standard deviations. 20 seconds between two competitors in one should therefore not be scored the same as 20 seconds between two competitors in another.


You can't really use the decathlon's scoring for crossfit because the events are events that nobody has ever done before, and since a lot are mixed modal, the people doing a lot of the eventsare probably the best at them in the world.
__________________
My Workout Log
  Reply With Quote
Old 07-24-2010, 06:08 PM   #134
Jason Casiano
Member Jason Casiano is offline
 
Profile:
Join Date: Jul 2009
Location: Milford  CT
Posts: 10
Re: 2010 games scoring - impossible to catch up?

Quote:
Originally Posted by Matt Haxmeier View Post
I wonder if they don't keep everyone because of logistics? Or because it would be embarrassing to have the CF games winner also come in dead last on some other event?
Definitely logistics. Less heats to run. And it gives it a countdown to the finals type vibe. Imagine if all the teams in the NCAA tourney just kept playing each other until the winner of the most games was crowned national champ.
  Reply With Quote
Old 07-25-2010, 03:48 PM   #135
Michael V. Erickson
Member Michael V. Erickson is offline
 
Profile:
Join Date: Oct 2007
Location: Concord  California
Posts: 85
Re: 2010 games scoring - impossible to catch up?

There are problems with the "standard deviation method" of scoring that cannot be ignored.

Take the following hypothetical example:

DL(lbs) "SD" score Rel % score

1 510 1.46 100
2 495 1.10 97
3 480 0.73 94
4 465 0.37 91
5 450 0.00 88
6 435 -0.37 85
7 420 -0.73 82
8 405 -1.10 79
9 390 -1.46 76
Sdev 41.08
Avg 450.00

Presume this example is a contest between 10 crossfitters and is a max DL. The ten competitors happen to be evenly distributed about an average of 450 pounds. Everything looks well behaved. What happens if "specialist" score is added?

DL(lbs) "SD" score Rel % score

10 600 2.20 100
1 510 0.73 85
2 495 0.49 83
3 480 0.24 80
4 465 0.00 78
5 450 -0.24 75
6 435 -0.49 73
7 420 -0.73 70
8 405 -0.98 68
9 390 -1.22 65
Sdev 61.24
Avg 465.00

Competitor #10 scores much better than the other competitors in this one event. Notice how his score effects the SD scores of the other competors: they are compressed to the middle. The bottom two scores have actually improved relative to the first set of scores. Ironic, a superior compettior enters the contest and the two previous worst scores improve. The problem is the distortion of scores between the first ten athletes. Their performances relative to each other has not changed but their relative scores to each other have shifted a lot. Note that this is not the case with the relative performance scoring: the 390 lb lifter still has a relative score to the 510 lb lifter of 76% (65/85). Say the 600 lb lifter, the specialist, goes on in the following events to score average or mediocre scores, that is, he is not a factor in the final outcome of the contest. In the relative performance scoring method he does not effect the outcome one iota, it's as if he wasn't even there. Not so in the SD scoring, all of the other scores are distorted in relation to each other because of his presence. What if the "specialist" manages to score above average (but not near the top) in the other contests? It will be hard for the more well rounded athletes to overcome the huge score racked up in his specialty.
  Reply With Quote
Old 07-25-2010, 03:53 PM   #136
Justin McCallon
Member Justin McCallon is offline
 
Profile:
Join Date: Oct 2007
Location: Atlanta  GA
Posts: 551
Re: 2010 games scoring - impossible to catch up?

Michael -- good points. I thought about this in the case of someone missing a 1rm altogether, and my solution was to account for outliers. I didn't consider this for a really high lift. Fixed now.

That said, I would imagine that there is a better way to do this that perhaps you can think of. Maybe simply giving everyone a score based on the median, and making the median baseline at 100?
(currently I had it scoring based on the best performance)
__________________
My Workout Log

Last edited by Justin McCallon : 07-25-2010 at 04:14 PM.
  Reply With Quote
Old 07-25-2010, 04:23 PM   #137
Justin McCallon
Member Justin McCallon is offline
 
Profile:
Join Date: Oct 2007
Location: Atlanta  GA
Posts: 551
Re: 2010 games scoring - impossible to catch up?

Ignore the median part. That was stupid.
__________________
My Workout Log
  Reply With Quote
Old 07-25-2010, 04:32 PM   #138
Mauricio Leal
Affiliate Mauricio Leal is offline
 
Mauricio Leal's Avatar
 
Profile:
Join Date: Jun 2009
Location: Oakland  CA
Posts: 839
Re: 2010 games scoring - impossible to catch up?

Again, you guys are basically finding a technical problem where in reality there is none. Even if one event is skewed due to a powerlifter fudging the SD, you do realize that that person would most likely be immediately smashed by whatever the next event was that wasn't PLing oriented. And this would occur at sectionals. True specialists can't even make it out of those anymore, so by the time you get to regionals and the finals you're not really gonna have a problem with outliers.
__________________
CrossFit Cypher
www.crossfitcypher.com
  Reply With Quote
Old 07-25-2010, 06:24 PM   #139
Michael V. Erickson
Member Michael V. Erickson is offline
 
Profile:
Join Date: Oct 2007
Location: Concord  California
Posts: 85
Re: 2010 games scoring - impossible to catch up?

I'll just mention this and go away: The standard deviation calculation is meaningless for small populations. Using this scoring method would distort the relative scoring between athletes in unpredictable ways and would be grossly unfair. It is not fixable. I did not remark on this method a year ago because I thought this idea would just fall by the wayside after people thought about it for awhile.

The only problem with the decathlon scoring method is its non-linearity which advantages specialists. Make it linear and it's a good fit for Crossfit contests. That's what proportional performance scoring does. It is simple, easy to calculate, fair, measures what Crossfit is supposed to be all about; work capacity across broad modal domains. It does not reward specialists, it measures relative work done fairly and simply.

That said, given the set-up of the events at the 2010 games, I can see why the organizers again chose the placement point method for these games. The relative performance scoring is the most difficult to apply to the mixed mode time limited events. Some competitors finish and get a time (easy to score) some competitors don't finish on time and somehow you have to calculate the work done. How does one rep of 315 pound DL equate with one rep of double under? I can see the problem, but not a clear solution. I thought the games were brilliantly and creatively conceived and executed. I feel a sense of guilt for staying home and watching for free.
  Reply With Quote
Old 07-25-2010, 06:53 PM   #140
Justin McCallon
Member Justin McCallon is offline
 
Profile:
Join Date: Oct 2007
Location: Atlanta  GA
Posts: 551
Re: 2010 games scoring - impossible to catch up?

Quote:
Using this scoring method would distort the relative scoring between athletes in unpredictable ways and would be grossly unfair. It is not fixable.
It's not exactly unpredictable..

As long as there aren't specialists competing, it's not a problem at all. And unless there are groups of, like, twenty specialists, there is no issue.


Here's a scenario that came up at Regionals (affiliate event) that frustrated me based on rank-scoring.

The first event was, basically, a team Ground-to-Overhead event. Most the totals were between 850-950 (953 was second place). CrossFit Wilmington came in around 1050. Then, they got beat by groups of teams in other events by a couple seconds. They ended up taking second overall because the rank-scoring method didn't accurately judge their dominant performance. They could have totaled 954 in the first event and they would have received the same score.

I think the standard deviation score is slightly imperfect, but only very slightly when you deal with outliers reasonably. Proportional scoring doesn't take into account the fact that losing by 20% on a 5k run is not the same as losing by 20% on something like "King Kong" (as an extreme example).

Oh, and also -- Proportional scoring has the same exact problem as the Standard Deviation scoring that you pointed out!
__________________
My Workout Log

Last edited by Justin McCallon : 07-25-2010 at 07:02 PM.
  Reply With Quote
Reply


Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is Off
HTML code is Off
Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Scoring in CF Games Comps Justin McCallon Community 8 07-18-2010 07:02 AM
2010 CF Games Volunteers Britt Dowling Competitions 0 06-28-2010 06:33 AM
2010 Crossfit Games Scoring Rolf Whitney Competitions 2 05-05-2010 01:19 PM
Games Scoring Tom Seryak Competitions 112 02-01-2010 12:27 PM


All times are GMT -7. The time now is 10:48 AM.


Powered by vBulletin® Version 3.6.8
Copyright ©2000 - 2019, Jelsoft Enterprises Ltd.
CrossFit is a registered trademark of CrossFit Inc.