CrossFit Discussion Board  

Go Back   CrossFit Discussion Board > CrossFit Forum > Competitions
CrossFit Home Forum Site Rules CrossFit FAQ Search Today's Posts Mark Forums Read

Competitions Competitions, contests & challenges

Reply
 
Thread Tools
Old 07-25-2010, 08:30 PM   #141
Michael V. Erickson
Member Michael V. Erickson is offline
 
Profile:
Join Date: Oct 2007
Location: Concord  California
Posts: 85
Re: 2010 games scoring - impossible to catch up?

Quote:
Originally Posted by Justin McCallon View Post
It's not exactly unpredictable..

As long as there aren't specialists competing, it's not a problem at all. And unless there are groups of, like, twenty specialists, there is no issue.


Here's a scenario that came up at Regionals (affiliate event) that frustrated me based on rank-scoring.

The first event was, basically, a team Ground-to-Overhead event. Most the totals were between 850-950 (953 was second place). CrossFit Wilmington came in around 1050. Then, they got beat by groups of teams in other events by a couple seconds. They ended up taking second overall because the rank-scoring method didn't accurately judge their dominant performance. They could have totaled 954 in the first event and they would have received the same score.

I think the standard deviation score is slightly imperfect, but only very slightly when you deal with outliers reasonably. Proportional scoring doesn't take into account the fact that losing by 20% on a 5k run is not the same as losing by 20% on something like "King Kong" (as an extreme example).

Oh, and also -- Proportional scoring has the same exact problem as the Standard Deviation scoring that you pointed out!
With proportional scoring specialists competing are no problem at all. They either keep up, or not, in their non-specialties.

I understand your frustration with the rank scoring. Proportional performance scoring would have credited the gap between first and second. In the example you gave 100-(100x953/1050)=> 9.24 points rather than one point. If you could give me a link to the competition results I can re-score using pps and tell you if your favored team indeed would have won in that scoring method.

20% is a huge difference in either event (5K or KingKong). Why do you think one is more significant than the other? Depends on the mission doesn't it?

RE: "exact same problem". Are you referring to one of my examples? Could you point out exactly what you mean. Thanks.
  Reply With Quote
Old 07-25-2010, 08:52 PM   #142
Justin McCallon
Member Justin McCallon is offline
 
Profile:
Join Date: Oct 2007
Location: Atlanta  GA
Posts: 551
Re: 2010 games scoring - impossible to catch up?

Quote:
Originally Posted by Michael V. Erickson View Post
With proportional scoring specialists competing are no problem at all. They either keep up, or not, in their non-specialties.

I understand your frustration with the rank scoring. Proportional performance scoring would have credited the gap between first and second. In the example you gave 100-(100x953/1050)=> 9.24 points rather than one point. If you could give me a link to the competition results I can re-score using pps and tell you if your favored team indeed would have won in that scoring method.

20% is a huge difference in either event (5K or KingKong). Why do you think one is more significant than the other? Depends on the mission doesn't it?

RE: "exact same problem". Are you referring to one of my examples? Could you point out exactly what you mean. Thanks.
Nevermind on my last point. As long as you go off the Median and not off the highest score, that problem isn't there.


Regarding the King Kong vs. 5k deal -- This is really pronounced in CrossFit metcons. Let's take 2 theoretical competitors in the 9-7-5 snatch/muscle-up event.
Adam can do 7 muscle-ups in a row.
Bob can do 11 in a row.
They are equal on snatch and have equal cardio abilities otherwise.

This all suggests to me that Bob is about 18% better on these movements ((7/11)/2 = .18). But what happens in competition?
Adam does 7 muscle-ups in a row (20s). Then he rests 30s. Then he does another 2 (10s). Then he does all the snatches in 30s. Then back to the rings and he gets 5 (20s). Rests 30s. Another 2 (10s). All Snatches in another 30s. Gets to the rings, does 3 MU's (20s). Rests 30s, gets 2 more (10s). Another 30s and he finishes his snatches. Total time = 4:30

Bob, on the other hand... All 9 MU's (30s). Snatches (30s). 7 MU's (30s). Snatches (1 min). 5 MU's (30s). Snatches (30s).
Total time: 3:00

So, Bob is really only about 18% better at these movements, but he wins by 33%. On just the muscle-ups, Bob is 36% better, but he moves 100% faster. I think this is fairly realistic.

Some other events, for whatever reason, tend to produce people with close distributions. Rowing for whatever reason has a much tighter distribution than running, especially among CFers. Almost every beginner at both rowing and running (even lighter ones) can row a faster 5k than they can run. But, the world record 5k run is much faster than the world record 5k row.

Point being that some events yield very tight normal distributions (or not-normal) and other events will yield very spread out normal distributions. A truly dominant performance in one event might be breaking the median by only 10%, whereas in another event a third of the competitors might be at that level, with the winner much higher.


Here's the event I was referring to:
http://scores2010.crossfit.com/scori...affiliate,144/
I know how proportional scoring works and I can enter it into Excel, but that looks like a lot of data entry -- do you have some kind of shortcut? Also, just by plugging in the the top 2 teams, it looks like the #1 team should have still won by a little under 1%. I wonder if things change when everyone else gets added in.
__________________
My Workout Log

Last edited by Justin McCallon : 07-25-2010 at 09:04 PM.
  Reply With Quote
Old 07-26-2010, 09:07 AM   #143
Michael V. Erickson
Member Michael V. Erickson is offline
 
Profile:
Join Date: Oct 2007
Location: Concord  California
Posts: 85
Re: 2010 games scoring - impossible to catch up?

Quote:
Originally Posted by Justin McCallon View Post
Nevermind on my last point. As long as you go off the Median and not off the highest score, that problem isn't there.


Regarding the King Kong vs. 5k deal -- This is really pronounced in CrossFit metcons. Let's take 2 theoretical competitors in the 9-7-5 snatch/muscle-up event.
Adam can do 7 muscle-ups in a row.
Bob can do 11 in a row.
They are equal on snatch and have equal cardio abilities otherwise.

This all suggests to me that Bob is about 18% better on these movements ((7/11)/2 = .18). But what happens in competition?
Adam does 7 muscle-ups in a row (20s). Then he rests 30s. Then he does another 2 (10s). Then he does all the snatches in 30s. Then back to the rings and he gets 5 (20s). Rests 30s. Another 2 (10s). All Snatches in another 30s. Gets to the rings, does 3 MU's (20s). Rests 30s, gets 2 more (10s). Another 30s and he finishes his snatches. Total time = 4:30

Bob, on the other hand... All 9 MU's (30s). Snatches (30s). 7 MU's (30s). Snatches (1 min). 5 MU's (30s). Snatches (30s).
Total time: 3:00

So, Bob is really only about 18% better at these movements, but he wins by 33%. On just the muscle-ups, Bob is 36% better, but he moves 100% faster. I think this is fairly realistic.

Some other events, for whatever reason, tend to produce people with close distributions. Rowing for whatever reason has a much tighter distribution than running, especially among CFers. Almost every beginner at both rowing and running (even lighter ones) can row a faster 5k than they can run. But, the world record 5k run is much faster than the world record 5k row.

Point being that some events yield very tight normal distributions (or not-normal) and other events will yield very spread out normal distributions. A truly dominant performance in one event might be breaking the median by only 10%, whereas in another event a third of the competitors might be at that level, with the winner much higher.


Here's the event I was referring to:
http://scores2010.crossfit.com/scori...affiliate,144/
I know how proportional scoring works and I can enter it into Excel, but that looks like a lot of data entry -- do you have some kind of shortcut? Also, just by plugging in the the top 2 teams, it looks like the #1 team should have still won by a little under 1%. I wonder if things change when everyone else gets added in.
Justin,
Very interesting stuff. Give me a little time to digest and I'll get back to you on this.

Quick remarks: Rowing simulates pushing a boat through water which gets exponentially hard as you try to increase your speed. Not true of running (wind resistance is a minor factor relatively speaking).

I can see the problem with the affiliate competition, the second place team won three of four events, the first place team dominated on only one event.

More later,
Mike
  Reply With Quote
Old 08-01-2010, 09:28 PM   #144
Michael V. Erickson
Member Michael V. Erickson is offline
 
Profile:
Join Date: Oct 2007
Location: Concord  California
Posts: 85
Re: 2010 games scoring - impossible to catch up?

Justin,
Took me awhile to get back to this. Here is the link to my proportional performance scoring for the SW Regional (WFS):
http://spreadsheets.google.com/pub?k...=1&output=html

I am convinced by your arguments that all of the scoring methods have problems, including PPS. Note in the above link I tried something to compensate for the compressed scores on the team row to give them equal "weight" with the first two events.

There is a link at the above site to an article on Decathlon scoring methods. They have been tweaking their scoring methods to try to give equal weight to each event for over 100 years and they're still at it. For the whole 100 years they've been doing the same events, so their task is far easier than figuring out how to score CF events equally since they change all of the time. I think we may have to just live with the placement point method.
  Reply With Quote
Old 08-02-2010, 02:39 AM   #145
Dimitri Dziabenko
Banned Dimitri Dziabenko is offline
 
Profile:
Join Date: Apr 2007
Location: Toronto  ON
Posts: 1,080
Re: 2010 games scoring - impossible to catch up?

I think there are too many theoretical difficulties to make a consistent scoring system. The rank system, provided all competitors are allowed to compete, or scores are reset the second day, together with a good balance of workouts (which I thought was missing this year) will do fine.

The best you can do, is try to score events evenly, with the standards being set by other competitors - proportional sounds a bit better in theory, but would not be as fun to watch. Somebody winning by standard deviations is lame.

As for the attempt to find the fittest man, by theoretical arguments, from a sample size of 9 workouts, not knowing what you are looking for, is bound not to produce anything meaningful.
  Reply With Quote
Old 08-08-2010, 04:59 PM   #146
Justin McCallon
Member Justin McCallon is offline
 
Profile:
Join Date: Oct 2007
Location: Atlanta  GA
Posts: 551
Re: 2010 games scoring - impossible to catch up?

Why is winning by standard deviation "lame?"
__________________
My Workout Log
  Reply With Quote
Old 08-11-2010, 04:20 PM   #147
Pär Larsson
Member Pär Larsson is offline
 
Pär Larsson's Avatar
 
Profile:
Join Date: Apr 2010
Location: San Diego  CA
Posts: 178
Re: 2010 games scoring - impossible to catch up?

I don't see the need for "normalizing" points after cuts.

I do see the problem with a single not-so-great performance before the first cut having a vastly exaggerated effect compared to a single not-so-great performance the last day.

Simple Solution (tm):

Place Points
1 1
2 2
3 3
4 4
.
.
.
10-15 12pts
16-25 17pts
26-35 XXpts
etc etc

Or more simply put:
Top 20% - points generally by placement
21-40% - all lumped together
41-60% - all lumped together
61-80% - all lumped together
81-100%ile - all lumped together but with enough points to probably kill their chances at top spot

Then you don't have to dramatically change the scoring for each round of cuts - or at least not change it as much.

Just adjust the numbers I put down above to where a person who is a great all-round athlete like Spealler but happens to hit a rough spot in an early WOD that dramatically disadvantages him, still has a chance to work his way back up, so he's not going into the latter stages facing an impossible uphill battle.

We're not looking for great crowd entertainment here, we're looking for the most GPP, most overall "fit" human on the planet. That's hugely on the shoulders of the guys putting together the workouts, which is unfortunate, but still a better situation than announcing every WOD a year in advance or having a standardized sequence of events. Now add the constraints the HQ people face with regards to venue, time, spectating etc and I think they did a pretty damn fine job. Sure they could do better, but no matter what they do they'll be criticized, so I wouldn't worry too much about the chatter in this thread.

Other than maybe the scoring issues with having:

* workout favouring heavy strength athletes early on - easier for those heavy strength athletes to get in a good position before the cuts
* workout favouring light, short, endurance athlete later on - impossible damn near for him to catch up

...and v.v.

All in all I think they did a good job running the event. 2011 will have further refinements, no doubt.
  Reply With Quote
Reply


Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is Off
HTML code is Off
Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Scoring in CF Games Comps Justin McCallon Community 8 07-18-2010 07:02 AM
2010 CF Games Volunteers Britt Dowling Competitions 0 06-28-2010 06:33 AM
2010 Crossfit Games Scoring Rolf Whitney Competitions 2 05-05-2010 01:19 PM
Games Scoring Tom Seryak Competitions 112 02-01-2010 12:27 PM


All times are GMT -7. The time now is 07:40 AM.


Powered by vBulletin® Version 3.6.8
Copyright ©2000 - 2019, Jelsoft Enterprises Ltd.
CrossFit is a registered trademark of CrossFit Inc.