Calculation method for rankings?
-
Topic created by johncampbell on Mon Feb 24, 2014 at 10:46 pmLooking at results yesterday and the ranking points has me intrigued by the ranking calculation methodology. On the green course my stats were Pace 8.46min/km, PCR 3739, CGV 4392 - ranking points 69.2 Petr Hartman ran the red course and his stats were pace 10.21 min/km, PCR 5094, CGV 4693, ranking points 76.13. While I understand that the red course is more difficult, I am interested to know where does the per/km come into play? How is the PCR and CGV determined? From a course planning perspective the only difference between green and red is the distance. I am interested if someone can tell me what per/km time I would have to run to have got the same points as Petr. Not that my ranking really matters but interested to understand the methodology.
-
Reply by ErikEddy on Tue Feb 25, 2014 at 10:27 amhttp://www.dvoa.org/rank/calcrank2013.php
I don't fully understand either. My understanding is that the rankings become "more accurate" as the year progresses. I believe your score at each event will actually change throughout the year since the "field strength" is based off of everyone elses rankings.
My question is... do the rankings totally reset? Or are they based off of previous year's results (to determine "field strength)? -
Reply by Vadim on Tue Feb 25, 2014 at 12:11 pmPrior to 2013, rankings started fresh each year and we needed at least 4 races to run calculations. Since last year your rankings score is available and can be used for current calculations. But if you did not run last year or are a new to orienteering, the system will use score 50 as the initial value and readjust during calculations and over the year.
-
Reply by Vadim on Tue Feb 25, 2014 at 12:13 pmAs for the PCR and CGV numbers, I'll let you know later have to refresh my memory )))
-
Reply by Steve on Tue Feb 25, 2014 at 6:40 pmrankings are comparative. Let us say that Tom beat Bill on red, while Joe beat Harry on Green. Since Joe and Tom were doing different things, we can not compare. Next week, Tom decides to run green, and beats Harry but not Joe. We can now say that Joe is the best, Tom is next and Harry and Bill are some where near the same, based on how much Tom beat them on the different weeks. As Erik and Vadim said above, the more runs we have, the better this system works.
The system needs to adjust everyones ranking several times to arrive at a ranking also. As Vadim said, we are assigned a ranking of 50 out of 100. As the system runs the calculation, Some people will get bumped up, while others get bumped down. Since we were comparing different courses, we then have to run the program again, with the new numbers. Again, some will rise, and some will fall, but not by as much. When the computer decides that the change per run is negligable, it then posts the results, until you go and run another event, adding data,and demanding a new ranking.
The course gnarly value is done the same way, only the courses are evaluated using the runners as the constant. The Personal course ranking is again done the same, but this time you are the constant.
Hope I helped. -
Reply by Vadim on Tue Feb 25, 2014 at 9:34 pmThank you Steve for explanation. The magic numbers are 100 iterations until deviation is less than 0.001. The more data we have, the longer it takes to calculate, etc.If someone ran only once, their score expected to be around 50 +/-. But then they do not play significant role in overall calculation.
-
Reply by hughmac4 on Wed Feb 26, 2014 at 2:30 pmHmm, I don't see any scores at all. What am I missing?
-
Reply by j-man on Wed Feb 26, 2014 at 7:12 pmI think the scores are the carry over, year-end scores for 2013. They don't show up at the individual race(s), but rather in the total row.
-
Reply by hughmac4 on Thu Feb 27, 2014 at 2:11 pmHmm, ah, I see, in the individual records. And it's not actually showing scores yet (other than in the totals), so maybe something administrative needs to happen to move to 2014, and Vadim will take care of it when he's back from his travels? and has the time.This question comes up regularly.
I have a feeling that I'm in the same boat as John in comparison to the Red runners. Faster per k time, and on a LONGER course, but significantly lower score, assuming I'm infering the right numbers from each of our personal score pages. I'm comfortable with the fact that when I trounce them in a future race on the same course, their scores for Batsto will plummet, and mine will go up, because there will now be something to 'compare' to. :)Of course the 'broken' (and hard-to-wrap-my-mind-around) piece of this scoring black (to me) box is between people who will NEVER be on the same courses. But (correct me if I'm wrong) over the course of a year there should almost always be at least one overlap at some level, possibly several persons removed from a direct comparison. And when I look at the final year end results they generally look 'right' to me ... although of course that's only my gut talking, with no empirical evidence.We could:- let it be: be comfortable with the fact that we're trying to compare apples to oranges (or reds to greens to blues to whites) and there's no great way to do that but this does a good enough job.
- re-examine: invest in putting 'smart people' who know how to do this stuff to 'fix' things.
- revise the goal: stop trying to compare folks across the club, and score intra-class (or course?) only.
While #3 would produce the 'fairest' results, I think there is significant value in cross-class comparison, not to mention that it's just plain FUN!So should we (re-)try #2, in case there are new or different scoring models out there in the world that might more accurately do the job? Maybe! Chuck it at Attackpoint and see what comes back (even my eyes are rolling)? I would like to see -- to make this easily accomplished if some stat PhD want's to get their dissertation on this -- would be for the results data to always be accessible in some standardized way (a CSV export query, or an API that gets straight to the backend) so that folks can 'play'. I might even play with that a bit myself! While I might be able to scrape all of the results data from the website to generate the data to play with (I did this several years ago when the 'new way' was being developed), I don't want to again. :)In the meantime I'm pretty happy with the current system! It's pretty opaque to a simpleton like me, but as I mentioned, when I look at year-end results (the actual GOAL of the product) it seems to meet my instinctual, gut-feeling expectations. -
Reply by hughmac4 on Thu Feb 27, 2014 at 2:27 pmOr if I read Vadim's post more carefully, I would have realized that there must be 4 events before the scores populate. I think! :)
-
Reply by ErikEddy on Thu Feb 27, 2014 at 5:30 pmI see what Clem is saying. There are no ranking values for this year yet. John's 2013 ranking value was 69.2 and Petr's was 76.13. The "Total Score" values are not related to Batsto.
-
Reply by johncampbell on Thu Feb 27, 2014 at 10:46 pmThanks everyone for the responses. I have a better understanding of the system being continuous and the part that matters is the year end. Bottom line, shed some lbs, get more in shape and get my butt out to more DVOA events!
-
Reply by furlong47 on Thu Feb 27, 2014 at 10:55 pmWe had a loooooooong winter meeting the one year where Kent explained how the rankings system worked, complete with a giant wall flow chart with pieces of paper moving around. We probably should have made a video of that
-
Reply by anniemac on Fri Feb 28, 2014 at 1:10 pmCorrect me if I'm wrong but we are not doing Kent's way now, we have changed things slightly since then.
-
Reply by hughmac4 on Fri Feb 28, 2014 at 5:05 pmErik & Clem:Aha! Doh. Indeed, and well spotted (and explained).-Hugh
-
Reply by furlong47 on Sat Mar 1, 2014 at 11:56 pmThere were some changes a year or two ago, but the general structure is still similar.
-
Reply by Vadim on Sun Mar 2, 2014 at 9:51 pmThe world is not perfect and I'm not perfect too ))) The Total Score at the bottom of individual results is a glitch with few more glitches to add. It was Okay when we had Rankings data generated for the year, but it does not work correctly when there is no data yet. Oh, well, time is not on my side anymore, but I'll make a note and add this to the list of to do things.
In general calculations did not change much, we just made Rankings fully integrated into web site without processing on a side.
-
Reply by Len on Mon Mar 3, 2014 at 8:12 amI didnt mind the 'total' score showing a value. but I assumed it was my total from last year.
You could fix this easily by adding an asterisk stating 'The Total score will show last year's value until 4 events have been run in the current year' :)








