| fupersly
- 12 Aug 2014
Total Posts: 231
|
Apologies in advance if this ends up making no sense at all, but I wanted to throw this out there (possibly to my own detriment) in case it causes the synapses of the more intelligent/knowledgeable folks out there to fire in (even partial) agreement.
Basically, I was wondering about this during the tournament (and even asked a few other folks) if it would make sense to enter the results of matches in a tournament in reverse order (from the end of the tournament to the beginning) in order to get a more accurate snapshot of player skill level.
Now, before anyone goes crazy telling me I'm crazy, I think my reasoning for this is sound - that is, by the time you reach the end of the tournament, you are playing players that are closer to your actual skill level (if not your rating), so any differences between your current rating and the rating of those you play at the end of the tournament are a better indicator of your actual skill than what your rating dictated at the beginning of the event. This is especially true for players with ratings that are based off of a small sample size, or who play in much more isolated player bases.
Take Doug Howard, Gabe Garboden, Jeremy McKiernan, and Rafael Flores as test cases for this theory. If you look at their ratings, they're all in order compared to their finishes:
Jeremy - 1247.84 (25th)
Doug - 1143.04 (26th)
Gabe - 1071.92 (39th)
Rafael - 1017.6 (42nd)
However, Jeremy and Doug competed in a bracket with players having much higher ratings than theirs, while their wins and losses early in the tournament were against players with significantly higher or lower ratings, making those matches far less telling than the ones they played toward the end of the event where their play throughout the event more accurately slotted them. As such, I would expect their ratings to be well above Gabe's and Rafael's and more in line with the players they faced off with in their bracket (mostly 1400-1500s).
I'm not saying this is definitely a better way to go, but I am curious what the difference in the ratings would look like if the calculations for this whole tournament were run chronologically backwards rather than forwards. Is this something that could be done easily enough that it wouldn't be terribly painful to compare and see what it would look like? Or am I entirely off base here and it's not even worth checking it out?
|
|
|
| goran
- 13 Aug 2014
Total Posts: 428
|
| Ratings for tournament can be calculated in order, or all at once for a tournament. Our program is set up to do it in the order the matches are played. Reverse order does not make sense. It would do a minor mess up to the ratings, but would be fixed over time through sets and challenge matches.
|
|
|
|
| fupersly
- 13 Aug 2014
Total Posts: 231
|
goran said: | Ratings for tournament can be calculated in order, or all at once for a tournament. Our program is set up to do it in the order the matches are played. Reverse order does not make sense. It would do a minor mess up to the ratings, but would be fixed over time through sets and challenge matches. |
Well then, how different would it be to do the "all at once" approach vs. in order? Maybe that's more in the spirit of what I was suggesting, even if my particular logic wasn't entirely sensible. 
|
|
|
|
| goran
- 13 Aug 2014
Total Posts: 428
|
| Manually, it's easier to do each persons results all at once, the system we use on ahw applies tourney matches in order. Q or Travis might know how much effort it would take to change. I don't believe it would have a huge effect on the results. The current way is better because it corrects ratings in real time. All at once could have an unknown start at 1000, and effect everyone one he plays at their 1000 rating instead of going up after each win.
|
|
|
| tableman
- 13 Aug 2014
Total Posts: 690
|
Doesn't it make sense to calculate ratings in chronological order? If I play a match in May, and then one in June, it makes sense to calculate the May match first, then the June, otherwise the point changes caused by the June match would be skewed.
Same principle in a tournament but the time frame is much more condensed. Doesn't it similarly make more sense to calculate a match I play at 3 pm on Saturday before a match I play at 7 pm Saturday? Etc.
|
|
|
| jasonstevens
- 14 Aug 2014
Total Posts: 176
|
tableman said: Doesn't it make sense to calculate ratings in chronological order? If I play a match in May, and then one in June, it makes sense to calculate the May match first, then the June, otherwise the point changes caused by the June match would be skewed.
Same principle in a tournament but the time frame is much more condensed. Doesn't it similarly make more sense to calculate a match I play at 3 pm on Saturday before a match I play at 7 pm Saturday? Etc. |
It does to me. If they aren't done in the order played then it makes ALL the data inaccurate.
|
|
|
|
| fupersly
- 14 Aug 2014
Total Posts: 231
|
jasonstevens said:
It does to me. If they aren't done in the order played then it makes ALL the data inaccurate. |
I think the point I was trying to make (that I'm not sure if Mark is supporting with his comment or not?) is that yes, matches that take place a month or more apart should definitely be taken as individual events for the purposes of ratings calculations. However, let's go back to the Jose Mora example in the mid-to-late 90s and think about what would happen in the ratings in that scenario:
Round 1, an "unrated Jose" plays a player with a much higher rating and beats them. While it's probably correct for Jose to take a lot of points from that match-up, the losing player is going to get heavily dinged because they lost to a far lower-rated player who actually wasn't that much lower in skill than they were (and in fact, probably has a considerably higher skill level). This would be more obvious when looking at the final standings to see that "unrated Jose" should have been rated significantly higher and has done significant damage to the ratings of several players in the process.
My logic (convoluted as it may be) then, is this: if "unrated Jose" was looked at through the prism of who he was playing when he finished the event, we would have a more accurate model of his skill level when tracing it back to that first player he played. Pretend for a minute that it's Mark Nizzi (the first sub-2000 player in the current ratings, to avoid the 1/2 K-value protection) who gets beat in that first-round match - when considering the adjustment in points, he should probably only lose a fraction of the points compared to the points he would lose if "unrated Jose" were more accurately rated at that point. An "unrated Jose" who finishes in the Top 5 and knocks off several highly-rated players to get there should quite obviously have a higher rating, regardless of how the calculations are done.
And before anyone says there are no "unrated Jose's" out there, there will be eventually. Also, just because someone isn't Jose and the spread of the effect won't be as large, if the goal is to have the most accurate ratings possible, I think evaluating a player's performance within a single tournament is probably more accurate than looking at it as individual performances throughout the rest of the year against a less diverse group of players.
|
|
|
| Q
- 14 Aug 2014
Total Posts: 497
|
| If player unrated, they don't influence opponent rating. (like an incubation period) until they play 5 people and 20 games
|
|
|
|
| fupersly
- 14 Aug 2014
Total Posts: 231
|
Q said: | If player unrated, they don't influence opponent rating. (like an incubation period) until they play 5 people and 20 games |
Okay, but what if "unrated Jose" did that against 5 people who were all of low ratings, then the scenario I painted plays out. Would it be more or less the same thing, or am I just totally not understanding how the rating system works?
I still don't understand how what I'm suggesting here doesn't have at least some value to it. In fact, isn't the "incubation period" even worse in the case of "unrated Jose", since by the time he would be near the end of the tournament, his non-influence on his opponents' ratings would cause him to have an even bigger impact on the players he might be beating in his 6th, 7th, or 8th match?
|
|
|
|
| Q
- 14 Aug 2014
Total Posts: 497
|
Just correcting facts if I see errors.
- A new player starts at 1000 until they play 5 people & 20 games. Their rating changes per the regular rating calculations. Their opponent (unless also new) does not have a rating change. There are 3 tiers of players: New, Inactive, Active
- You mention the "1/2 k protection". That no longer exists. The k-value is the same regardless of a player's rating.
|
|
|
|
| fupersly
- 15 Aug 2014
Total Posts: 231
|
Q said:
- You mention the "1/2 k protection". That no longer exists. The k-value is the same regardless of a player's rating.
|
Interesting - I must have missed the memo on that. (or more likely forgot, since I'm pretty good at that, too)
|
First |
1 |
Last
Forums Home / Tournaments and Challenges / "Reversed" Ratings Calculations?
|
