Forums Home / The Lobby / Seeding FAQ (
View Older Thread |
View Newer Thread)
First |
1 |
Last
| fractalzoom
- 13 Aug 2011
Total Posts: 201
|
I came up with all the advantages and disadvantages of the current seeding system and the rating system. I tried to be as evenly handed as possible.
I've posted this on my blog - which may be easier to read (www.pucksnmallets.wordpress.com). It's long, but bear with me.
Current seeding system:
Q: Hey man, how's this thing work?
A: Pretty simple. Just look at the last major tourney and seed everyone that way they finished last time.
Q: Well that's easy, huh? Well... wait. What about people who've never been to a tourney - how are they seeded?
A: Meh - we just randomly throw them somewhere on the bottom. Unless we *know* them, then we put them where we think they should go.
Q: Sound pretty arbitrary to me
A: Nah... it's not really arbitrary, it's just that if you've never been to a major tourney we have to have allowances for this. Take Randal Leistikow, he'd never been to a tourney, but we know he had beaten some good Colorado players and earned his rank through a challenge match victory. We think he got seeded correctly.
Q: Well.. what about someone who went to one tournament, but missed the most recent one?
A: We have a system for that. For each major tournament you miss... you drop 3 spots. Then we tack on .5 right before the tourney to make sure you don't share the same spot as someone else. There are limits to this, though.
Q: Such as...?
A: Well... we classify players. Say someone finishes master a couple times. This player is established as an ultra-pro player, and if they miss a tourney on top of having a bad outing, it would be unfair to the person who had to play this person early on. So, we put them at the bottom of the "ultra-pros" which is 12th. This is what happened with Travis for this tournament. He missed the last tournament - and finished a little bit lower than master in the last tourney he competed in.
|
|
|
|
| fractalzoom
- 13 Aug 2011
Total Posts: 201
|
Q: Ah.. ok. So let me summarize. The basic structure is the finish of the last tourney, plus we fudge for non-participation, and we fudge for *established* players.
A: Yes. Of course, the non-participation punishment isn't really that severe, and although the number 3 for losing rank is arbitrary, it's applied evenly to everyone. I guess the biggest point of contention is this last point. Someone could look at Mark Robbins and say he's an ultra-pro, and someone could look at him and just say he's a pro. You look at his last tourney (his lowest finish ever) and say that he's definitely a pro, but look at the history of all of his playing - and you could justify him being an ultra-pro. This is the part of the current system that gets the most flak. It's based on how well "known" this player is. A player from outside the established community would have a hard time getting properly seeded. This system seems to work well for the size of air hockey right now, but it doesn't scale up very well should we have tournaments with 200 or more players. Imagine how a tournament director would seed a tourney in such a way. I might play a major tourney with 60 people early in the year and finish 30th, but if were put in a tourney with 120 people - should I be seeded 30th? Or 60th? The current system would say 30th... and you would benefit from playing in smaller tournaments with a weaker field. Not only that, but if we were to have a tourney with 200 players, figuring out exactly where to seed a new player would be incredibly difficult if not impossible. How do you determine if someone should be put in at 80th, or 160th? Players are also terrible at self-analyzing. Cory Dzbinski came to his first tournament in Vegas and if you would have asked him what his rank would be - he'd probably tell you he was a "4", but he's closer to a "2" (sorry Cory).
Q: Any other ways you can change your seed?
A: Yes. Play a challenge match. If you think there's someone ranked higher than you - play a challenge match, which is a grueling 3 of 5 sets or 4 of 7 sets (depending on how the defender decides). If you challenge a higher ranked player and win - you take his spot, and everyone between your ranks drop one.
|
|
|
|
| fractalzoom
- 13 Aug 2011
Total Posts: 201
|
Q: How many chances do you get to challenge someone? Can they decline?
A: As many times as it takes, and they can't decline, but a defender can push off the acceptance of a challenge set a certain amount of time. Challenge matches are fun ways to encourage players to travel to play epic and very meaningful games. If you beat the #1 guy - you get to be ranked #1 in the world! The downside of this is "regional clumping". Two years ago Dan Meyer played in his first major tournament in Houston. A bunch of guys from Chicago finished in various spots (Mike Y finished 33rd, Dan 49th, Nick H 53rd). So, after the tourney, they all played challenge sets. Nick beat Mike, then Dan beat Nick, and then Mike beat Dan. So, their rankings changed from 33rd, 49th, and 53rd to 33rd, 34th, and 35th.
Q: Have other problems come up?
A: Yes. In the 2009 tournament, Mark Nizzi (a guy who generally finishes in the top 5) dropped out of the tournament mid-way through... and finished 48th. According to the system, this is his rank. Now, before the next tourney, he'd get moved up to a reasonable seed (top 10, at least) but in the meantime, here he is with this low ranking. What if he wants to challenge the #25 guy for his rank? He's technically below that person and can challenge for a higher rank. Should he be able to? Mark could challenge anyone in the top 16 to get the right matchup going into the next tourney. If he picked up the #8 rank to get the matchup he wanted... would the tournament director move him up to #6? A small movement in seed could change the matchups for all the top players where there is a style advantage, and someone would get the short end of the stick. What if Mark challenged the #16 ranked person and lost? Would he justify a top 10 seed? Where exactly would you put him?.. it becomes a difficult position to put any tournament director. Where Mark was placed in the 2011 Vegas tourney was ultimately determined by vote, and this displeased a number of players.
Q: Ok, I get it, I get it. The current system tries to be as fair as it can... and it has those challenge matches, which are really cool despite you talking about "regional clumping"... who comes up with this stuff anyway? So, this all well and good. Are there any other ways to seed a tourney?
A: Yes. Goran Mitic a couple years ago proposed we keep track of our scores and rate air hockey players statistically using the Elo method.
|
|
|
|
| fractalzoom
- 13 Aug 2011
Total Posts: 201
|
Q: Wait, let me stop you right there. I hate math and all of its witchery. I came here to play air hockey, not muddle with numbers all day.
A: Can you add and multiply?
Q: Yeah - that's easy. I thought you were gonna get all crazy with statistiacal cluster analysis, Hilbert spaces and Ferma's last Theorem on me. I can add and multiply.
A: Ok, then. Le's start from the beginning where you're not rated just yet. Goran thought that in order to get rated, you should have to play 5 different people a best of 7 set. We figure that if you play 5 different people, that we'd have a good idea of how skilled you are. So, le's say there is a new player who wants to get established. He (player A) starts with 1,200 points (everyone starts exactly the same)- and he plays someone (player B) with 1,300 points. So, we take the difference in ratings (1,300-1,200=100) and look at the chart here: http://airhockeyworld.com/ahw_rating.asp. That Elo guy calculated the odds someone would win based on this system. So, with a difference of 100 points, Player B is expected to win 63% of the time.
Q: What does that mean?
A: Well.. according to the ratings, Player B is expected to win ... so if he beats Player A- he'll get some points, but not as much as if Player B pulled an upset. Does that make sense?
Q: So- I get more points for beating someone really good, but not as many for beating someone not as good?
A: Exactly.
Q: So, how many points exchange hands? How is that calculated?
A: Well. Context is important here. We, as a community, put more emphasis on big tournaments and challenge matches than when players just get together to play during a weekly event. So, we can multiply the final result by a factor of "importance" or "value of match". Ok ... so here's the basic calculation:
Rating' = Rating + Value Of Match * (Set Won or Lost - Win Expectancy)
Rating' = New rating
Rating = Old rating
Value of match = (see chart below)
Set Won Or Lost = 1 for win, 0 for loss
Win Expectancy
I's pretty simple, here's an example:
Example 1: Player A has a rating of 1200 and Player B has a rating of 1300. Player B is expected to win 63% of the time, based on their rating.
If player A wins a challenge set his new rating would be 1219 =(1200 + 30(1-.37)) and player B would have a new rating of 1281 (1300 + 30(0-.63)).
If it was reversed, and player B won ... because player B is *expected* to win more often than not, the point swing isn't as great. If A lost, he would only be 1189, and B would only go up to 1311. An 11 point swing instead of a 19 point swing.
|
|
|
|
| fractalzoom
- 13 Aug 2011
Total Posts: 201
|
Q: Ok, I get it. Pretty simple math. So ... one thing bugs me. How is "value of match" determined? Seems arbitrary that it's 30 for normal Mitic sets, 60 for State Tourneys, and 90 for Nationals.
A: True, but Goran has run the numbers with different factors. Thing is, with Air hockey ... we don't have a ton of matches entered into the database. If everyone played hundreds of mitic sets each year ... we could have a low factor, and you'd slowly creep to a very accurate rating. Right now ... after you play a dozen or so sets, your rating should be pretty accurate. Also, some players tend to play each other *a lot*. Instead of having lots of challenge matches and seeing this "regional clumping", you'll get an accurate proportion. Let's use Dan and Q as an example. If you look at their history, they each take sets off of each other. Q wins slightly more than Dan", but if Dan were to challenge Q for his #9 spot, after a couple challenge matches ... odds are good that Dan would win one of them, and then they'd be ranked #9 and #10. But if they just play for ratings, if Q wins 63% of the time ... the Mitic system will push them so that Q will average about 100 points more than Dan to reflect Q's actual win percentage. Now ... back to "value of match" ...the higher the number, the bigger point swings there will be. There should be a balance. Too high point swings, and someone could be rated 12th, then play a couple matches and go all the way to 24th, then back up to 16th. A lower value would just take someone many, many matches to get to where they should be in terms of rating, and if someone makes a big jump in skill ... it'll take more matches (data points) to adjust their rating to the proper spot. Ultimately, this number is used equally for everyone, so long as these values are consistent, it doesn't really matter what the values are... the system will work.
Q: Ok.. I think I'm getting it. So.. the other system had some fudge factors to deal with people missing tourneys and whatnot. How does the Mitic system deal with some of these complicated problems?
A: After 2 years (1 year) of inactivity players with a rating total of more than 1200 will lose 10% of the difference of their rating and 1200. This does not have any effect on the calculations of prior matches.
After 2 years of inactivity the player will be dropped from the rating list. If the player plays after they drop they will be treated as a new player. This does not have any effect on the calculations of prior matches.
|
|
|
|
| fractalzoom
- 13 Aug 2011
Total Posts: 201
|
Q: Any other rules I should be aware of?
A: You must play 5 different players with at least 1 player having an official rating to earn an official rating. Until you have an official rating, your rating is "provisional". When a player with an official rating and a player with a provisional rating play a match, only the rating of the player with the provisional rating will be affected. This is so an established player isn't punished by someone who may be really good and is way underrated. Playing new people should be encouraged.
Q: So ... what does this have to do with seeding?
A: Seeding a tourney with this system in place is that we can rate everyone who's been keeping track of their sets. You just lay everyone out by Mitic points, and that's your seeding list, which makes the process very simple and completely objective.
Q: Yeah ... but what if I don't want to keep score... I just want to play for fun.
A: It's neat to have a record of all your matches. You can track just how well you were at any point in time. Wouldn't you like to see how you've been performing over time? Most players like to have some kind of record of that time when you finally pulled out that upset against a rival.
Q: But the seeding isn't done like this, anyway ... why should I care to record all my sets?
A: Well. Someday, we may decide that a statistical ranking system will be better for seeding. Until then, recording your scores will help us evaluate the Mitic system so we can see how accurate it is. Plus, we'll have a record for how well you've been performing ... for many players, having this record in a database is nice to have.
Q: Well ... I play in Houston, and we play handicapped best of 5 sets. How do you apply Mitic ratings for this?
A: Right now, you don't. There is a way you could do it, but the math gets messy. I know Houston doesn't like to pummel beginners into bloody paste as it can be very discouraging... but more established players have begun to waive their point spread so that they can have their set count towards the Mitic ratings. Even if the set isn't a best of 7 for a normal Mitic set ... Goran has been working on what kind of "value of match" values one could do for a "best of 5", or even a single game. Having this in place lets completely new players that aren't willing to play a best of 7 get into the system.
|
|
|
|
| fractalzoom
- 13 Aug 2011
Total Posts: 201
|
Q: So, let's say I have a regular practice partner who's pretty good and beats lots of other people, and I match up very well to this person, and I just beat him all the time... isn't beating the same person over and over a limitation in the Mitic system?
A: This has happened. Again with the example of Mark Nizzi. He and Keith Fletcher (both highly rated players) played many sets, and Mark has won them all. Looking at the difference in Mitic points... Keith should have won more than what happened and the points are skewed towards Mark. Goran has considered having another factor for limiting the number of times one can take points from a single person over a certain period of time to deal with this because certain style matchups can give a player a skewed advantage in Mitic points.
Q: So many numbers! I thought I had it figured out, but there are all these weird rules in place!
A: Not really. The number that changes is just that "value of match" for the most part. The others are statistical ways we align the points, much like the current system punishes players for not showing up in a tourney.
Q: So, let's see if I get the basics... if you win a match you go up based on the skill of your opponent and the importance of the set. If you lose, you go down the same.
A: I think you got it.
|
|
|
|
| fractalzoom
- 13 Aug 2011
Total Posts: 201
|
Q: Ok, ok. So, what are the main differences between the two - any advantages one has over the other?
A: Well, the current system's big advantage is in challenge matches and encouraging really important challenge matches when there are no tourneys going on. The Mitic system acknowledges this, and adds a point bonus to these challenge matches so that there's a bigger point swing (and making it more significant). There are 2 ways in which the Mitic system deals with complicated problems better than the current system. The first is the "Nizzi issue"... in the first case where Nizzi drops out of a tournament early. Instead of losing his rank from 5th to 49th - he will lose Mitic points, but it won't be as drastic of a move. Nizzi will be nudged down, but the movement won't be nearly as drastic. Nizzi can then play anyone he wants to recover those lost Mitic points, and he'll benefit by getting a better seed in the next tourney (should we adapt the Mitic system for seeding). The other thing is in large tourneys with many unknown players. It's much easier to get a Mitic rating by playing people in your area than it is to make it to a national tournament. If a player from Russia were to show up to one of our tournaments, we can look at his Mitic rating to see where he would be seeded. Without the rating system in place, the current system would have a difficult time figuring out where that person should be. The current system does a good job of seeding 1-32, but has a problem dealing with seeding people much further down the line. The Mitic system also does a better job of telling people "how good they are" compared to the field. There may be a big jump in skill between 7th and 8th place, but the current system doesn't measure "how much better" the Mitic system gives you a better idea how you compare to everyone else. The Mitic system also encourages playing more frequently. Each set gets you closer and closer to your true rank. The more data in the system, the more accurate the system is. One downside to the Mitic system is that it takes a lot of work to maintain this database, and if tournaments aren't being seeded in this manner, there's less incentive to record your scores.
Q: Well then, that was much longer than I thought it was going to be.
A: Yes, sorry about that.
|
|
|
|
| fractalzoom
- 13 Aug 2011
Total Posts: 201
|
| sorry.. tried to go in and edit the goofy characters... got a couple.
|
First |
1 |
Last
Forums Home / The Lobby / Seeding FAQ
|
|
