#Login Register
Intoducing the new draft sheet!
  • 0 Vote(s) - 0 Average


07-19-2014, 09:26 PM #11
Odoacer AC Operations Team
*
Status: Offline Posts:154 Threads:16 Joined:Apr 2014
Quote:How are the rankings compiled? Is it simply just averaging out each individuals rating based on all the submitted draft sheets? Or is there a more in-depth way you calculate them?

It is not an averaging of results. What you are doing by filling out a draft sheet is preferential voting. There are many different preferential voting systems, each with its own set of pros and cons. I chose the Ranked Pairs method because it has the characteristics we need -- specifically, the ability to deal with lists where some candidates are omitted, the ability to withstand wildly inaccurate rankings, and the ability to deal with lists where the preference-levels are not evenly spread (eg. only listing top-tier players, or listing five top-tier players and one newbie). The wiki page explains the algorithm fully so I won't do it here, but I will address the concerns you posted.


Re: Trolls: The Ranked Pairs method does not do mathematical averaging; it examines each pairing of players and the votes that people are implicitly casting for them, and naturally dismisses data that goes against the majority opinion. Right now you, Bone, are ranked right above Dario on the compiled list. That means that in the contest of "Bone vs Dario -- who would I pick?", the majority of voters chose you. Now if someone makes a list and puts you below everyone else in the draft pool, in 102nd place, we only have to be concerned about your pairings where the vote count is tied. It is unlikely that "Bone vs potdawg" is one of them, so that contest would be unaffected. "Bone vs Dario" could be, so that new list would cast the deciding vote to drop you below him, but then again, listing you at rank 12, just below Dario, would have exactly the same effect. Trolling is not easy to do in this system; I've tried a few different things -- including submitting my list in reverse order -- and the final results were almost completely unchanged.


Re: Unknown players: There's no harm in leaving names off your list. Since you're basically casting votes for "A vs B -- who would I pick?", the omission of a name is simply a non-vote. That contest will be decided by others who did include those names on their lists. Cast votes on the people you know; don't worry about the others.


Quote:The general consensus is having 2 captains in each CTA pick a team simply doesn't work. For one it's time consuming, and it also a lot of times yields uneven teams. This to me though isn't much different, the way I understand it anyways. It's just basically telling every individual out there "Hey you're the captain, now pick a team from this list of players" It's the same system, only on a larger scale.

The idea is to replace the current "captains draft" method of picking teams with one that is better. Not perfect, mind you -- just better. With that in mind, there are three key improvements this new system has over the old one.

1. Instead of sitting in a game for 10 minutes while players are drafted, all of the drafting is done ahead of time, up-front, before the game is even made.

2. Instead of relying on two, lone individuals to accurately judge the skill levels of the other players -- some of whom they may not even know -- we rely on community consensus. We even benefit from the opinions of people not in the game.

3. Instead of alternating picks -- a method that has almost no chance of resulting in optimally-balanced teams whether it's done by captains or not -- we use algorithms to create balanced teams in a way that couldn't be done before. I will explain this point further:

Consider a normal draft where there are 7 above-average players and 1 bottom-tier player. Their ratings are: 10, 9, 9, 8, 8, 7, 7, 1.

The best we can expect from alternating picks (with 2 picks for the penultimate round) would be these teams:

Code:
Green   Red
10        9
9         8
8         7
1         7
-----------
28       31

This is not ideal, but it is what we get. By using a bit of math to divvy up the players for us, we can get slightly better teams:

Code:
Green   Red
10        8
9         8
9         7
1         7
-----------
29       30

Note that the top three players are on the same team. That could never happen in the current system (assuming skill is judged accurately) even though it is sometimes necessary.


Re: Bias and low-tier players: I'm grouping these together because they're essentially the same -- a concern about players who are not accurate judges of skill for one reason or another. I would point out that the "captains draft" raises the same concerns, only to a greater degree. A captain has no one else keeping his biases in check. And a single player is unlikely to know all of the other players' skill levels. Compare the potential failings of one captain (because it only takes one bad captain to ruin a draft) to the potential failings of a consensus-based poll. I'd pick the draft sheet results over the captain any day.

I do think there is value in the lists that low-tier players create. They are often ignored by top-tier players, so they are likely the best ones to ask how well they stack up against their peers. I certainly have no idea if kinggrimet is better than Zero7, and I don't pay attention to their draft order, but they probably would. Should low-tier players be able to vote on higher-tier players? Yes, I think even they can tell you that ElAsesino is a tougher opponent than Turbo. Maybe they get some of the closer calls "wrong", but from what I've seen so does everyone else, and like I explained above, minority opinions have a very limited impact on the quality of the final list. Besides, the more players -- low-tier or not -- casting the "obvious" votes, the more tamper-proof the system becomes.
This post was last modified: 09-10-2014, 10:35 AM by Odoacer.

07-19-2014, 11:59 PM #12
Astrok n00b
*
Status: Offline Posts:31 Threads:0 Joined:Apr 2014
I like this system and I think it will yield better results than captains (particularly with the draft order we use which imo is not an ideal one considering most player pools in a game).

I'm just wondering something and I know you've already considered it, what happens to multiple names? For instance, I'm in there as SnowWhiteZorro but I will likely not be using that name much anymore (if at all). Are other aliases manually 'linked' to a main alias or would you have to input an entirely new name?

07-20-2014, 02:14 AM #13
Bone Still n00b
*
Status: Offline Posts:239 Threads:37 Joined:Apr 2014
Thanks for the explanation Odo. If it has me higher than Dario, then it's way more accurate than I initially thought.

07-20-2014, 06:39 AM #14
Odoacer AC Operations Team
*
Status: Offline Posts:154 Threads:16 Joined:Apr 2014
(07-19-2014, 11:59 PM)Astrok Wrote:  I'm just wondering something and I know you've already considered it, what happens to multiple names? For instance, I'm in there as SnowWhiteZorro but I will likely not be using that name much anymore (if at all). Are other aliases manually 'linked' to a main alias or would you have to input an entirely new name?

The names are pulled from the list of aliases used in CTA games in the past 30 days. Then they are run through namecheck and the aliases are removed, keeping the alias that has the most games played (in the past 30 days). Your new name will appear in the draft pool when it has more games than SnowWhiteZorro. Because old aliases aren't removed from existing draft sheets, it may be possible to list the same player multiple times on a sheet, but namecheck is also used on all the sheets before they are compiled, so that's not a problem.

07-20-2014, 06:53 PM #15
Cheejudo n00b
*
Status: Offline Posts:93 Threads:6 Joined:May 2014
Bone, you don't have to be a mid or high tier player to be able to know who is better in comparison to others. That is a pretty ridiculous statement.
I can't play my guitar very well, so obviously I can't recognize when a musician is any good. I'm white, I cant dance .. obviously I can't understand when someone is skilled at dancing. Im sure ALL coaches of professional sports teams are or were elite top tier athletes.. right?

ABChee REIGN SUPREME
[7:05:19 PM] FwitG : im more attractive and successful than you'll ever be 'abchee'

07-20-2014, 11:30 PM #16
benevolence AC Administrator
*
Status: Offline Posts:397 Threads:35 Joined:Mar 2014
I get what you're trying to say Chee but I think you might be over exaggerating it. I think Bone's point is that newer/less-skilled players don't have the knowledge to accurately rank players that are somewhat similar skill wise. Obviously everyone will be able to determine that yoshi is a skilled player(for example), but would the newer/less-skilled players have the knowledge to determine whether he is more or less skilled than akma/tobiashi/flash/unreal/tengo/astrok etc who are all relatively close skill wise?

Anyway I think Odo has definitely shown that there is a lot more to this ranking system than a bunch of people putting players in some sort of order. In the last couple of days we have tried this system a lot in CTA games and I personally have only seen 1 or 2 that have been unbalanced. Considering it has only just been implemented, and considering that there is more fine tuning to do(from what I understand), I think it is a pretty good start.

07-21-2014, 05:19 AM #17
Nuts n00b
*
Status: Offline Posts:30 Threads:1 Joined:Apr 2014
Is there a way to make a tab where spectators can be listed in order of ranking inside games and have it include name checking for smurf names to rank them where they should be? Unless the idea is to totally abandon captain picking that is. I think it would help with more people wanting to captain if they had that as a guideline for picking.

07-21-2014, 08:30 AM #18
niveus AC Developer
*
Status: Offline Posts:362 Threads:64 Joined:Mar 2014
Nuts I think you have the concept slightly skewed. We are moving away from captains and team picking in the server. Our algorithm will take care of balancing teams, this should resolve a few of our main issues such as taking 10 minutes to start a game because of selecting teams. This should also make a refs life easier, in the sense that they will make less decisions.

The algorithm is hooked up to the namecheck therefore smurfs will no longer be an issue Smile
This post was last modified: 07-21-2014, 08:30 AM by niveus.

07-21-2014, 11:27 AM #19
Nuts n00b
*
Status: Offline Posts:30 Threads:1 Joined:Apr 2014
Yeah thats why I said unless we are giving up on captains picking completely. So far it seems to be working out pretty well. Havnt been in many blowouts yet and should only get better as more people fill theirs out.

07-23-2014, 01:28 PM #20
cyro Forum Administrator
*
Status: Offline Posts:743 Threads:153 Joined:Jan 1970
(07-23-2014, 01:09 PM)gates Wrote:  Is there a possibility we'll get a released rankings list of the average voted player rankings?

It's on the to-do list Smile

tocs.






Forum Jump:


Users browsing this thread:
1 Guest(s)