Page 1 of 1

Rating system - Implementing checks and balances

PostPosted: Mon Aug 18, 2008 12:30 pm
by alster
Concise description:
  • A check requiring a new player to finish a certain number of games (e.g. 25 or 50) before being able to rate another player.
  • A check deleting all ratings handed out by an account being deleted (e.g. for rule violations).
  • A balance deleting given ratings after a certain period of time (e.g. one calendar year).


Specifics:
Self-explanatory from the concise description.


This will improve the following aspects of the site:
  • Generally I get the feeling that a lot of criticism of the newly implemented rating system is caused by (i) perceived unfair ratings that cannot be moderated, only responded to by writing and (ii) the fact that the ratings (no matter how outrageous) will stick to the end of time. By implementing a few checks and balances, I believe that quite a few unfair ratings can be prevented from even being handed out as well as causing quite a lot of players to come to terms with the new rating system as they know that in the long-run, it will indeed even out. Most importantly though, the suggestions here would, if implemented, not require any moderation (which I believe was one of the underlying purposes of implementing the rating system to begin with).
  • A check requiring a new player to finish a certain number of games (e.g. 25 or 50) before being able to rate another player. This will cause ratings to: (i) be handed out only by players reasonably dedicated to the site, (ii) be handed out by players who have become somewhat familiar with “common” standards and have had the time to ponder what kind of rating standard he or she wishes to abide by, and (iii) avoid (albeit rare) situations where multis are created for the sole purpose of hunting down a particular individual player from also pushing down the targeted player’s ratings.
  • A check deleting all ratings handed out by an account being deleted (e.g. for rule violations). This will get rid of upset and unsportsmanlike individual’s ratings, the later which may very well be as outrageous as the rule violation itself. One can, for example, imagine a player being busted as a multi using his last games to take revenge by handing out straight 1-1-1s. EDIT: Multis ratings shouldn't be there in the first place either; one player one rating seems more reasonable.
  • A balance deleting given ratings after a certain period of time (e.g. one calendar year). It is not uncommon, even though the exception, for players to get into brawls and fights on this site. This may lead to the players handing out shitty ratings, using the ignore function and then never see each other again. By letting all ratings expire after a certain period of time, ratings handed out after such events will eventually disappear. Further, it is also not uncommon (but again, an exception) that some players simply hand out outrageously bad ratings just for fun or out of spite. Again, by letting all ratings expire after a certain period of time, ratings handed out after such events will eventually disappear. At the end of the day, as ratings disappear, all players’ ratings will reflect what has occurred during the specified time period (e.g. then during the last year). As the rating system should reflect some sort of “community standard” or “peer-review” it seems fair that the ratings only go back that far. This especially in cases where perhaps a player has indeed changed his ways, perhaps he or she was obnoxious in the past but has matured into a much nicer player over time. Then this suggestion would also reflect in his or hers ratings.

Re: Rating system - Implementing checks and balances

PostPosted: Mon Aug 18, 2008 1:16 pm
by King Dain II
Excellent idea. Could you at a check/balance which goes: the top 5% and bottom 5% of ratings are deleted, this should not change the average, but it does delete high ratings given just because the members are friends and low ratings given for no reason.

Note: not my idea, but I think it is great. Someone posted it somewhere earlier on in a ratings discussion.

Re: Rating system - Implementing checks and balances

PostPosted: Mon Sep 15, 2008 7:32 am
by alster
Jebus... was this such a blatantly bad suggestion(s) that the thread just died?

Re: Rating system - Implementing checks and balances

PostPosted: Mon Sep 15, 2008 7:44 am
by hiddendragon
Why is it when you defeat opponents they leave negitive ratings, Game 3230257, this is just one example of like 20 negitives I have recieved...is there anyways to clear them? is it right to be rated negitively simply for winning? Is there some way to punish those who leave negitives simply because they are poor sports?

Re: Rating system - Implementing checks and balances

PostPosted: Thu Sep 18, 2008 9:07 am
by PLAYER57832
alstergren wrote:Concise description:
  • A check requiring a new player to finish a certain number of games (e.g. 25 or 50) before being able to rate another player.


No .. this sounds good at first thought, but the problem is some folks already bash new people. I think having new people join is a GOOD thing, (and know CC feels it is!) we want to encourage GOOD behavior, helping not encourage people to diss newbies.

That said, I know I went back and looked at some of my earliest comments (under the old feedback system) and was sorry I could not change them later. Now, you can, BUT only if you play that person again.

Instead of a "no rating until 25-50 games", I would like to see a SPECIAL rating for newbies only. It would be a "2-way" system. New people would have a special newbie only rating tag set. We would get to rate them "normally", but the ratings would be colored differently, indicating this is a newbie rating. This would be indicated somehow in the details section also -- maybe an asterisk or a different color to indicate this rating was given to them when the person was new. Then, after a certain time period -- maybe 6 months, maybe 500 or 1000 games, the newbie rating would either be combined with the regular rating (BUT the original detail screen would still show that it was a newbie rating, so that folks who actually look into the details can see that the person might have changed, BUT it would no longer be shown for the "quick glance") In most cases, the person will have played the rater again and the old rating will have been replaced, but this way the information is retained, but will only affect the overall score while the person is still relatively new. It will also make it easier for re-raters to see. If I rated someone poorly initially, but see that I gave them that rating when they were new, then I am much more likely to just replace the rating, rather than trying to "average" as I would if I knew the rating was for when they were already established as a player and already knew CC etiquette.

alstergren wrote:
  • A check deleting all ratings handed out by an account being deleted (e.g. for rule violations).


  • EXCELLENT IDEA!!!!

    The poorest ratings I have gotten are from people who were subsequently banned and such. You could sort of tell under the old system just based on the type of comments those folks left, but now ... you have to actually know that this person is a jerk by name. I know the games themselves cannot be deleted, but this would be a nice compromise.
    alstergren wrote:
  • A balance deleting given ratings after a certain period of time (e.g. one calendar year).


  • This was suggested before. The idea now is that any rating will be changed after you play the person again. I think the basic problem is in the above 2 issues. Other ratings really should stay. In the case where a long-standing player really goes through a complete change, then the ratings will show that over time. BUT, since the old ratings were accurate, they should stand. The exceptions, as I said above, are new people who just don't know CC "rules" yet (either to rate or in the case of being rating for making judgement errors, not following CC etiquette) and those who are just all-around jerks and who eventually get ousted as a result. The rest falls in the range of "you are entitled to your own opinion". Subjectivity is part of the system.

    alstergren wrote:Specifics:
    Self-explanatory from the concise description.


    This will improve the following aspects of the site:
    • Generally I get the feeling that a lot of criticism of the newly implemented rating system is caused by (i) perceived unfair ratings that cannot be moderated, only responded to by writing and (ii) the fact that the ratings (no matter how outrageous) will stick to the end of time. By implementing a few checks and balances, I believe that quite a few unfair ratings can be prevented from even being handed out as well as causing quite a lot of players to come to terms with the new rating system as they know that in the long-run, it will indeed even out. Most importantly though, the suggestions here would, if implemented, not require any moderation (which I believe was one of the underlying purposes of implementing the rating system to begin with).
    • A check requiring a new player to finish a certain number of games (e.g. 25 or 50) before being able to rate another player. This will cause ratings to: (i) be handed out only by players reasonably dedicated to the site, (ii) be handed out by players who have become somewhat familiar with “common” standards and have had the time to ponder what kind of rating standard he or she wishes to abide by, and (iii) avoid (albeit rare) situations where multis are created for the sole purpose of hunting down a particular individual player from also pushing down the targeted player’s ratings.
    • A check deleting all ratings handed out by an account being deleted (e.g. for rule violations). This will get rid of upset and unsportsmanlike individual’s ratings, the later which may very well be as outrageous as the rule violation itself. One can, for example, imagine a player being busted as a multi using his last games to take revenge by handing out straight 1-1-1s. EDIT: Multis ratings shouldn't be there in the first place either; one player one rating seems more reasonable.
    • A balance deleting given ratings after a certain period of time (e.g. one calendar year). It is not uncommon, even though the exception, for players to get into brawls and fights on this site. This may lead to the players handing out shitty ratings, using the ignore function and then never see each other again. By letting all ratings expire after a certain period of time, ratings handed out after such events will eventually disappear. Further, it is also not uncommon (but again, an exception) that some players simply hand out outrageously bad ratings just for fun or out of spite. Again, by letting all ratings expire after a certain period of time, ratings handed out after such events will eventually disappear. At the end of the day, as ratings disappear, all players’ ratings will reflect what has occurred during the specified time period (e.g. then during the last year). As the rating system should reflect some sort of “community standard” or “peer-review” it seems fair that the ratings only go back that far. This especially in cases where perhaps a player has indeed changed his ways, perhaps he or she was obnoxious in the past but has matured into a much nicer player over time. Then this suggestion would also reflect in his or hers ratings.


    I wrote my comments above.

    As for the "peer review" part... see Jiminskis "community review" suggestion.

    Re: Rating system - Implementing checks and balances

    PostPosted: Mon Sep 22, 2008 4:14 pm
    by alster
    PLAYER57832 wrote:This was suggested before. The idea now is that any rating will be changed after you play the person again. I think the basic problem is in the above 2 issues. Other ratings really should stay. In the case where a long-standing player really goes through a complete change, then the ratings will show that over time. BUT, since the old ratings were accurate, they should stand. The exceptions, as I said above, are new people who just don't know CC "rules" yet (either to rate or in the case of being rating for making judgement errors, not following CC etiquette) and those who are just all-around jerks and who eventually get ousted as a result. The rest falls in the range of "you are entitled to your own opinion". Subjectivity is part of the system.


    The problem is e.g. that (i) people quit and (ii) people may very well never play again. Also, (iii) the rating given may be wrong.

    The suggestion in question removes the impact in these situations without compromising the underlying principle of the rating system (i.e. a peer review system not demanding moderation).

    PLAYER57832 wrote:As for the "peer review" part... see Jiminskis "community review" suggestion.


    No, that has nothing to do with my suggestion.

    cf. http://www.conquerclub.com/forum/viewtopic.php?f=4&t=53597&hilit=Jiminski

    I simply used those words in order to describe what the ratings should show (i.e. not adding an additional layer).

    Re: Rating system - Implementing checks and balances

    PostPosted: Thu Sep 25, 2008 4:21 am
    by windowmere
    PLAYER57832 wrote:
    alstergren wrote:Concise description:
    • A check requiring a new player to finish a certain number of games (e.g. 25 or 50) before being able to rate another player.


    No .. this sounds good at first thought, but the problem is some folks already bash new people. I think having new people join is a GOOD thing, (and know CC feels it is!) we want to encourage GOOD behavior, helping not encourage people to diss newbies.


    Maybe we shouldn't be able to rate newbies until they have a few games under their belt - who wants to be judged by how they did on their first try?

    alstergren wrote:
  • A balance deleting given ratings after a certain period of time (e.g. one calendar year).


  • In the case where a long-standing player really goes through a complete change, then the ratings will show that over time. BUT, since the old ratings were accurate, they should stand.


    I don't think they should - if I'm deciding whether to play someone what matters is how they are now, not whether or not they were an ass in April 2006.