The issue with average

[hatterson]

There's an issue with the new ratings, or at least with the perception of them. The basis of this can be illustrated with the following quote: "The average person thinks he isn't." That is where the problem lies. In today's society, especially American society, people are conditioned to believe that they are special, that they are important because there is a belief that if people aren't told this their feelings will get hurt or they will have poor self esteem or some such nonsense.

So how does this apply to ratings? Well that's simple if we look at the guidelines given by Lack and Co regarding number of stars in a rating.

1 = Bad, 2 = Below Average, 3 = Average, 4 = Above Average, 5 = Excellent

Hrm, that doesn't seem consistent with what people do (I'm including myself in this). People assume that if they play a good game they should be rewarded with a 5 or at the very worst a 4. This just doesn't make sense. In an 10 player game if you pick random players you ideally would have 1 person with a 1, 2 with a 2, 4 with a 3, 2 with a 4, 1 with a 5. That would be the closest fit to a bell curve. If 100 people are rated then theoretically 2 should get a 1, 14 should get a 2, 68 should get a 3, 14 should get a 4, 2 should get a 5.

In reality I get somewhat insulted if I get a 3 and I'm very insulted if I get a 2. I recently get a rating finalized in a game where a guy gave me a 2 and I was kinda upset about it. The more I think about it the more I realize that neither I nor anyone else should really be expecting a 5. Do I log on and take my turns? Yes. Do I try and take them quickly? Yes. Does that make me 'Excellent'? It shouldn't, it should make me roughly average or above average.

So that's my thoughts. Did I really have a reason or a point to make, kinda but not really. I was just bored at work and thought I'd throw you guys some reading material

[owenshooter]

In today's society, especially American society, people are conditioned to believe that they are special, that they are important because there is a belief that if people aren't told this their feelings will get hurt or they will have poor self esteem or some such nonsense.

yeah, that is why kids no longer keep score in games and everyone gets a shiny trophy... you know, kind of like the new shiny medals!!-0

[tzor]

I'm basically thinking a 4 is "good" and a 5 is "great."
A 3 is "not bad but not really good"
A 2 is "bad"
A 1 is "OMG BAD BAD BAD"

[MeDeFe]

I'm basically thinking a 4 is "good" and a 5 is "great."
A 3 is "not bad but not really good"
A 2 is "bad"
A 1 is "OMG BAD BAD BAD"

That's pretty much what the instructions say, isn't it?

[gdeangel]

What it boils down to is that some people want to conform this site to a bell curve for ratings. A few thoughts: 1) The ratings are not for skill, and I've never see any sociologist argue that "politeness" follows a bell curve. 2) bell curves are useful ways to distribute ratings when you are allocating a limited resource - like seats in the freshman class of a prestigous university *ahem* - or welfare money to the poor. Here, we've got an unlimited resource, so it just doesn't make sense.

Under the old system, there was +, - and neutral. How many times did you see someone get a neutral??? Maybe because knowing what is average in this context is pretty much meaningless... you only want to know whether someone is really good or if they suck donkey b***s.

[tzor]

I'm basically thinking a 4 is "good" and a 5 is "great."
A 3 is "not bad but not really good"
A 2 is "bad"
A 1 is "OMG BAD BAD BAD"

That's pretty much what the instructions say, isn't it?

Now quite. The "instructions" say that 3 is average. I'm suggesting it's mediocre. We could, for example, have everyone be super great at what that stat, all superior, all great to play with. The "average" would be then 4 or higher. What we have is the problem of Lake Woebegone where all the children are "above average."

[TheLucas]

It's nice to be an average player. High ranks might avoid you but a ton of ordinary folks will be willing to play in your games.

[hatterson]

What it boils down to is that some people want to conform this site to a bell curve for ratings. A few thoughts: 1) The ratings are not for skill, and I've never see any sociologist argue that "politeness" follows a bell curve. 2) bell curves are useful ways to distribute ratings when you are allocating a limited resource - like seats in the freshman class of a prestigous university *ahem* - or welfare money to the poor. Here, we've got an unlimited resource, so it just doesn't make sense.

When the ratings are defined as

1 = Bad, 2 = Below Average, 3 = Average, 4 = Above Average, 5 = Excellent

it lends itself to a bell curve comparison. If you want to argue that a 3 (defined as average) is insulting to a CC player because we're better than a random sample of people from whom the 'average' would be determined then OK, but that wasn't really the point. The point was about average in general. "The average person thinks he isn't" Lets say you work at a company of 100 employees and you get to annual review time. If 68 people were given a review of 'average' there would be an uproar. People don't like to think of themselves as average, they associate average with bad, they think they're better than average.

Under the old system, there was +, - and neutral. How many times did you see someone get a neutral??? Maybe because knowing what is average in this context is pretty much meaningless... you only want to know whether someone is really good or if they suck donkey b***s.

Under the old system neutral wasn't 'average'. It was similar to ebay's system where positive means 'good enough', neutral means 'kinda bad' and negative means 'horrible'

Edit: To clarify, I wasn't complaining about the new system, I actually like it. I just thought about the average thing and thought I'd write a post about it.

[PepperJack]

As well as everyone believing they are above average, there is an issue with how subjective the rated areas are. Fair play? It is described to encompass everything one would assume it should, but is that how its being interpreted? I'd be willing to bet that there are a fair amount of people who read "fair play" and think "strategy." Attendance? I think everyone should get a 5 if they take their turns in 24 hours (so long as RT hasn't been agreed to) but some people seem to think that anything more than 2 hours means you're an inattentive douche. Attitude? Wow, this is like trying to pogo stick drunk through a minefield. Let's face it, this is the internet, nearly everything can be misconstrued and generally is. This is what happens when people can't hear your inflection and you hate retarded smileys (like I do).

So there, probably a little off topic but possibly contributary.

EDIT: Another thing I forgot. Is it even possible to be "Excellent" for something like fair play? I find I'm using mostly the 1 and 5 stars. These issues seem more black and white to me. And if someone can explain the difference between "Above Average" and "Excellent" in relation to fair play then let me know. I enjoy splitting hairs and pointless semantic debates.

[hatterson]

I enjoy splitting hairs and pointless semantic debates.

Dear diary....Jackpot

EDIT: Another thing I forgot. Is it even possible to be "Excellent" for something like fair play? I find I'm using mostly the 1 and 5 stars. These issues seem more black and white to me. And if someone can explain the difference between "Above Average" and "Excellent" in relation to fair play then let me know.

To me an excellent player in terms of fair play is someone who always makes the 'fairest' move, always keeps their word in treaties, does all negotiation in public, etc. Someone who is Above Average almost always makes the 'fairest' move, keep their word in treaties unless it would lead to them losing, etc. Someone who is Average is that respect sometimes doesn't attack the strongest player if he likes them more than another guy, sometimes backstabs to win the game, maybe has a hint of a secret truce for a round. Someone who is Below Average attacks based on 'boredom' or something, makes occasional errors in treaties, has some issues with private treaties. Someone who is 'Bad' attacks only their friends, always goes back on their word and always has secret treaties.

Now this is in an ideal world. I don't even use these settings. I personally use the 4 if they're good 5 if they're really good 1 if they deadbeat strategy.

[PepperJack]

Now this is in an ideal world. I don't even use these settings. I personally use the 4 if they're good 5 if they're really good 1 if they deadbeat strategy.

Now you've got me thinking that the real issue is not people's disillusion that they are great. Obviously we have both read the new ratings page, as have many others, and interpreted the rules differently. There is no shortage of threads claiming that this or that is flawed/fantastic (j/k - nobody thinks the new system is great based on the threads).

Isn't the real problem that the ratings just aren't descriptive and there is no direction for how they should be used? The page tells us what issues to consider but not how to consider them.

I'm not saying that I want to be spoon fed a product descriptor or have a detailed decision tree to follow to come to the conclusion that pretty much everyone deserves a 3 but the general vagueness absolutely has to be fixed. If everyone was working off of the same definitions instead of their own then an actual discussion about the validity of the new system could occur.

Another issue to consider is the actual level of knowledge of the community. Sure, there was an announcement. Thats great. Did everyone read it. No F'in way. So there's a renegade group of raters out there that might think 1 star is great b/c the Milky Way only has one star and we're pretty much the most bad ass solar system around. I can't imagine the damage these people are doing. Oh the humanity. (hyperbole intended)

[PLAYER57832]

Is it even possible to be "Excellent" for something like fair play? I find I'm using mostly the 1 and 5 stars. These issues seem more black and white to me. And if someone can explain the difference between "Above Average" and "Excellent" in relation to fair play then let me know.

To me an excellent player in terms of fair play is someone who always makes the 'fairest' move, always keeps their word in treaties, does all negotiation in public, etc. Someone who is Above Average almost always makes the 'fairest' move, keep their word in treaties unless it would lead to them losing, etc. Someone who is Average is that respect sometimes doesn't attack the strongest player if he likes them more than another guy, sometimes backstabs to win the game, maybe has a hint of a secret truce for a round. Someone who is Below Average attacks based on 'boredom' or something, makes occasional errors in treaties, has some issues with private treaties. Someone who is 'Bad' attacks only their friends, always goes back on their word and always has secret treaties.

Now this is in an ideal world. I don't even use these settings. I personally use the 4 if they're good 5 if they're really good 1 if they deadbeat strategy.

I would say someone who helps you to understand the game, learn strategy and so forth deserves an "excellent".

BUT, truthfully, I don't like this system at all.

[FabledIntegral]

I'm basically thinking a 4 is "good" and a 5 is "great."
A 3 is "not bad but not really good"
A 2 is "bad"
A 1 is "OMG BAD BAD BAD"

That's pretty much what the instructions say, isn't it?

Now quite. The "instructions" say that 3 is average. I'm suggesting it's mediocre. We could, for example, have everyone be super great at what that stat, all superior, all great to play with. The "average" would be then 4 or higher. What we have is the problem of Lake Woebegone where all the children are "above average."

Incorrect. Average means what is typical, aka

"2. a typical amount, rate, degree, etc.; norm."

Therefore if everyone was super great everything, and were over the top - everyone would still be average then, and it would still be a 3.

Just as you can have in college, a class of complete morons, yet the average grade will be a C+. And then if you have an entire class of brilliant students, that take the same class, the average grade will still be a C+, even if a lot of those brilliant students would have gotten an A if they took it with the morons.

[PepperJack]

Just as you can have in college, a class of complete morons, yet the average grade will be a C+. And then if you have an entire class of brilliant students, that take the same class, the average grade will still be a C+, even if a lot of those brilliant students would have gotten an A if they took it with the morons.

I don't disagree with what you're saying here but the system can't/won't work this way. Granted having enumerated ratings lends itself to the idea that there should be a proper bell curve distribution, however, without a "master rater" (the professor in your example) such distribution has little likelihood of actually occurring.

The more likely scenario (and the reality of the situation) is that everyone thinks they are great and grades their peers with the children's rec league mentality that there are no losers and everyone deserves a participant trophy. Then when someone who grasps the abstract idea of average or another who just uses the new feedback system exactly like the previous system gives that fluffy dream world scratch and sniff sticker loving 'tard a real rating, the shit hits the fan.

The problem is uniformity. These ratings are useless till there is clarity on what actually demands a specific rate. I'm thinking of retracting my previous statement and demanding to be spoon fed a thorough definition because while subjectivity is great for creating threads like this, it is obviously terrible for rating system.

If F'in gymnastics can take something that is subjective and have all their judges come with .002 of each other why can't we? The reason, of course, is no one is grading on the same criteria. Objective definitions please.

[Thezzaruz]

Now quite. The "instructions" say that 3 is average. I'm suggesting it's mediocre. We could, for example, have everyone be super great at what that stat, all superior, all great to play with. The "average" would be then 4 or higher. What we have is the problem of Lake Woebegone where all the children are "above average."

I might not be an english professor but taking "not bad but not really good" as meaning "mediocre" does seem to be quite a bit off from the meaning of those words...

[CubColtPacer]

It's certainly an interesting issue. I feel like there might be too many score points right now making too fine of hairs to split. What's the difference between a 4 and 5 on these issues? Not very much. Most opponents just don't do a lot noteworthy in either gameplay or chat. Should you leave them 3's for being average? Or leave them 5's? Does 3 indicate a problem with the player, or does it just indicate nothing special that the player did? There's a huge difference between the 2, and it's impossible to know under the current system which one the player falls into.

I know Lack does not want the principles of the rating system to change. It has to be easy to use, easy to moderate, and measure behavior instead of gameplay. So any solution will have to involve those 3 things.

To accomplish that first part (easy to use) I would change the rating itself. I'd leave the categories and what things are covered under each category. For the actual rating scale, I'd use a 3 star scale and define each number as this:

3 stars: This person had no/very little disruption to the playing and enjoyment of my game in this area.
2 stars: This person was moderately disruptive to the playing and enjoyment of my game in this area.
1 star: This person was a major disruption to the playing and enjoyment of my game in this area.

It's still up to the player to define what a major disruption is to them, but the lines are much easier to define for each individual. It's pretty easy to see who are your regular players, who are your players who you have a little bit of a problem with but are maybe just a little annoying, and those players who you want to avoid at all costs.

Addendum: I know that they want to steer away from comment boxes, so I want to distance this from the rest of the proposal so the baby isn't necessarily thrown out with the bathwater. I would also put in something called the bonus star. This is something that is not in a specific category, but is an overall award. You give it to people who just went above and beyond to make this game a thoroughly enjoyable experience. When you give one, you are required to put a comment in saying why you feel this player goes above and beyond the standard.
The players would have the ability to delete your comment if they want to. Since it is all positive though, they shouldn't have a reason to do that, nor should there ever have to be a single bit of moderator involvement. This would also allow something for players to try to strive for to get, which simply further encourages better behavior on the site. At the same time, players who just show up and take their turn every day and do nothing special will still have their very high rating, which encourages players to play with them.

I realize that there are lots of proposals flying around the site right now. I just thought I'd add my thoughts on how to make this system a little easier to define while keeping Lack's stated goals intact. I think this would eliminate a lot of the current issues with the rating scale, and at the same time while the definition remains subjective, it becomes a lot more consistent from person to person then the basic scale does now. Most people IMO will classify moderate and severe disruptions in similar ways, and also using disruption also provides people with a much clearer way to figure out what they need to know, and that is to tread carefully against players who may impede their enjoyment in a particular game.

[PepperJack]

For the actual rating scale, I'd use a 3 star scale

This is what I have a tendency to do now for fair play and attendance.

1 = You have taken away from my CC experience. You are crap
3 = You take nothing away nor do you add anything to my CC experience. You are filler.
5 = You have added to my CC experience or have awed me with your performance/effort. You are good.

Even numbers be damned.

For attitude its either 1 or 5. You are a douche or you are not. Most people don't chat enough to even make me entertain the idea of semi-douche baggery (ie, a 3).

To me, these categories ,as currently constructed, don't lend themselves to 5 separate tiers. Hence I emulate, as best I can in this system, the previous system of neg, neutral, pos. If other people are seeing it this way too then it seems to me that the old system might have had something right. Now, I drink.

[Timminz]

The issue with average.....is that people haven't given enough time to see what will be average. Regardless of what anyone thinks SHOULD be average, there is an actual average rating. Over time, this average rating will become more consistent (won't fluctuate as much), and will make prejudging people relatively easy.

[PepperJack]

The issue with average.....is that people haven't given enough time to see what will be average. Regardless of what anyone thinks SHOULD be average, there is an actual average rating. Over time, this average rating will become more consistent (won't fluctuate as much), and will make prejudging people relatively easy.

I can't disagree with more. There should not be a public (for lack of better term) definition of average. What there should be is a high level decision of what average is. So far as I can tell this decision was made, its called 3 stars.

Your opinion is pretty much what I find to be wrong with the new rating system. The ratings should be uniform based on a set of standards (that are currently lacking) yet you state that user should define "average."

"Average" has obviously been set by TPTB. There's a page that says 3 stars is average. Leaving the average in the hands of the unwashed masses is a terrible idea. There is, in my mind, an obvious need for objective defintions on what behaviors merit what ratings. If you want a 5 star average then you should petition for a 9 star system. I am furious right now, if you knew how long this took me to type due to relative drunkness, you would be too.

This system requires work. Its not terrible but its not as close to achieving its intended purpose as the last system.

[hephestes]

The issue with average.....is that people haven't given enough time to see what will be average. Regardless of what anyone thinks SHOULD be average, there is an actual average rating. Over time, this average rating will become more consistent (won't fluctuate as much), and will make prejudging people relatively easy.

Does anyone know what the true average of ratings is so far? A quick scan of the scoreboard suggests it'll be around 4.7-4.8. Seems like a pretty stupid system. A 5 is above average, a 4 is below average. There's really only two ways to go, since there is no option to give 4.75 stars.

And with a huge number of people giving straight 5s, the true average is being way skewed. It's like diving or figure skating at the Olympics, where the lowest score anyone gets is a 9 out of 10.

People need to start giving out more 3s. Most every rating should be a three, people.

[Timminz]

There was a poll in one of the numerous other threads. Someone took the results of the poll, and determined the average of what people give as an average rating to be about 3.8. The ratings system has only been in place for 4 days now. Saying the ratings are flawed, so soon, is similar to getting bad dice in 2 games, and saying there must be something wrong with them.

[cicero]

This thread Amend the guidance on ratings ... over in Bugs & Suggestions also takes issue with the word 'average' ...

[hephestes]

Saying the ratings are flawed, so soon, is similar to getting bad dice in 2 games, and saying there must be something wrong with them.

No, it's like saying I have a brain and can predict where the ratings are going. You don't have to experience everything for an extended amount of time to know that it isn't good.

And if the real average does end up north of 3, when 3 has been set as "average" by the mods, then there is a flaw in the system. The flaw is that people don't like to be average, so they give higher than average marks. I don't need another month of the system to know that.

[Timminz]

And if the real average does end up north of 3, when 3 has been set as "average" by the mods, then there is a flaw in the system. The flaw is that people don't like to be average, so they give higher than average marks. I don't need another month of the system to know that.

The flaw is not with the system, it is with the people using it, and even taking those misconceptions into account, it can still work out into a usable system.

[PLAYER57832]

And this thread is a poll on various options .. to keep the new system or change to a few other ideas.

Can't say what will come of it, if anything, but at least it is a try.
viewtopic.php?f=4&t=53466#p1379074