Quality control for new maps

[gdeangel]

I love the diversity of maps at cc. I never play classic if its avoidable. I have to say, though, that the last two map releases I have largely sat out. For every well constructed map like New World, there seem to be three niche maps that only really work with certain play combinations, dominant strategies, and/or heavily weighted to luck of the dice in the opening rounds. It seems to me that the foundry is broken. I have attempted to get a couple of map ideas off the ground, and also to post some feedback on maps there in the past. It is entirely unclear that there is any effective quality control going on there. For example, 1v1 gameplay seems to be overlooked by most map makers (i.e., avoiding starting bonuses and such). Instead, it seems to be a race to churn out more maps by individual map makers, and the idea of giving out map-maker medals based on merely having a map quenched with no QA doesn't help matters.

So I suggest that one way to address this is add a final testing phase with core cross-map regulars to try them and provide official comments that the mapmaker MUST address before a map can be quenched. A second suggestion is that to earn a mapmaker medal, rather than just having them be automatic for quenching the map, it should be based on the map earning a minimum level of new games per month IN THE SECOND MONTH OF RELEASE. Based on the popularity number, mapmakers would get gold, silver or bronze, and could get a medal for each map only if each map on its own passed the popularity test.

Thoughts?

[Qwert]

these is for map foundry forum

[hulmey]

I love the diversity of maps at cc. I never play classic if its avoidable. I have to say, though, that the last two map releases I have largely sat out. For every well constructed map like New World, there seem to be three niche maps that only really work with certain play combinations, dominant strategies, and/or heavily weighted to luck of the dice in the opening rounds. It seems to me that the foundry is broken. I have attempted to get a couple of map ideas off the ground, and also to post some feedback on maps there in the past. It is entirely unclear that there is any effective quality control going on there. For example, 1v1 gameplay seems to be overlooked by most map makers (i.e., avoiding starting bonuses and such). Instead, it seems to be a race to churn out more maps by individual map makers, and the idea of giving out map-maker medals based on merely having a map quenched with no QA doesn't help matters.

So I suggest that one way to address this is add a final testing phase with core cross-map regulars to try them and provide official comments that the mapmaker MUST address before a map can be quenched. A second suggestion is that to earn a mapmaker medal, rather than just having them be automatic for quenching the map, it should be based on the map earning a minimum level of new games per month IN THE SECOND MONTH OF RELEASE. Based on the popularity number, mapmakers would get gold, silver or bronze, and could get a medal for each map only if each map on its own passed the popularity test.

Thoughts?

There is a neeed for play testing maps but lackattack doesnt want to do it for some reason

[Kemmler]

too many maps that only work with freestyle. smaller 1v1 maps are based too much on the drop, and if you make larger ones freestyle then first player wins

[Thezzaruz]

Thoughts?

I fully agree. A limited play testing stage is sorely missed IMO and the latest debacle (Galapagos) with a map being changed over night (without much discussion or though at all) messing with existing running games just can't be allowed to happen again.

[Kemmler]

Thoughts?

I fully agree. A limited play testing stage is sorely missed IMO and the latest debacle (Galapagos) with a map being changed over night (without much discussion or though at all) messing with existing running games just can't be allowed to happen again.

it's still a broken map even with changes. way too luck oreintated

[cairnswk]

....
Thoughts?

I agree with most of your thoughts on this, but why aren't you saying this in the Foundry where maps get made.

If there is any lack of quality control from the foundry it's probably because no-one comes into the foundry enough to make the comments that are needed to ensure checks and balances are addressed.

Solution: gather up your friends and head on over there, and get some suggestions/comments happening.

[gimil]

Well done to cairnswk Spot on as always.

Now my turn!

The map foundry is a communiy run place where people make maps that they want. We at the foundry try our best to bring the best quality out of all maps that come throuh our forum. Unfortunatly from time to time a sub par map gets through myself and oaktown because we are simply human. Me and oaktown don't get paid for what we do and we don't have limitless amounts of time to decided to quaility control on anything and everything that goes through the foundry. Thats where we have the rest of the CC community come in to post there comments and concerns to add to the quality control. However the current foundry community isn't as busy as it once was and that means that there is a bigger starin to get the same quality we always produce.

To make a long story short, if something is broken, we take faith that community will come along and nicely let the foundry community know what the problem is and hopefully they can paricipate in the fixing process.

The foundry isn't a them and us place. Its a place for the CC community to come together to participate and help in the map making process. We are not perfect, but if you come along in help we come that bit closer to perfection

[Herakilla]

ya gimil and cairns summed it up

there is a quality control and guess what? its YOU


Only YOU can help us make maps

[edbeard]

Instead, it seems to be a race to churn out more maps by individual map makers, and the idea of giving out map-maker medals based on merely having a map quenched with no QA doesn't help matters.

yes. this is what we talk about all the time. it's all about who has the most maps and medals. these are our goals.

not every map is going to work for every person. it doesn't mean there is a problem with the process. right now a lot of maps coming out are more niche based but the majority of the maps in the main foundry are classic style maps. this means you'll be getting a load of these maps in the somewhat near future. people will then complain that too many classic style maps are coming out.

So I suggest that one way to address this is add a final testing phase with core cross-map regulars to try them and provide official comments that the mapmaker MUST address before a map can be quenched. A second suggestion is that to earn a mapmaker medal, rather than just having them be automatic for quenching the map, it should be based on the map earning a minimum level of new games per month IN THE SECOND MONTH OF RELEASE. Based on the popularity number, mapmakers would get gold, silver or bronze, and could get a medal for each map only if each map on its own passed the popularity test.

Thoughts?

no. people put months into making a map. they don't get quenched on accident. just because a map isn't popular doesn't mean it is bad. niche maps don't hurt you. you don't have to play them. people make maps they want to play. if you want maps suited to your desires, then get involved in the process and give feedback (this won't really satisfy you though) or make a map.


right now there is a testing phase. it's the NEW tag. unfortunately there isn't any disclaimer that NEW maps should be played at your own risk since there could be errors or poor gameplay. would I prefer a testing system? sure. should things these people suggest be mandatory? of course not. just because you don't agree with how a map plays doesn't mean it's bad. also, putting the map out to the public lets you see how it plays in all game types and sizes. like cairnswk said, there's already a quality assurance process. it's the foundry process. it's not perfect but it does prevent loads of awful maps from getting out there. more people are always welcome.

[blakebowling]

Well done to cairnswk Spot on as always.

Now my turn!

The map foundry is a communiy run place where people make maps that they want. We at the foundry try our best to bring the best quality out of all maps that come throuh our forum. Unfortunatly from time to time a sub par map gets through myself and oaktown because we are simply human. Me and oaktown don't get paid for what we do and we don't have limitless amounts of time to decided to quaility control on anything and everything that goes through the foundry. Thats where we have the rest of the CC community come in to post there comments and concerns to add to the quality control. However the current foundry community isn't as busy as it once was and that means that there is a bigger starin to get the same quality we always produce.

To make a long story short, if something is broken, we take faith that community will come along and nicely let the foundry community know what the problem is and hopefully they can paricipate in the fixing process.

The foundry isn't a them and us place. Its a place for the CC community to come together to participate and help in the map making process. We are not perfect, but if you come along in help we come that bit closer to perfection

gimil,

but I would like to see the "Beta Maps" Feature (which has been ignored in the past, but more people seem to like it now)

[Qwert]

Maybe next time-lack need to anounce 2-3 days before upload map with new changes.
I dont see any fault from Cairnswk,hes work on map and if map need to change to fix some thing,he doing that,and after that Lack take map and upload on site.

[cairnswk]

....
but I would like to see the "Beta Maps" Feature (which has been ignored in the past, but more people seem to like it now)

we'd all like that feature blakebowling, but until it happens, your best way to esnure quality control is simply to attend and comment more on maps. that way there can be many thoughts on the gameplay processes being put into the mix.

[gdeangel]

....
Thoughts?

I agree with most of your thoughts on this, but why aren't you saying this in the Foundry where maps get made.

If there is any lack of quality control from the foundry it's probably because no-one comes into the foundry enough to make the comments that are needed to ensure checks and balances are addressed.

Solution: gather up your friends and head on over there, and get some suggestions/comments happening.

As you see in my post, I've gone to the foundry a couple of times to make suggestions. It is like a festering bunch of photo shopping backslappers, usually dismissive of ideas and more worried about making the shade of green a little more teal and such. Since the issues I raise above are for the whole cc community, I think the post belongs in GD, not the foundry, but it is interesting that once noticed, we have back to back's from the foundry regulars all parading in here to tell me I don't know what I'm talking about... just like is the case in the foundry. Why the hostility gents? Aren't you interested in how your product ends up on the user side? If I don't know what I'm talking about, then I'm sure the people who play on the maps should be the ones telling me that they are satisfied with the new maps coming out.

I've seen how a few maps get quenched. The foundry community will throw out some formulas for numbers of territories, maybe a couple of astute people will post about boundaries, starting positions and real gameplay issues, but its hit of miss as to whether a map maker takes those suggestions into account, or blows them off with a dismissive post. Yes it takes months. No that doesn't translate into quality gameplay if the months are spent debating whether unit circles should go 1 or 2 pixels further to the left. Nowhere does someone actually review/simulate the map in an exhaustive set of test cases (how will it perform in 1v1? 8 player freestyle? no cards? etc.) and make the refined adjustments that assure excellent product.

I'll be frank... the way the foundary churns out maps has devolving into creative lab that's great for an academic design lab. There's a reason you don't see the stuff that gets build in design school, even award winning stuff, brought directly to market. I'm not knocking the quality of map design generally. But in reality, to maintain site quality, there should be some intermediate steps between the design lab you call foundry and the users here that make up the rank and file. Yes, you need to strike a balance between creative innovation and quality standards, but "design by democracy/peer review" is only the first step - you seem to be a little to dismissive of the end-user review. Too bad.

[Qwert]

Why the hostility gents?

Hostile? ME?

[foregone]

....
Thoughts?

I agree with most of your thoughts on this, but why aren't you saying this in the Foundry where maps get made.

If there is any lack of quality control from the foundry it's probably because no-one comes into the foundry enough to make the comments that are needed to ensure checks and balances are addressed.

Solution: gather up your friends and head on over there, and get some suggestions/comments happening.

As you see in my post, I've gone to the foundry a couple of times to make suggestions. It is like a festering bunch of photo shopping backslappers, usually dismissive of ideas and more worried about making the shade of green a little more teal and such. Since the issues I raise above are for the whole cc community, I think the post belongs in GD, not the foundry, but it is interesting that once noticed, we have back to back's from the foundry regulars all parading in here to tell me I don't know what I'm talking about... just like is the case in the foundry. Why the hostility gents? Aren't you interested in how your product ends up on the user side? If I don't know what I'm talking about, then I'm sure the people who play on the maps should be the ones telling me that they are satisfied with the new maps coming out.

I fail to see the hostility from the "festering bunch of photo shopping backslappers" that have posted here. Very few mapmakers ever just blow off commentary from what I've noticed and in fact welcome it. If you've got ideas on gameplay I think most would be quite happy to try an accomodate your thoughts and take it into consideration. I, for one, would appreciate if you made gameplay commentaries on my map.

[cairnswk]

... Why the hostility gents? Aren't you interested in how your product ends up on the user side? If I don't know what I'm talking about, then I'm sure the people who play on the maps should be the ones telling me that they are satisfied with the new maps coming out.
...

i don't believe were being hostile dgeangel, but we are interested to hear your views on our maps.
And to make this point quite blunt, i've just taken the time to troll through your feedback topics.
you have 780 feedback posts, but only approx 10 of these are in the Foundry on actual map topics.
Perhaps you'd consider lifting that run rate now that you've raised the topic about quality.

[hulmey]

... Why the hostility gents? Aren't you interested in how your product ends up on the user side? If I don't know what I'm talking about, then I'm sure the people who play on the maps should be the ones telling me that they are satisfied with the new maps coming out.
...

i don't believe were being hostile dgeangel, but we are interested to hear your views on our maps.
And to make this point quite blunt, i've just taken the time to troll through your feedback topics.
you have 780 feedback posts, but only approx 10 of these are in the Foundry on actual map topics.
Perhaps you'd consider lifting that run rate now that you've raised the topic about quality.

[BaldAdonis]

I'm with gde on this. I've tried commenting on maps about gameplay, and things that will critically harm maps, but it all gets drowned out by requests to move a circle up 2 pixels, or change the colour of the bonus legend. It doesn't help that foundry regulars don't play many games or styles, so they don't notice glaring problems. The foundry is not an inviting place for people who want to change the gameplay; instead it's suited to people who want to change the aesthetics.

From viewtopic.php?t=58319

I've always wondered where mapmakers get their game play information from. Some new maps come out and play brilliantly on all kinds of settings (like Rail Europe), while others (like New World, Das Schloss, Battle For Iraq) are clearly designed with one style in mind (4-8 players, 2 player/team, flat rate/no cards, resp.) and end up atrocious in other styles (lopsided first player victories, stalemates, too many dead ends). When I look through map discussion threads, it's good that you're trying to cover game play early, but most of the changes suggested are bonuses or aesthetic. Then the players who discuss things don't seem to play a lot of games (I suppose they're busy making maps ), so it would probably help a lot to call in some people who play a variety of styles to give ideas of how maps will play out. At the very least, you'd avoid another Das Schloss recall.

[jiminski]

The Classic Revamp troubles with the circles, dragged me into the Foundry. I had always just left the forum to to brew in its own creative juices; it had not poked me up to then. In fact, prior to that time it had only added to my experience; playing the good maps and leaving the ones which were not to my taste. I never considered that the foundry should not make Age of Realms for example, even though i did not wish to play it. Many people find the more inventive gameplay maps more fun! Perhaps i'll come around and give them a go one day!

Anyway, in the first instance I went there with much the same feelings as angel.
(i was rather heated about the little white circles )
But if i am fair, in general the foundry works as well as the individuals who bother to participate. There are cliques of course but if you make a valid argument, as in any arena of discussion, people listen and often implement with great grace!

Garnishing games with various exuberant kinds of attacking options etc needs to come prior to a map makers decision to make a map though i think. If a cartographer wishes to make a relatively straightforward map it is perhaps unfair to implement a four-headed, killer, flying otter which appears and randomly attacks all armies within a 4 territory radius, using its radioactive incisors!
The map makers deserve some say in the fundamentals.

However, as the cartographers are a relatively hidden group, gaining access to them to impart 'great' new map ideas is not so easy.

Map ideas from your passing idiot (like me) get lost in the mire .. a bit like in the Suggestions Forum.

Perhaps there could be a specific arena where gameplay could be discussed generically.. even with test groups using a simple Dummy map to examine and develop any new ideas!

That way cartographers could better filter ideas from a broader base and focus on more inventive gameplay prior to their next project choice; after which they have generally fixed the concept and complexity of play.

[Incandenza]

I'm a regular poster in the foundry, tho I'm not a mapmaker per se. As such, I'm familiar with the process and a lot of the mapmakers, and a lot of what I've heard in this thread is simple lunacy.

1. there are dozens if not hundreds of permutations to the game settings, so to insist that every map work equally well for a 1v1 freestyle between noobs and an 8p escalating between colonels is insane, assuming that's even possible. For example, you know what map is absolute rubbish for 1v1 sequential? Classic. Doesn't mean it should be pulled or reworked.

2. the foundry is not an unfriendly place, but if you come in and say nothing more than "this map sucks", you'll be ignored. If you see problems, and can advance potential solutions, I guarantee people will listen. And you have to be persistent, don't just post and come back a month later and feel outraged that your plan hasn't been immediately implemented. Look at different maps, get to know the regulars, keep posting!

3. the only pre-quench game testing system that currently exists is for a mapmaker to literally print out the map, invite a few friends over, and play it. That's it. Sure, a testing site would be marvelous, especially for complicated maps, but as that's not a system currently available to us, all we can do is try and ensure that maps are as balanced as possible.

4. too many people (I'm looking at you, gdeangel) play a bare handful of games on a map then cry to the heavens that it's unbalanced. These are people that are much more comfortable being part of the problem instead of part of the solution. It takes quite a few games to really get a handle on a map. Patience must be an encouraged virtue.

5. The idea that mapmakers should have to drag people into their threads to comment is ludicrous. Mapmakers are not allowed to advertise in GD, tho many if not all put an image or link in their sigs. The foundry is open to all.

6. Just because a map isn't wildly popular, doesn't mean it's not a good and worthy map. Sure, I'm not about to jump into a Valley of the Kings game (sorry, cairns), but I would actively agitate against someone trying to have it removed.

7. Details matter. Colors, army centering, pixellated borders, these are the fine details that need to be addressed at some point in a map's development. To dismiss these essential elements would be like criticizing a Van Gogh because it has too many brushstrokes.

[Pedronicus]

I have just returned from the foundry and i left my thoughts.

[gdeangel]

I'm a regular poster in the foundry, tho I'm not a mapmaker per se. As such, I'm familiar with the process and a lot of the mapmakers, and a lot of what I've heard in this thread is simple lunacy.

1. there are dozens if not hundreds of permutations to the game settings, so to insist that every map work equally well for a 1v1 freestyle between noobs and an 8p escalating between colonels is insane, assuming that's even possible. For example, you know what map is absolute rubbish for 1v1 sequential? Classic. Doesn't mean it should be pulled or reworked.

At least cover the basics, if possible. That means give some thought to 1v1 where neutrals go and how many to start with. That also means looking at team vs. sequential, and cards vs. no cards with 3, 4, and 8. Those give the best view of distinct balances IMHO. But rather than constructively come up with a QA checklist of the best things to test, you can keep on whining that it's impossible to do a constructive testing if you prefer. If that was the case, we'd never have working software on the shelf.

2. the foundry is not an unfriendly place, but if you come in and say nothing more than "this map sucks", you'll be ignored. If you see problems, and can advance potential solutions, I guarantee people will listen. And you have to be persistent, don't just post and come back a month later and feel outraged that your plan hasn't been immediately implemented. Look at different maps, get to know the regulars, keep posting!

Sounds kind of lame. So we just keep talking around in circles for months and hope the idea gets picked up. I guess that's mapmaking by democracy...

3. the only pre-quench game testing system that currently exists is for a mapmaker to literally print out the map, invite a few friends over, and play it. That's it. Sure, a testing site would be marvelous, especially for complicated maps, but as that's not a system currently available to us, all we can do is try and ensure that maps are as balanced as possible.

Well, playing on a simulated hard board game is not a bad idea, and not too out of the question given the number of hours the mapmakers are spending on these projects. Also, they really need a development environment for this site - as evidenced by the constant crashes on every roll out. If they need an excuse to set one up, beta testing of maps would be a good one.

4. too many people (I'm looking at you, gdeangel) play a bare handful of games on a map then cry to the heavens that it's unbalanced. These are people that are much more comfortable being part of the problem instead of part of the solution. It takes quite a few games to really get a handle on a map. Patience must be an encouraged virtue.

Some of us are quicker than others in assessing the strategic possibilities of a map. As I said about Arms Race, it's not a bad freestyle map, or a team map, but it was utterly squandered with poor startup settings for 1v1 play, and I can tell that from the 2-3 1v1 sequential games I've played on it, and there is just no need to waste more time on it in a 1v1 seq setting.

5. The idea that mapmakers should have to drag people into their threads to comment is ludicrous. Mapmakers are not allowed to advertise in GD, tho many if not all put an image or link in their sigs. The foundry is open to all.

Face it, the reason the maps are in the sigs is bragging rights. That's well deserved. Many maps are great, and I generally have respect for mapmakers, many of whom are the same who gave us really great maps. It's just that lately the mapmaking process is selling short their efforts. There should be better QA. And nobody said anything about dragging in the masses to comment in foundry. That would probably lead to just more nonsensical "design by democracy" taking even longer. A simulation is needed, or at least a set of "feature tests" that verifies the playability with specific settings.

6. Just because a map isn't wildly popular, doesn't mean it's not a good and worthy map. Sure, I'm not about to jump into a Valley of the Kings game (sorry, cairns), but I would actively agitate against someone trying to have it removed.

I agree. But we seem to be in a quantity rather than quality mode ATM, as well as a race to build in novel features to be design ground breakers without really thinking about how a map will play. I am not a fan of conquer man, but I think the map is and example of design genius regarding gameplay. Even though its not popular, that doesn't mean it should be pulled. But I don't know that the map wouldn't have gotten made if there was no medal for it (actually, it was quenched before there even were medals.) The point of my suggestion about medals is just to quell some of the possible incentives that have people making maps just to get them out there even though they might detract from the site experience.

7. Details matter. Colors, army centering, pixellated borders, these are the fine details that need to be addressed at some point in a map's development. To dismiss these essential elements would be like criticizing a Van Gogh because it has too many brushstrokes.

You can have the ideal coloration, perfectly rendered and contrasted fonts, and the most innovative and aesthetic visual design, but if the game play is lopsided, that's just not what the site is about. Showboating of photoshop is fine with me... it's hard to think of a map that is really atrocious... but that's only half of the story, yet it seems, on my casual forays into the foundry, to be what 90% of the attention is given to. Don't decrease the attention to design... but match that amount of attention when addressing the gameplay. I know that's a problem because of no simulation, but even just thinking about it and thinkign about what works and what doesn't work in similar maps that are available to play is a starting point.

[hulmey]

I have to agree with OP that not enough attention is given to game play. Oaktown comes in and gives his idea's and if they are implemented then he stamps it! im quite sure oaktown isnt a "gameplay expert" and that more people are needed in this area!

However, i imagine it is more than extremely diffcult , nah, near on impossible to construct a map that will excel in 1vs1, doubles , triples, quads, single flat and escalating!! Then you also have to add the freestyle or sequential element as well. My GOD, how is it possible to make a map to serve ALL these different types of game play.

Maybe, the ADMINS, should start categoring maps on the game finder page, according to how they play???

[tenio]

lack or other "higher ups" have stated before that having a testing area is much harder then it seems


points are an issue

how the game is counted

just having to make the game get out of points

need for a separate server or server area to host the games

and more