So, as anyone who frequents these boards probably knows I’m in the process of tackling a mid sized project. I’m only about 3.5mo into the current iteration, and I’m working on the tactical layer, the ‘in game’ side of the project.
The overall game design is a sort of unusual entry, a 3rd person view turn based, party based game. You basically lead a mercenary band that you recruit, train, etc. Movement out of combat uses a real time system and movement in combat uses a turn based system - both modes use a 3rd person camera and the modal transition is pretty seamless - in fact I’m designing around keeping this transition as seamless as possible.
I’ve recently started to add different ‘classes’ including different attacks and abilities, specifically designed to offer strategic tradeoffs and the like. The problem is that without adequate time to kind of digest the abilities, experiment with them and understand some of the tradeoffs - players won’t really ‘get’ them. The game is, in essence, a strategy game and I can’t just rely on selling abilities on wiz-bang visuals. In actual gameplay, abilities, different classes and roles would be slowly introduced to the player. They’d have time to experiment with these somewhat individually and learn how things fit. But in short, early pre-alpha I’m not in a position to craft that kind of experience properly and full on.
I’m wondering about how people working on larger or mid scale projects handle play testing, at what points in the project they open the door, and what kinds of value they aim to get out of the process. I’d really love to hear from some of the real pro’s like @zombiegorilla - and if bigger games even do ‘out of team’ play tests at these early phases?
If they do, what kind of feedback are they looking for, and what do they expect?
I find that I’ve been building tutorials and the like to help guide my play testers - but its so premature and I don’t want to have to constantly update and iterate on tutorial information when things are still ‘in motion’.
Testing on any game is great. The indie curse is none of us can get enough testers nor do we look for them in general. Best get testing if you can - preferably with a nice way for people to feed back, auto email perhaps?
Yeah, this is part of the problem. It’s a time consuming process to get play testing. I’ve had 2 rounds already, I do another every month or so. I’m considering doing a third round today.
What do you really get out of it? What kinds of feedback do you expect from players? How much effort do you put into building experiences specifically for play testers… a ‘demo’?
It’s kind of fun, you collect some feedback - you get some praise, you look at some criticism, it feels good in general… but is it actually worth putting real time into? Is it valuable as anything other than a ‘sanity check’ at early stages?
I don’t really get many testers, I have a hardcore gamer, me, my brother and a guy at Sony, and that’s it. I’m going to rope in Unity to test it later (I’ll trick them by saying its hardcore Unity 5 or something).
Heh… I’ve been experimenting with a bunch of stuff. Some people I just give them a download link with no context, others see a video so they know the genre, others are friends or people I’ve kept in contact with. A few people on these boards, etc.
I know people say, “play testing is good always and often” and I can definitely see the value in games where the mechanics are a bit on the simpler side (not that my mechanics are rocket science - but it’s a turn based strategy game and some people have never played one before and don’t know how ‘action points’ work, etc).
But beyond ‘conventional wisdom’ – on a larger project – is constant play testing at early phases really worth spending time on, and if so, to what goal? Or is it really just worthwhile as a quick sanity check to make sure things aren’t a train wreck early on.
If you’re at a stage where you have to fill in the blanks constantly (for gamers / testers) and you’re not getting the message across then it’s too early. I’ve done it, doesn’t work well…
You shouldn’t have people coming back to you saying, well I don’t like this on 40 items and you’re explaining well yeah BUT it’s going to do this / that.
But as soon as you have a fully sorted prototype, that’s the time to get it out the door (warts and all) in alpha… Every month we have a testing session after a couple of days away from it (we pass ideas for a couple) then play it again, sometimes it’s like yeah that really works, others it’s like WTF was we thinking?
What kind of feedback should you expect and how do you measure it?
As a gamer, if someone gave me a 2/3 finished XCOM:EU map, with one weapon and one enemy type… would it be fun?
Would it even be fun if overwatch was buggy and failed to trigger once?
I’ve actually played some games during early access releases, and holy crap does the last 6 months of polish make a difference. I’ve felt games were total duds before they really got that last layer of paint. Is it really realistic to think that players should have fun with larger games right off the bat? And if so, how much fun?
Yeah, this is kind of a mixed thing - I’ve been trying to keep it intelligable enough that someone who is reasonably experienced in genre should be able to ‘get it’… hopefully(?)
But … different people need different amounts of blanks filled in. You know?
It’s also very hard to tell what people actually did and experienced - I could see play testing being far more valuable if people all recorded video - so you could see how your expectations differed from their play experience. But random response on the internet… kinda much harder.
I asked a very similar question a few weeks ago, here was a very interesting response:
I think this is pretty sound. his advice here is very different from most you hear: “as close to a release candidate as possible” … “polished level” … “finely polished”
Am I wasting my time trying to do these rounds before reaching “finely polished release candidate”?
What @ and the other person you quoted said are very much my own experience in throwing my little experiments out for feedback.
People say you gotta throw it out there as soon as possible and get that feedback and yet I found that just doesn’t work. As I began to wait longer and longer and put more and more of what I had in my head into the actual prototype… then throw it out it for feedback… it made a huge difference.
Instead of getting feedback on why doesn’t this do this, you should add (something that is obvious and certainly intended to be added but I just hadn’t done it yet) and so forth I got better feedback. I think @ and the other one are spot on and I posted the same kind of thing over in the Game Design Feedback Friday thread in the past. Most people do not seem to be able to see where you are going with something. Instead they can only see literally… what they can see.
It doesn’t matter if you say “this is an early work in progress” in their minds if something isn’t in the game at the time they play test they think you have never considered it and have no plans to do it. Basically you end up getting a lot of feedback that is completely worthless.
And it does make sense. I mean when you really think about it. People don’t know what is in our minds. And let’s face it if we gave the same one page game design to 10 different people around here to develop it is likely we’d see 10 very different games come out of it. Because each person would see things differently. Each person would implement the same things in very different ways.
So yeah… we cannot expect people to know what exists only in our minds.
Yes, it’s a sanity check. But also, things that you think are simple or make sense or are cool won’t necessarily be so in the eyes of players. They don’t see things how you see them, and you don’t know how they’ll see them until you put it in front of them to find out.
So, the goal is to test what the experience is like for your players, because if you don’t find that out from actual players then you’re designing blind. And if you’re designing blind there’s quite a chance that you’re spending implementation time doing things that aren’t actually helping your game.
As an example, my current focus in the prototype I’m building is the initial learning curve for the player, making it as smooth as possible. I have no idea of the impact of a change I make until I put it in front of real players, get them to give the game a spin, and see if and where they have trouble picking stuff up.
I don’t really listen to feedback too early on for exactly the reasons you describe. Instead my early testing is almost all observational.
I’ve often quoted Henry Ford:
As the designer we need to be the ones to solve the problems. Feedback is sometimes useful in this, but it’s not nearly as useful as identifying the problems. And that’s where observational play testing is critical in my opinion. I want to identify the things that my players have trouble with so I can smooth them out, and I don’t want to waste time “fixing” things that my players aren’t having trouble with.
So I do ask for feedback with early prototypes, but what I’m mostly doing is watching the people play the game.
Make sure you put in enough effort to make the game clearly communicate its rulesets and mechanics. Relying on player experimentation to learn can backfire if a player just is frustrated and quits. Imho that kind of “usability” can make or break a game. It is also hard to playtest, because you basically need fresh testers every time and ideally you’d be observing them. Back when I was working on a simple mobile game I used non-gamers as test persons and I was baffled how little they’d experiment. Literally the only interaction possible and needed was to draw paths for spaceships to follow, just like in “Flight Control” on iOS. Some testers just blankly stared at the screen waiting for something to happen. Even though there was a message explaining what they need to do just seconds earlier. Animated pictures would have been the most foolproof thing I could think of to solve this specific problem, but I abandoned the project before it got there. PC strategy gamers sure will be willing to put in some more thinking and reading. I think at the very least mechanics and important differences in units or equipment should be explained with text, and as well integrated into the game’s UI as possible. You probably already did that, I’m just mentioning it just in case.
Imho this war of mine did an excellent job of communicating its mechanics very clearly in a way that fits the theme and doesn’t break immersion. “I wish I had a shovel…”
Even if the instruction is still on the screen it can be missed. That sounds silly, but we need to remember that our players aren’t looking at the whole screen all at once. If their attention is in one place and we put an important message somewhere else it can still get missed.
While it’s not an approach I want to take, I now understand why some games have forced tutorial sections that force you through every action and ram every instruction down your throat
Good point! I can’t remember at which time of development I made that observation but later at least I had those messages as modal popups that stopped gameplay and needed to be confirmed to go away.
Depending on the type of game, and the problems that can arise from players missing aspects of what they need to do, I think “non-consensual game tutorials” are the lesser of two evils.
I don’t mind them the first time through. It’s on repeat plays where they’re a pain. I played a game just last night where there was about 15 minutes of forced intro/tutorial.
I have no experience with larger projects. But with my smaller projects I’ve found you can only get a first impression once. If someone has learned the ins and outs of the game through extensive play testing, they might just ‘get used to it’ and stop noting the flaws.
So if you only have a limited pool of testers, its might be worth holding a few back till the late stages of development.
At this point, my abilities are pretty managable in a play test - I think I have something like 6 or so, spread across 3 characters. 4 of these are vanilla active attacks, so they’re pretty straight forward, one is a debuff (which is always harder to communicate) and one is a passive ‘stance’. There are 3 weapon types: an archer, dual wield and sword/shield.
Is it realistic to expect a player to figure out how to use this stuff in a 10 minute session while also learning the game’s basic mechanics (how action points/movement points work, ranges, etc)?
There are other more ‘advanced’ strategies that are very unusual for the genre, you can push enemies around in your turn by just walking into them for instance. Right now, most players would probably assume this was a bug (since I don’t have an animation set for it yet).
Is there any real point in including more than this number of abilities in a 15 minute play test level? Will having a list of 12 abilities at their disposal just overwhelm and confuse people since it’s being introduced all at once instead of slowly leveled into one at a time?
For the genre of game, I’m essentially building a somewhat contrived demo environment. I can’t help but wonder if my efforts have started to invest too much in making sure that the contrived experience of early play testers is adequate before a full vertical slice is in place.
As someone who’s tested it, no it isn’t… When I play games I try to put gamer hat on and I’m probably one of the worst / most ideal testers ever… Simply put because I get bored very easily if the game isn’t engaging.
I’ve played tactical based / turn based RPG’s, mechanics felt very much like a toned down / clunky version of dragon age (the first one). I got used to it quickly, besides from the end turn / end combat. In which a scroll to player via MMB is a simpler way to select the next player.
One of the main things, it really didn’t have much “context”… I know you’re not a fan of story but one of the main things was… Why should I care? There’s a metric ton of turn based games out there (collected over the years), it outlines what I have to do at the beginning, go defeat someone… Although, I’m not sure why it actually really matters? I’m not sure if I was paying attention and / or just didn’t really get the point? It was cool testing out a new game, but the novelty wears off pretty quickly.
When you look at games like morderheim, they have a lot of cool stuff in the combat system and if you’re going to rely on a mechanic it has to be amazing… All I really did was click on an NPC until they were no more, I didn’t weigh up what wep’s I had statistically, I didn’t look into how to approach a challenge. There didn’t appear to be anything intricate I needed to think about to get to my goal.
I could be wrong, there could be much more to it. Unless you weave that in somewhere adding subtle direction I’d never know / care.
At the moment, it appears like a template as opposed to a game… Although you’ve only been at it three months, in another three it will be completely different and in another three same again. So hold off for now…
I think this is really the primary challenge when you start dealing with some types of more complex games. Especially in early play testing. My thought on this is that most of the time, in order for your average gamer to really start to enjoy a game - you need to give them enough context and enough to ‘care’ about to really make it work.
Let’s say that my project was a diablo clone. At this point in the project, I would basically have a half built area, a handful of abilities and the core control scheme - maybe a skeleton of the loot system with 2 or 3 items that get random values.
The core game play loop in diablo is the overall progression system and loot grind. The core play loop in my game is the team progression, character specialization and loot grinding.
The parts that I have worked out are the core mechanical systems, but the core game play loop isn’t there - so - why should a player care? Is it realistic to think that the player will have a full, satisfying experience before the ‘progression layer’ is in? Does that mean that I can’t get really worthwhile player feedback for another 3-6 or so months?
Most of us are familiar with Diablo, and i think it’s a nice example - what needs to be in place for a good play test of a Diablo clone? Is diablo playable before you have a full vertical slice?
What if your diablo clone only had 1 equipment slot and it took another month or so before you’d be able to have 3 or 4 slots. Would diablo work with only one equipment slot? Is it worth spending time and effort collecting feedback at that point?
You need to get fresh people in to know if the games’ fun, is it too easy/hard. Are people actually going to even play into the later stages of the game or are they just quitting after the first few levels.