I’m curious to know what everyone’s strategies are when it comes to recruiting game testers for your projects? Do you just do all the testing yourself? Do you give the game to friends? Family? Do you recruit strangers somehow? If so, how do you get them to commit to play, and to report back standardized, useful data?
Any thoughts, ramblings, etc welcome.
I just get my friends to test them.
As most of my friends are gamer’s anyway, I don’t really need to get them to commit as such.
They will just keep playing until they either complete the game or get bored (hopefully not the latter).
After that they tell me what they thought of that game and what could be improved.
Do you have any concerns that they’ll hold back though? Given that they’re your friends…
Once a year I take my project to a public convention and watch people play it for two days. That’s really useful, because it focuses on polish and interaction rather than just finding bugs - how long do people remain interested? At what points do they stop playing? What things do they particularly like or find frustrating? Are there bits that are hard to grasp?
For the finding bugs type of testing though, automate it as much as possible. Put a button in your test builds that takes a user to a web form which logs their info into a database, and get as much of the info from pre-filled choices (radio buttons or the like) as you can. You could even collect a lot of it, such as device information, automatically so that the user doesn’t have to enter it. It doesn’t need to be anything too fancy, either. There are bug reporting tools out there you can probably deploy/sign up for and use, and there’s stuff like Google forms, or if you can do simple web/database scripting you could roll your own in just a couple of hours.
For that automated bug-collecting type stuff, I’m finding Lumos pretty useful (http://www.uselumos.com/).
For the actual testers though, I’ve never had much success with friends/family. I do have a few friends who are into gaming enough to make their opinions valid, but I’ve generally found that when you ask friends to do testing, it’s rare that they’ll put in enough time to actually give you any conclusive results.
This time around (with Coldfire Keep), I’m considering trying to recruit testers for TestFlight builds through TouchArcade (or similar). It’s concerning though, because when you’re dealing with strangers, there’s really nothing stopping one of them from being a dick and posting screenshots, videos, spoliers, etc online. Of course, you could have them sign an NDA, but that would require a lawyer to actually create the NDA, and a means to enforce it over what could be some pretty great distances. It’s basically something that I would never want to bother getting into - just too much hassle. BUT, strangers certainly make the best testers…
I could also try to recruit you guys - i.e. other Unity developers. In general, this would be a lot safer because other developers understand the responsibilities a little better, but developers also play games differently to “normal people” - i.e. even though I’d consider myself a “gamer”, I don’t think that I look at games with the same type of eye as a non-developer does.
So I’m rambling a bit, but my point is that I think it’s quite hard for a small, indie developer to get their games properly tested before release. It’s hard to know what the best approach is.
Well I think the best way is simply to release your game for free on the app store, put it in say Canada only. Now you are competing with 1000s of other games so discoverability is probably not an issue you could get real life data if you set up some analytics and then delete the game before you release the real game.
A big unity game did it, bladeslinger.
It’s a huge risk though. As soon as you put it out there, a cracked version is going to start doing the rounds, and you’re essentially taking away the choice for people who want a legitimate copy.
I think it’s also harmful to a developer’s reputation anytime you take something down from sale. People assume the worst - like there were big problems with it or something.
All just thoughts though - I don’t have any actual data on any of that.
I dont think so there is not much risk at all, if you have analytics you are going to get real live data, you can actually figure out where people are quitting (because they will have no real reason to keep playing if they dont want to). I dont think it hurt bladeslinger in fact it probably got alot more interested about it.
It’s definitely an interesting thing to think about. I might not have a choice of where and when it goes on sale, so I’ll call that “Plan B”.
In the meantime, I’m keen on the idea of Unity developers testing for each other. It’s something that I’d almost like to see formalized and organized… just by someone that has more time than I do 
Since the last 2 people I sent something and told them specifically the part I wanted to test only to listen to them nitpick about stupid things like the simple test level I built(the terrain doesn’t look very good kind of shit LOL) and not pay much attention to what I asked.
Was only asking them to check how my 3rd person camera was colliding with walls so what the rest looked like didn’t mean anything haha.
So Im not sure how to answer since friends aren’t working out so well for me
Here’s the deal , for a low budget indie game , friends are fine .
BUT if you have a budget over 100k or so you really should pay someone to actually test it at least for a few days .
AND if your going on consoles you NEED to pay an entire team to test your game
I “eat my own food” so to speak. I can’t expect users to use/play something that I wouldn’t use/play for hours on end myself.
I’ve been developing and testing enterprise grade apps for over 20 year, so I know how to test, what to look for and what’s wrong with my apps at many levels. You should push yourself to test your own stuff. It will make you a better developer for all kinds of apps, not just games.
Unless they’re game developers, not for testing. They don’t know what to look for. They can give your feedback about your game, but it’s not the same as testing it.
I’m one of those strangers.
I’ll test stuff because I think a game looks interesting and I just feel like helping the developer. I’ve also been directly asked to test games, but I only test stuff I’m interested in playing. I test for free, because I learn something from the testing process as well. It’s a win-win. I’ve done both solicited and unsolicited testing here in the forums.
Ah, good, a rambling section…
I nit pick at everything. If there are technical issues, those are facts, not opinions, so that is pretty straight forward. The opinions I give need to be taken with a grain of salt and most developers understand that. If a game has issues early into it, testing won’t last long. If there is something wrong with a game, I will try my best to find it.
I absolutely hate when I don’t get line-by-line feedback on what I’ve tested and I’m asked to continue testing. Either you’ve fixed it and I’ll be sure to test that it’s fixed, or you haven’t. It’s that simple. If someone says it’s all fixed and I find out a lot of things are still broken, I’ll be pissed. Standards? I just create long lists. lol
Anyway… I hope some of this has been helpful.
I’ve been in that situation a bunch of times too - testers pointing out things not working where it’s REALLY obvious that I just haven’t finished those bits yet, or in some cases haven’t even started on them. It’s can be kind of amusing… and very frustrating.
If it is just your camera why not post a web player version here?then we can test it.
Camera?
Sorry, I was half asleep when I posted that I read Nubz post and thought that it was yours.
Don’t forget what happened with Fez. Phil Fish hired some intern and the game was broken when he was trying to showcase it. He should have hired professional game testers who were looking for bugs.
I don’t think that’s a good way to look at it. It implies that a low budget indie game shouldn’t aim for quality.
No, they are not fine.
Never ever use Friends and family for testing your game, since they lack obviously of objectivity and don’t have the professional eye, unless they’re gamedevelopers by themselves.
Well at the time I never really thought of that but that is a good idea the next time.