Ok this is going to sound so newbie…don’t hang on that…I need real answers.
How do you vette a studio?
What do you look for when trying to hire a studio?
What should they be showing you?
What should you absolutely demand to see or move on to the next studio?
What is it that client’s typically get burned on? Studio’s not completing work? Sucking the budget? Absconding with IP?
and not to end on a negative note:
What does a studio need the most to succeed in your project?
I’ve hired software development firms before for large projects and have senior developers capable of determining if the studios are technically skilled…but game development doesn’t seem to have a “typical” expected business exchange.
Some of the “studios” I’ve been talking to don’t want to give references because the projects are NDA’d or that each member is actually doing their own work (though the studio is advertising it as the studio’s portfolio).
Some of these “studios” aren’t incorporated or a have business license but instead are groups of people who share the work load and advertise themselves as a studio. This seems to be more common than not. That the studio is a collection of people all over the world doesn’t mean it’s a bad studio. I’m all for telecommuting.
Some studios don’t have sample contracts…they don’t even use contracts. Just straight invoice.
Thanks to the community in advance, I know this will be an interesting discussion.
I think a simple, straight forward, easy measurable on a studio is see what games they have released.
Go and play some of those releases to determine quality of game design, art and professionalism.
If a studio hasn’t released any games, or can’t show you anything… move on to the next.
I’d be nervous about a distributed studio. AAA games are 90% content and 10% tech (well maybe not quite that, but you get the gist), and the content needs to be strongly directed and coherently stylized. If you have artists, probably the bulk of a game development cost on a very large title, you want them in the same studio with creative directors looking over their shoulder to nudge it all in the right direction so all the assets look and act like they belong together.
The team needs to be agile (Theres no excuse to be blowing clients money on horrifically slow waterfall methodologies in 2012) , it needs to have a history working as a unit, and its creatives need to be creative, its techs need to be fucking wizards, and its managers need to “get” that time is money and slipped deadlines can doom a project.
If you’re talking about file sizes, true. If you’re talking about anything else, false.
Depending on the game I’d say anywhere 60-90% of the time invested in the development is coding/technology related.
And with motion capture suits in the mix now the art/graphics side of things in a lot of games now can be extremely streamlined.
Nah, in my experience, with AAA games you’re looking at probably 80% of the dev time is on content. There’s usually a lot more folk on the art/design side of things than the programming side.
Motion capture doesn’t particularly streamline stuff - the data you get is often noisy and messy - you’ve got to have animators clean up the mocap data and mold it to your game. What it does is give you is more realistic looking animation.
Take a game like BF3. You’re telling me 4 times more development was spent on the content than the development of the Frostbite engine improvements and coding the entire game, multiplayer and everything else code-wise associated with it?
I guess each scenario is on a case by case basis though.
I don’t agree with the fact that coding is 90% or that content is 90%. I have worked on several console titles and I I know of 1 that was mostly content while another was mostly coding. You can’t assume that up front, thats just silly.
I do agree that motion capture data even when using a expensive system over a kinect or cheaper variant give a lot of noise but you can simply cancel that out of you use interpolation. But thats not the real problem of animating with motion capturing it’s that for a good animation you mostly want an exaggeration of reality. Not always but in most cases this is true. Think about cartoony games or games with a lot of action in them. I guess the best place for using motion capturing would be in games like LA noire or heavy rain.
Now to answer the questions of OP:
I don’t but if I would I would use project management tool to see how all tasks are progressing. Check at least on a daily bases and try to schedule tasks with a max execution time of 4 hours or so. This can give a clear image about progression and quality to speed ratio.
That depends what the goal is. If you’re going in for a game driven by a new technology get yourself a team of coding ninja’s. If you’re gonna build a giant RPG get yourself an army of artists. What is the quality your aiming for etc etc.
A released game that shows the skills that you are going demand.
This is yet again a very project dependent question. So you have to be more specific.
Lay down clear contracts. The best way to prevent both client or studio to be burned is to define every agreement before hand. If you don’t have this clearly defined what you expect of each other you are going to be in trouble.
It needs to really believe in the project and it should have some say in the creative process. If the studio just feels like they are factory workers producing content/code they are not at there best. Commitment is created by involvement.
Yeah, ok, if it’s a proprietary engine then the dev time is significantly larger in terms of tech development. If you’re using middleware or updating an existing engine then it’s probably a lot less.
But usually an engine - or at least large chunks of tech - will be reused in many games so I’d probably consider that a parallel development process to the game.
That’s an exception, and in fact, I wouldn’t even consider the frostbite development part of the dev time of BF3, because it will be used for other games, including DICE’s own future games, but also other EA studios. It was a project in and of itself.
Side note: Farfarer is employed by Rockstar, and is working on GTA5, just to put things in perspective
For instance if you look at C&C generals you see in the credits that there are way more testers and voice actors then there are artists or engineers. But if you look at the amount of artists vs designers/engineers you see that it’s almost an equal amount. Portal is also a low content game which probably had more programming and design behind it then artwork. However if you check out games like Skyrim or red dead redemption it’s easy to say that there is a lot more art content then code weighed in hours.
It’s just how you look at it and what your description of triple A is.
When you’re talking about AAA games, of which the vast majority are very heavily narrative based action adventure games then sure, content creation is going to out-strip technical development man-hours by a wide margin. On the flip-size, a multiplayer only title, strategy, board or puzzle game like say Echo Chrome – no so heavy on the art department.
Lets also not forget we’re talking about outsourcing, on the unity forums, I think it’s pretty safe to assume this isn’t for a AAA quality PS3/360 title so splitting hairs over this probably isn’t really going to help the OP much. Someone else bought up the AAA thing, the OP didn’t specify what kind of quality he was looking for exactly or even what kind of game this would be for.
Generally though I wouldn’t work with anyone that can’t show their worth - if you need a studio to make the whole game then make sure they can do everything - code, art, animation, ui, audio, music, service integration, polish ect before you commit to working with them.
More communication about specific information can let you know more about the studio’s creativity,responsibility and execution. These are important for cooperation. Contract is always necessary, the more details and clear definitions it has, the better the work will go in future.
Client will get burned on not completed work, but it often comes with sucking the budget. Be sure both sides have a good communication about the project process, if there is really something wrong happening, ensure both sides know it well in time. Sometimes once client can accept the reason of problems, the delay or more budget will be approved.
Our game has been mostly coding. There are tons of 2d assets and some 3d but this hasn’t been the cost suck. The cost suck has been reaching the limitations of the freelance developer and finding out too late for whatever reason. That’s why the question about exploring studios.
I’m getting questions on what we’re trying to do so I’ll answer that.
*Web based casual game in Unity to be deployed on Facebook and website.
To get to test:
*Using 2d and 3d art. Mono/c#. APIs from server are documented.
*Most assets and interaction design have been created or near feature complete. Placeholders for some screens need to be implemented. To go into test.
[Studio will not be tasked with anything to do with the server. APIs are ready and documentation is being updated. The server stable and near feature complete for going into beta. Of course still rooting out bugs. Server and client are talking to each other. On poll does not seem to be working from client.]
*Grid for game world appears to be working. Characters are pathfinding, Z-order appears to be working.
*Animations need to be polished up.
*Camera needs re-positioning
*Counters need to work, status panels need to work
*Possible shader for plants
*Performance issues (framerate slows to a crawl)
*sound effects for characters (background music is done but not implemented)
While in test work that needs to continue:
*Paypal hook-up
*Facebook i-frame integration
*Social sign in needs to be hooked up
*Avatar maker assets need to be assembled in Unity and hooked up.
Now that I look back at this question I realize what I ment to ask is “What is typical for the game industry?” Since I’ve been in the process of interviewing studios I realized they must be vetted the same way I would any software consulting firm for an application development project. So now I’ve returned to my roots and only will consider studios who will comply with my expectations for a $15M IBM contract (mine should be less than $20k, complete). If they can’t comply, you are dealing with hobbyists. The fact is the game industry needs the Software Development Life Cycle applied to it because YOU ARE BUILDING SOFTWARE and it’s your money!
1. Portfolio and references…if they don’t have it run away. Verify that they did indeed create those items in their portfolio by explaining how they did it, talking to their clients and seeing what others think of the final product. If they use the excuse that the portfolio is under NDA ask to talk to their clients and have their client show you. If they use the excuse that the portfolio belongs to the individual team members and aren’t part of their company…either they show you or you run away. 2. SDLC. Understanding of the Software Development Life Cycle and proper implementation. If you don’t know about it…learn about it before engaging a studio and ask them how they apply it. Most studios won’t know. If they are unwilling to apply it then run away. 3. Requirements and Scope. Make sure you have documented what your game will behave like, look like and the goals the player will have. Make the studio stick to the scope. Do not give them creative control unless you want your project to fail. You can add to the project after you get the base functionality down and they know how to run your project. 4. Code Reviews. I know some of you can’t read code. Regardless, make the studio walk you through the code and explain to you what the modules are at a high level. Have an expert look at the code and approve it for you before making payment. You are not abusing them even though they may think so and writhe in pain. It’s your money…you want a quality product that runs. If they don’t agree to at least weekly code reviews before they receive payment then run away. 5. Code Test. You are paying for a product that must run consistently. You want quality. Do not accept code or give payment before the code passes test. If you don’t know what this is then you need to learn about it. Test early, test often. Quality, quality, quality. 6. Builds. Make sure you get frequent incremental demo builds. Have them put their watermark in it if they whine about being paid. A demo build is not SOURCE CODE. Source Code is the deliverable you are paying for after everything tests out. A demo build will let you play with it and ask for changes or catch performance issues. They should not be charging you for a demo build. It takes a short amount of time for Unity to generate a demo build. If they complain because they have to debug the code for the demo build too bad. They needed to debug for test anyway and should be debugging often. 7. Engagement Contract. Should have a breakdown of what you will receive (deliverable), project plan, release schedule, milestones for payment and warranty. If they won’t do this then they aren’t a software company…they are hobbyists. Run away! 8. NDA. Make sure the owner of the company signs an NDA. Check their corporate status and business license. Verify, verify. If you can’t find anything run away. Ask for resumes of the staff…verify, verify. 9. Use an Escrow service. If they want you to show them the money…put it in escrow. Use an escrow service that allows you to track their progress and approve the deliverables. Odesk is a good place to do this. I’m sure there are others. 10. Team Viewer. Use TeamViewer to check on the progress of your contracted studio. Set regular project check in times and make them do it. If they won’t agree to this and writhe in pain then find another studio. 11. Good Faith Money.DO NOT give them any good faith money until you have agreement to all of the above and are able to sign off on a detailed Engagement Contract. It’s your money, remember that!. You want a quality product in a timely manner. Any studio who wants good faith money without providing the above is a studio of hobbyists. They will have an extremely high probability of failing your project and tell you on the day they’re supposed to deliver it to you. If they started working without the contract that is their choice. DO NOT PAY THEM UNTIL YOU’VE RECEIVED THE CONTRACT. 12. Punks. Yes I said it. Punks. Small teams of extremely talented individuals who are usually distributed, enthusiastic and bid low. They believe a code review or any of the above other contract requirements listed here are abuse. Your risk of losing your money and calendar days is extremely high. Again…run away!
Hmm, most of this seems pretty standard but there’s a few items you haven’t covered.
You mention
as if there’s only one possible life cycle model and its not clear from your post which model you’re expecting. I agree that its important to agree on a model before you start which will then affect everything else.
Similarly you say
Exactly how you document your requirements and scope will depend on your development model and your project. If you’re using a waterfall model you’ll need to document every last element, although you really shouldn’t use a waterfall model for game development. If you’re using a more agile methodology it will be more important to specify the audience and a list of priorities rather than a rigid requirements spec.
I think this is a bit strong. I wouldn’t give a studio complete creative control but do give them leeway to come up with creative solutions to the requirements. You’ll get much better work out of them and probably a better product.
Agreed, but this doesn’t mean you don’t pay them anything until the final product is delivered either. If you’re doing all the checks and control you mention then you should be able to pay them stage payments based on deliverables. Not receiving any payment until the final delivery is a huge risk for the studio and not one than many will want to take.
This pretty much hits the nail on the head. On the one hand you need to control the project and make sure its managed properly, on the other hand your PM requirements can’t be so onerous that no one good wants to work on your project. Never forget that you’re dealing with humans. You expect the studio to focus their resources on producing the best product for you - in the same way you need to focus your PM requirements so that you only ask for what is really needed. Asking for extra procedural stuff just because its easy to paste it into a contract wastes their time and your money.
I very much doubt you’d find many commercial game studios using the Unity forums for example or any game development forum for that matter. Instead you are much more likely to come across a group of mates doing it for a hobby or worse thinking they are a company yet are not incorporated.
Trouble is, at the budget you’re talking about, you’re not really in the league of ‘traditional’ game studio’s (e.g. ones which may have released budget projects on consoles/handhelds), but at best ‘internet’ companies who release simple browser based stuff. These are obviously much harder to vet, to find reliable ones, though they do exist.
For example if you were looking for a traditional game studio, one that would operate similarly to your business experience you really need to start looking in the ‘trade press’. In the UK that would involve getting publications such as MCV and Develop, both published by intentmedia. Develop magazine produces a yearly ’ Best Game studio’s’ book, which would be a god starting point. MCV has the ’ Sourcebook’, although that is more biased towards marketing, legal, QA, distribution services etc.
However as I said for your budget I’d doubt any of the companies in those books would be able to help, unless its a very simple game. They simply have too many overheads to cover which would eat up your budget instead of it going on coding and assets where it matters.
The only other ‘reliable’ places I can think of looking would be www.gamasutra.com who offer a list of contractors, but i’m unsure if any vetting process is in place.
As for the rest of your list, all pretty good stuff (in general) but it goes both ways, e.g. for a game studio or freelancer to vet the potential client.
However I would argue that some are more focused on capabilities only ‘traditional’ game studios could uphold, enforcing expectations to the same level from indie/internet companies is just not going to work, e.g. code reviews, code test and builds. Not that I disagree with any of those, just from a practical view point (dependant upon the complexity of the game) a low budget game, simply doesn’t have the development time to waste doing them too frequently.
Some specific replies to your points, though understand some of these comments are based on the budget you have provided.
I’d disagree with this. Unless you are a proven game designer and assuming they are a proven studio, they are in a far better position to guide the project to a success.
As long as its within the original brief, otherwise expect a change order and additional budget.
Just doesn’t seem practical to the level you are suggestion. By all means get an overview of how it works, but you seem to be looking for best code ever written and that’s just not going to happen. Frequently game programming is about find novel solutions or even hacks to achieve your goals and getting a finished product released on short deadlines. This does not provide an environment for producing ‘expert’ code. In addition as I said above demanding this too frequently will only hurt development progress.
Whilst you should rightly expect a working game and a decent level of QA from the developers, perhaps some unit tests, the level of testing you are asking for is really out of their scope and the budget. That level of QA would really need an external testing company, which would seriously increase the budget.
Instead I’d be ensuring in the contract there are agreements to fix bugs and provide free support for X months after release.
The only problem with frequent builds based on an arbitrary timeline is that it requires the game to be in a playable state for each build, that might not always be practical. There is nothing worse for a developer to be integrating a complex feature, to a ‘build’ deadline. It frequently wastes time and can be distracting.
Far better to agree before hand specific milestones for when builds will be available.
Not really neccesary and I think impractical for the type of studios you could get for the budget you are talking. Obviously you need a very good communication system set up, but this and some of the other suggestions start to require dedicated management team. Nothing wrong in that, except they will be taking a good chunk of your budget instead of going on the code and assets.
In summary, I think the list you give is a great starting point and should be carefully considered, but some points may be more applicable to much bigger budget projects. On the other hand for the budget you mentioned, you options are limited anyway. Unless looking at a using freelancers, or very small independent studio’s (i.e 1-3 friends with an incorporated company), any actual studio is likely to eat a good deal of your budget before you even get to coding or producing assets.
Atalanta: I didn’t say there was a single SDLC model. I said make sure one is applied. Pick the one you and your contractor like…but do it.
Atalanta: Agreed. The client should be able to do most of this from the user standpoint. Execution strategy would be the studio’s responsibility.
Atalanta: I’ve tried it…doesn’t work. Waste of time/$ and nothing but crap. If your studio/freelancer wants to be creative…they need to propose and client accept. It becomes a change order or future requirements. Studio should do it for free or delayed payment if it’s out of scope, wastes time, wastes money.
Atalanta: I didn’t say not pay them until the final product is delivered. I said it should be tied to the contract/milestones and only after the contract/milestone is agreed on and signed off. DO NOT GIVE GOOD FAITH MONEY UNTIL THE CONTRACT IS SIGNED OFF.
Atalanta: A client shouldn’t ask for procedural stuff because it’s easy to paste into a contract. A client needs to lay all this stuff out and be responsible for their share of it. The studio fills in the gaps and applies it. Yes I never forget that I’m dealing with humans…all the more importance to make sure everyone understands what they are doing up front. Everyone wants to be happy.
Well, nothing special, just as I implied, doing some of the same things you’ve listed, but from the developer perspective. It was also to point out that clients can be just as ‘poor/bad’ as you’ve found development studios can be.
So you’d want to know what the client has been involved with in the past, ensure they have the funding/backing, contract milestone payments, including upfront (25-30%). Basically common business sense stuff. Look them up online and see if anyone has left comments about them (good or bad).