For my current project I’ve got an anthropomorphic dog character that I want to animate for use in Unity. I’m looking for some advice on the best pipeline to do this. I last used mechanim about 4 years ago and there’s now a baffling number of animation tools around with overlapping features. I want to spend some time learning a good solid character animation workflow and thought I’d ask for advice on what other people would use to do this.
The character is a detailed NPC. Its quite expression-heavy and will require the mixing of bones and blendshapes to get the right results. My usual approach would be to create a model and rig it in 3ds max, add the blendshapes in there and then export to Unity.
I’m less sure about where to go from there.
What would be the best way to create the actual bone/blendshape facial animation sequences (some lip-syncing, some emotions) and layer them on top of other animations for walking/sitting etc?
Any suggestions of tools and tutorials to do this inside or outside unity would be appreciated.
Blendshape modifications happen independently of animations. Blendshapes is simple vertex manipulation while animations control the bones.
If your rig has facial bones then obviously you can use the standard animator system in Unity, if it uses blendshapes you’ll need some scripting to alter the blendshape values over time. I use LipSync Pro from the asset store which is quite good and automatically analyses sound files to provide the animation.
Thanks for the reply. LipSync Pro seems like it can queue up blendshapes and bone-based anims on the same timeline, which looks like what I want. Can this not just be done in the Unity animator though? Does LipSync provide other functionality?
To be more specific about this project, there isn’t a lot of animation to be done (less than 3 mins of speaking most likely) but what there is has to be highly expressive. Can you suggest any good tutorials on creating a facial rig for unity in 3ds Max? Is it possible to use the biped and add extra bones for lips etc, or is it better to build a custom rig?
I don’t use 3ds Max but it is always possible to add more bones if required using any 3D package.
LipSync Pro does expressions to which you can assign with blendshapes so it might be the thing you’r looking for. In some ways using blendshapes is better since it won’t interfere with any existing animations.
Why would bones in the face interfere with the rest of the skeleton? Surely I could still set up the character with mecanim and have my (bone-based) facial animations in a different layer? The advantage would be that I could then animate talking/expressions on a timeline in 3ds max and then export to unity.
This depends largely on the animation requirement. Are you locked into using motion capture animations? If so - your choice is pretty much set for you - you will need to use the humanoid rig.
Setting up ‘extra’ bones with the humanoid rig is - less friendly than using the generic set up - but since you are only dealing with one specific character - you can focus your efforts on that one character, rather than attempting to create a large supporting system for multiple characters.
Although ‘extra’ bones with humanoid is a little more involved - as long as you are setting aside R&D time I don’t think this is beyond anyones ability - as long as they are able to read. Plus humanoid provides several in-engine benefits - mirroring, built in IK, retargeting, among others.
But -
Depending how complex the rig is for the the anthropomorphic dog (how much is strays from the default humanoid rig) you might consider the generic route - that is less complicated, as long as you are not constrained to mocap animations.
Lip Sync Pro is a solid suggestion. Will require research on if it meets the detail needs you have.
TonyLi has detailed using this plug-in in past posts. Might look into his thread history to see how he takes advantage of it.
To be honest - I think I would do either morph (blend shapes) OR facial bones - instead of both in the same character. Just my personal workflow - I dislike having expressions driven by bones and splitting off the phonemes to be driven by morphs. This is only my personal preference - others may like that segmented set up.
This will require research and decision on the eventual system you choose to use.
On the actual animation process dealing with body animations as well as lip syncing and facial expressions I like to animate in passes.
All passes are animated - to - the soundtrack/audio.
First pass is body blocking followed by a refinement pass.
Third pass is blocking lip syncing (extremes) followed by a refinement pass to push or pull back on the mouth shapes.
5th pass is refining the body animation - polishing, and the 6th pass it blocking the expressions.
7th pass is polish on the lip sync and final pass on the expressions. This also includes blinks and any extra eye movement needed that has not already been created.
Regardless of the software - this is the workflow I use. I have not performed this workflow in Unity - yet- but it is soon approaching and I will be working on revising my own workflow to fit my requirements.
If you use the generic rig set up - any facial rig that does not use expression controllers or manipulators in Max will work. Basic old school rigging - with bones and look at controllers and orientation constraints.
Biped with extra bones is an acceptable set up, but the pelvis and clavicles default parenting has to be changed so it works in Unity correctly. I would also consider Biped or CAT before creating a completely custom rig, simply because of the benefits and tools both rigs offer over custom bones.
One reason is you will be controlling the body and the facial animations in one animation controller - and that could get very complex/messy if their is a lot of animations for the body and face.
I don’t know if this works - you will have to test it to confirm - though if it does - it will be a lot easier to animated to the sound track in Max that it will be in Unity. But check into the LipSync tool first - because it has a good reputation from solid developers who have used it for this exact type of character work.
“Biped with extra bones is an acceptable set up, but the pelvis and clavicles default parenting has to be changed so it works in Unity correctly”.
can you explain further or point to a tutorial on this? Is it something to do with the “triangle pelvis” biped options? This sounds like something I should do early down the line.
I’m also interested in the “look at” controller. Do you have links to any resources on manipulating bones using scripts? It won’t be anything too unusual, just overriding the animations to make a character’s head bone look at a target.
You can put facial animations and body animations on separate Animator layers, with the face layer masked to just the facial controls, so you can combine body and face animations. That’s exactly what layers are for.
That’s incorrect. Animations can control both joints and blend shape weights, and Unity imports blend shape keys just fine, at least when exporting from Maya (don’t know about Max). You can easily import blend shape-based facial animations into Unity.
You don’t need to use humanoid for mocap. I’ve imported mocap into Unity onto generic skeletons. I just imported the mocap clips into Maya onto a HIK rig for retargetting and cleanup and baked it out. Humanoid is useful for in-engine retargetting and IK, but there’s nothing about mocap that requires it.
Yeah change the default settings on triangle pelvis and triangle neck. By default biped legs are parented to the bottom spine bone, and the clavicles are parented to the neck. Turn on bones and links in the biped display options to confirm it is set correctly.
There are a couple IK videos on youtube that you can watch. They are pretty good about explaining this. You will use built in IK if using humanoid - this will serve as the look at constraint, or you will have to use an asset store IK system if using generic rig.
Of course this is an option - biped imports all mocap file types - but this is a person asking about the default biped set up - so I’m not attempting to explain that process. Importing mocap onto a generic rig is also an advanced process, and only works if the rig in 3D matches exactly the generic rig in Unity. It’s a simpler process to use humanoid.
Agree with all your other comments.
This is basic information on the built in IK system.
This is the google search I used. - https://www.youtube.com/results?search_query=ik+in+unity5
Check some of the other links as well. They are not exclusively look at headIK - they show the fundamentals of the IK system - which is better to know rather than how to specifically set up a look at system.
