I’ve having some difficulty implementing facial animations using Mecanim and I was wondering if anyone has tried this before and could offer some insight.
Here’s the problem:
I have my animator set up, and want to have a layer or more above for facial animations. That way your character can start speaking or reacting with their face only while otherwise animating normally. However, neither Override nor Additive seems to be giving me the desired result. If I use Override then I have to set up an Avatar Mask, but there is no option to mask out everything except the face, I have to mask out everything except the HEAD, which means that when the animation blends on I lose all head and neck movement, which ruins the animations. I tried using additive, but that’s not working properly either because it seems that Unity just ignores any subtle animations. Additive works if I’m having the character drastically move their face around, but if I want anything subtle, such as a frown or smile or some basic face idling then it doesn’t register it. Very odd.
I should mention that this is using a Humanoid rig, and for some reason it DOES work properly when using a Generic rig. If I set it to Override on a Generic rig it will only override the bones which the overriding animation has key information on, unlike in Humanoid where it just overrides EVERYTHING. But I would really prefer not to change over the character and all his animations to a generic system, since I’m using other features from the humanoid rig already.
Has anyone tried to get this working before and found a good route?
Alright I seem to have discovered a semi-solution/workaround, for anyone that happens to stumble upon reading this.
It seems that facial animations need to be BELOW any other animations in the layer hierarchy. I figured the obvious way to do things would be to layer them ABOVE, but apparently that’s not what mecanim wants. However if I put the facial animations on the Base layer and then have all the movement animations on a layer ABOVE (set to override) then the overriding works as it should and ignored the bones in the face as long as there is no keying information on any of the face bones.
This doesn’t work quite like I wanted to either, since it means that all movement animations have to have absolutely no facial animation. I would have been nice to have had subtle facial animations on the movement anims that could then be overridden when the character needs to talk or do a specific facial animation, but that’s not going to work it seems.
I think the reason for it working this way is that the facial bones are considered “extra bones” my mecanim, and so they’re able to work properly and mecanim is able to check if there is keying information and use override properly. Other basic body bones however get converted to whatever system mecanim is using for retargeting, and so it loses the ability to check if the bones have keying info for those bones and thus override doesn’t work properly.
This gets to be extra annoying because my programmers have told me that we NEED to have basic movement animation on the Base layer for some reason. Which means that I’m going to have to have TWO identical layers for movement: one on the Base layer, then above that the facial animation layer, then the second identical copy of the movement layer playing ABOVE the facial animations (probably set with a head mask to save some processing).
So this solution works, but it isn’t particularly pretty. Hopefully Unity will spend some more time on these blending layers and fix some of these problems at some point, but until the this’ll have to do.
Another way to do facial animation is to actually make uses of a parental hierarchy so that unity see both animation (for example body & face) separately.
The way of doing this is by making the “head” and the “body” into separate meshes.
You put a node/empty/point (or whatever kind of position coordinate thing your 3D software uses) at the position of the neck on the body. (For the sake of making it easy to understand, we’ll call that point/helper “BP”)
You make sure that the origin point of the head meshes is at the “same” place as the BP you added on the body’s neck.
You place the head meshes in the body’s asset so that it become the “children” of the BP.
Now, all you have to do is add a public script so that whenever you want the body to do an action and the head to do an action, both strings are “pulled”.
The main issue with this technique is that it require you to understand what you’re doing and coordinate 2 animations for the same asset. But the exceptional side to it is that you can actually manage the expressions and movement of the face through a more complete way by using the Avatar/Mecanim system. (Even if it’s wrote as "face, spine, forearms, etc., you can use it as a simple bone hierarchy system for a full complex facial system using multiple bones for the face. Basically, the Mecanim system is just a pre-made system of copying the movement and rotation data of parents & children bones.)
So, even more, this also allow you to make uses of the Avatar/Mecanim system to transfer complex expression and facial bones movement between multiple target characters.
This way of using it also allow you to make expressions in a way that doesn’t always follow the body. (For example, the character is hit. His face doesn’t show that he’s hit as he’s hit, but after he notice he’s hit and feel the pain. That happen as the player is already starting a new action unrelated to the “getting hit” animation of the body. It also allow complex animations unrelated to the body… like phonemes.)
Note that this method does have a heavy downside : by having 2 sets of armatures system and animation layout, it’s like if you’re using “2 characters” for the same. It does uses a bit more resources. (You planned to have like 100 animated NPCs on screen, with that, you’ll move down to 50.) Just saying.
Another downside is that you need to make sure that the head is well “attached” to the body. It require something like a joint of some sort between the head/neck sections of the 2 meshes. 98% of the time, it should be okay… but if the character is bare chest, it might be hard to “weld” correctly the 2 meshes together.