Differences between humanoid and generic animations

Unity Engine
1

We are trying to import an animation from Maya to Unity, in which the character takes a picture. We need the bones’ positions and rotations to exactly match the source take, including fingers, because the animation will be seen from up close.

In generic mode, the bones are correct. However, in humanoid mode, they are wrong.

We need to use humanoids, because we use inverse kinematics. We can’t stay in generic.

Here is some basic background information:

  • The animation file is called “TakePhoto.fbx”
  • The reference avatar, in T-pose, is called “TPose.fbx”
  • The Maya rig is based on Morph3D’s MCSFemale model.
  • Ultimately, we are trying to apply the animation back to the Morph3D model.
  • Animation compression is turned off everywhere.
  • Optimize game objects (from rig) is turned off everywhere.
  • Apply root motion is turned off everywhere.
  • We get no warnings or errors on import of any model or animation.

Our avatar definition (same in TPose / TakePhoto) seems correct:

349×919 42.3 KB
345×783 24.3 KB

(The right hand is mirrored).

Note that we are Enforcing T-Pose.

In order to debug, we created a script that shows skeletons in the inspector, and another one that compares bones position and rotation between two skeletons.

We have 5 test models so far. None of them have the same behaviour as the reference generic animation.

1889×973 196 KB

Here they are at a crucial point in the animation, where all fingers are solicited.

1889×971 231 KB

Legend:

  • A green dot indicates that the bone’s position and rotation is identical to the reference.
  • A yellow dot indicates that the bone’s rotation is correct, but its position is incorrect.
  • An orange dot indicates that the bone’s position is correct, but its rotation is incorrect.
  • A red dot indicates that neither the bone’s position nor rotation is correct.

We have a comparison sensitivity threshold, which right now is set to 1e-5. Even at 1e-1, we get orange / red dots. We are comparing position.xyz / rotation.wxyz.

We use incremental comparison to know where the process breaks. In other words:

  • Test #1 is compared to Reference
  • Test #2 is compared to Reference
  • Test #3 is compared to Test #2
  • Test #4 is compared to Test #3
  • Test #5 is compared to Reference

The reason for test #5, which is basically test #3 but imported directly from Maya rather than from FBX, is because this question on Unity Answers implied they got better results when baking directly from Unity. That doesn’t seem to be the case.

Since Test #3 is all green, we know that the problem doesn’t arise from retargetting from one model to another. Instead, as indicated by the errors in Test #1 and Test #2, the problem arises simply from using the humanoid system, even without any retargetting.

We’re aware retargetting is bound to have some discrepancies. However, we see no reason why simply using the humanoid system causes rotations and positions to be wrong. In other words, we want Tests #1 and #2 to be all green.

The one thing we haven’t experimented with is the muscle settings, which are for now all left at default (reset).

What are we doing wrong?
How can we correct the animation in humanoid?

2

From the images it looks like (maybe) the animation rig and the target rig are using different avatar definitions. What happens when they use the same avatar definition.

Are the animations created in maya using root motion? If root motion is created on the rig in maya and then removed on the import process - theoretically this could end up causing slight discrepancies.

Have you ran a test with root motion turned on for the animations?

3

We made sure all rigs use exactly the same avatar definition. There is no root motion applied in Maya; but even if this was the problem, we’d only see yellow dots (translation problem), not orange/red dots (rotation problem) starting at the shoulders.

4

Not sure if this is relevant - but I thought I’d pass it along since you mentioned Tpose several times, related - maybe there is useful info.

So each time the avatar definition was changed on the examples above - the avatar was changed on both rigs - target rig and imported animated rig?

5

Thanks for the suggestion. We tried sampling from the bind pose instead of enforcing and unfortunately it still doesn’t work. We also tried a better natural relaxed T-pose (non-enforced) which didn’t work either. We get the results similar to tests #2 / #3.

Yes, that’s correct.

We’re getting pretty desperate. Calling @Mecanim-Dev to the rescue?

1 Like
6

For the record, we tried updating to the latest Unity beta (5.4.0b21), but it doesn’t seem to help.

7

Some guesses … If you have more than two spine bones, humanoid will discard rotational data from the extras. That will cause a non 1:1 result.
If you have roll bones, unity will apply it’s own roll rotation, not the one in your data. But that shouldn’t affect the pose.
You should make sure your avatar is exactly the skeleton of your animation clip.
It’s I believe possible to get 1:1 with Humanoid. At least that’s what’s been stated before. I’d create the Tpose into your avatar definition outside of Unity - that way your sure and not relying on Unity to 'force bind pose etc.

Muscle settings I think wouldn’t have anything to do with this - yes the avatar is going into muscle space - but the muscle settings available through the initialisation are more to do with limiting motion ranges than anything else I think. Definitely not the thing you want to be doing when trying to get 1:1

The other guess is - Unity may not support exactly 1:1 in humanoid. I’d really like to investigate this a bit more myself. Have you got a close-up image reference of exactly the differences in pose your getting? I might have missed from the pictures but I can’t see exactly the pose difference / problems.

Oh also - same for neck. If you have more than 1 neck bone - it’s the same deal with Humanoid as for the 2 spine bones - Unity will ignore data from the extra under humanoid. This might result in differing relation / distance from eye to camera - if that’s what the problem is your seeing.

1 Like
8

We have 3 spine bones and 1 neck bone, but we don’t animate the third spine bone to avoid discrepancies with Mecanim.

Getting back to this problem two months later, we still haven’t found any solution.

9

@LazloBonin : did you file a bug with this? I’d be interested to look at it

If you do, please paste the bug number here.

There are really way too many variables to consider for me to take a guess without being able to play around with the files, and, maybe there’s a bug in there that we haven’t caught with our tests.

If you’re still stuck at this point, maybe I’d suggest moving to generic, and rolling out your own IK. It’s not trivial, but it’s totally not impossible either.

As far as working directly from Maya files, that’s pretty much voodoo as far as I’m concerned. We call the fbx exporter on the Maya file with the default options so you’re probably better off controlling the export options yourself.

(Edited to fix autocorrect typos)

2 Likes
10

@LazloBonin
There is a few rules to follow to get a lossless normalized humanoid animation.

Take a look at the last part ‘Retargeting and Animation Clip’ which give you good hint where to look at.

The lossless conversion is not guaranteed if you don’t follow those rules.
We are refactoring the avatar import pipeline in 5.5 to give better warning when one of those rules is broken.

1 Like
11

ludiq,

Are you making sure that for all your animation files, that under Rig tab in import, that you are copying avatar definition from one place? (Ie: from you character file). This eliminates a lot of possibilities for error.

12

Hm, is it just me or did that page get a lot more thorough recently? I remember reading it, but not that last section. Our animator will definitely have a look, but from a quick glance, I think we’re respecting all the rules. I’ll try to file an isolated bug report next week w/ all the debugging scripts used in this thread in case we can’t get it fixed. Thanks for your help.

13

We decided to wait for Unity 5.5 and the updated pipeline to make sure everything was properly setup with our avatar. If it still doesn’t work then, we’ll submit the isolated bug report.

14

I’m sorry to up this post after a year, but here in 2017.1 I still have the same issue. Precise finger animation in Maya don’t looks the same in humanoid rig. Same model, no retargeting, proper t-pose and so on… Did anyone find the solution?

860×680 63.3 KB

793×456 40.9 KB

15

do you have animation on you’re in between bone? they should be rigid(no animation) as they are not mapped to a human bone.

from Unity Blog
Skeleton rig requirements
In-between bones
In many case, skeleton rigs will have more bones than the ones defined by the Humanoid Rig. In-between bones are bones that are between humanoid defined bones. For example, a 3rd spine bone in a 3DSMAX Biped will be treated as an in-between bone. Those are supported by Humanoid Rig, but keep in mind that in-between bones won’t get animated. They will stay at their default position and orientation relative to their parent defined in the Humanoid Rig.

On the first image we can see that you’re thumb is composed of 3 phalange and a inner finger bone?
You can only assign three bone for each finger, so if any other bone have animation that could create those kind of difference.

Look at the importer when you import this file, on the animation tabs there is a foldout with importer message which should tell you if anything goes wrong when we do import you’re file.

16

3259553--251276--WomanRig.gif

Thanks for your input! I have animated only 3 finger bones, everything else stay static. But I thought I see where the problem is. Seems like mecanim don’t use my joint orientations, instead it define new orientations based on t-pose. It works good on all fingers, but not for the thumb. I put it “straight parallel to the ground half way (45 degrees) between x and z axis” for the T-pose but it’s not enough. When I animate it in maya I use my joint orientations, but when it come to mecanim the orientations are changing and animation don’t match.
Is it possible to get somewhere the IDEAL mecanim t-pose skeleton to match my joint orientations with it?

17

Ok, now I definitely see where the problem is.

  1. Maya HumanIK rig allow the thumb more freedom than muscle space. (3 axis for every bone) instead of Mecanim (2 axis for the first bone and 1 axis for all other)
  2. It can not be cured by restricting HumanIK this additional freedom, because the axis that Mecanim use in muscle space is not the same axis the original rig have. Instead Mecanim interpolate it unpredictably from the t-pose (see the weird bend attached). It’s quite predicted for the bones aligned to axis, but not fore the thumb. So when I animate only bend axis in Maya - it’s not the same as animating bend in muscle space.

So, as I understand it’s technically impossible to transfer Maya HumanIK animation to the muscle space with proper precision. Am I right? (Am I describe it clear?)

1020×695 128 KB

1 Like
18

Right, there is no roll axis in mecanim for finger.

It should be possible, it probably related to maya’s joint orientation.
Could you log a bug with your project ? and we will investigate.

Both HumanIK and mecanim was written by me and robert lanciault, we did a lot of test with imported HumanIK rig into mecanim and it was working pretty well.

2 Likes
19

Oh, you guys did an outstanding job! It’s awesome instruments! I thought I’m doing something wrong at some point, so I’ll doublecheck. But anyway, could you provide me with a link where exactly should I submit my project and bug description?

20

Bug reporting is done from within the editor, under Help->Report a bug. You get an email with the bug # after a couple of minutes.

2 Likes