I’ve been turning animation compression off–they cause problems even with generic avatars. (The default tolerances are way too high, and I’ve started using an AssetPostprocessor to lower them significantly.) I didn’t change the skeleton at all in Maya, I just made sure to only bind to HumanIK the joints that Unity understands, leaving the others in their bind pose.
The model is in a clean T-pose. The pose is slightly different in the avatar than in the FBX (modified in the avatar editor), but only down in the fingers.
One thing that I think clues at the problem, though, is that if you change the muscle ranges, the imported animation changes. If I take Spine Front-Back and change it from [-40,40] to [-180,180], the animation breaks. Now, if I was retargetting an animation from another character, I’d obviously expect that to break everything–it would be remapping the much smaller muscle range from the animation to a 360-degree range, and movements would be wildly amplified. But when I’m importing an animation for this avatar, I’d expect it to have no effect: the [-180,180] range is being mapped to the same [-180,180] range.
That’s obviously not how it actually works, but I haven’t been able to get a clear picture of why that happens from the docs. But a takeaway here is that the muscle ranges have an influence on the imported animation. If [-180,180] has a huge effect on it, it makes sense that a [-40,40] range could still have an effect, even if it’s very small. It’s not giving a 1:1 import with the big range, so it makes sense that it doesn’t give a 1:1 import with the smaller range either.
Note that the errors are small enough that people animating punches and sword swings would probably never notice (maybe 1-2cm in the fingers), but it’s a ton of error if a character is twining his fingers together. My first concern was whether that amount of error is actually expected in the humanoid system or not, so I didn’t spend too much time banging my head on it if it’s just an expected artifact. I do need to re-test with a simpler skeleton and compare, just haven’t got back to this yet. (I should probably write a script to export the world space positions of joints to make comparing between Maya and Unity easier, since the coordinate systems are different.)