Rigidbody follows transform, but it falls behind when transform's parent is moving

Hi! I’m experimenting with VR, and I’m stuck on this problem for days now. I’ve tried numerous ways to solve this, all ended with the same results.

I want the player’s hands to be physics based; the hands should constantly check for collisions. My problem is, when the character is moving with the Character Controller, the physical hands with the rigidbody (which are trying to follow the “real hands”) are falling behind. When only the player’s hands are moving (and not the character’s body), the Rigidbody follows the target nicely.

The hierarchy:

  • Character (With Character Controller)
    ** Hand Left
    ** Hand Right
  • Physics Controller Left (Following Hand Left)
  • Physics Controller Right (Following Hand Right)

The hands and character controller is updating in Update. I’m updating the Rigidbody in FixedUpdate. It feels like the FixedUpdate is behind frame(s) when the Character Controller is moving the Character.

On the picture I’m following the Hands with this simple line in FixedUpdate:
Rigidbody.MovePosition(Target.position);
But I’ve tried modifying the velocity and doing some math with the chararacter controller’s velocity, hand’s velocity. I tried applying forces. I even tried moving the character’s body with rigidbody too just as a test, same results. Nothing seemed to work, all ended up with this delay or introduced other problems.

I’m using Unity’s XR Dynamic Move provider to move the character, but I wrote a custom one too just to check and the results were the same.

Does anyone have any idea what’s up with this? I spent too much time on this and can’t figure it out. Probably I’m missing something here completely, I’m usually not working with physics in Unity. The issue has nothing to do with controller (hand) tracking because if I turn that off the issue still occurs.