Hand tracking in moving objects

Unity Engine
, ,
1

Hello! I have a network application that tracks my hands with the Vive Focus. I get the hands position of the rig with the following code:

if (hand.GetJoint(XRHandJointID.BeginMarker).TryGetPose(out Pose poseOut))
{
   handPosition = poseOut.position;
   handRotation = poseOut.rotation;  
}

if (HandTracking.subsystem.leftHand.Equals(hand))
   SendLeftHandPositionToServer(handPosition, handRotation);
else
   SendRightHandPositionToServer(handPosition, handRotation);

poseOut is the global position of my hands relative to the center of the scene. I need this position to set my avatar’s hand to show it to the other network users. I send “handPosition” to the server and set the avatar’s hand like this:

[ServerRpc]
private void SendRightHandPositionToServer(Vector3 rightHandPosition, Quaternion rightHandRotation)
{
   _avatarData.RootRightHand.position = rightHandPosition;
   _avatarData.RootRightHand.rotation = rightHandRotation;

   SendRightHandPositionToClient(rightHandPosition, rightHandRotation);
}

The problem is when my rig enters a moving object, like for example an elevator. I parent the rig to the moving object and my rig starts moving based on its behaviour. Because this movement is not registered by the rig, because I didn’t move in real space, my avatar’s hands stay in the starting point of the moving object. Is there a way to fix this problem?

If you need more context or code feel free to ask. Thanks :slight_smile:

2

I know this isn’t related to your question, but your second if-check (checking whether the hands are equal) can be replaced by checking if (hand.handedness == Handedness.Left).

To address your question though, hand data is not reported in game world space, but in “session” space (the origin is where you launched the app, and stays there, even if you teleport or use other locomotion to move in game coordinates). To account for this, you can add on the delta that you’ve moved from that origin manually.

3

Hey! Thanks for the reply. I don’t have access to the delta that I’ve moved, so I was trying to get the rig position. XROrigin does not work, because it’s always at (0,0,0) in my case. Is there a way to get the position of the rig? Doing so I can calculate a vector from the hands to the rig position and add it to the center of the moved avatar. Thanks!

4

I found the solution! XROrigin was always (0, 0, 0) in the build because it “resets” as I move through the real world space. The moment I enter the moving platform, it tracks the movement of the moving object (that is, the movement I have not done in real world). So I only had to do this:

[ServerRpc]
private void SendRightHandPositionToServer(Vector3 rigOrigin, Vector3 handPosition, Quaternion handRotation)
{
   SendRightHandPositionToClient(rigOrigin, handPosition, handRotation);
}

[ObserversRpc(BufferLast = true)]
private void SendRightHandPositionToClient(Vector3 rigOrigin, Vector3 handPosition, Quaternion handRotation)
{
   _avatarData.RootRightHand.position = handPosition + rigOrigin;
   _avatarData.RootRightHand.rotation = handRotation;
}

where rigOrigin is XROrigin.transform.position.
Thanks for the help :slight_smile: