IN-67537 - XR Hands Package Root Pose and Wrist Pose Do Not Match

We have found that VisionOSHandProvider randomly reports a wrist as untracked when the hand is reported as tracked and the root pose is valid.

I created a unity project using 2022.3.18f1 with VisionOS package 1.0.3 and XRHands package 1.3.0 embedded so I could add logging to both the VisionOSHandProvider and the XRHandSubsystem. In the VisionOSHandProvider, the logging outputs the frame count along with the root and wrist pose for the hand or “Not Tracked” for the wrist when NativeApi.SkeletonJoint.ar_skeleton_joint_is_tracked returns false. The logging in the XRHandSubsystem outputs the frame count, root pose, wrist pose, and provider reported update success flag value.

I ran the project on an Apple Vision Pro device running version 1.0.1 of the OS and just held my hands out in front of me without moving them for a few seconds. The hands remained in place (jittering as usual), but the captured log output contained many entries like the following from the XRHandSubsystem logging that I added.

[F621] {XRHandSubsystem} Right Hand Root/Wrist Pose Mismatch! Root: (-0.046251, 1.069394, 0.387736); Wrist: (0.000000, 0.000000, 0.000000); Success Flags: All

This output indicates that the VisionOSHandProvider is reporting a successful update for the root pose and the joints, but the root and wrist poses do not match.

According to the XR Hands package documentation, the root pose is, “Located at the wrist joint, the forward vector of the hand points in the direction of the fingers”, which implies that the root pose and wrist pose should always match and that is clearly not the case with Apple Vision Pro.

I am thinking this could be part of the issue with the hand render jittering, but has already proven to be an issue for our extrapolation of hand root pose data as setting the root pose on the hand in an IXRHandProcessor does not change the wrist location.

On top of the above, I just realized that the XRHand::SetRootPose method is an extension method in XRHandProcessingUtility and when it is called it applies the new root pose rotational offset to every hand joint (including the wrist). In the case that the root pose and wrist pose are the same this would not be an issue, but when they are different this would cause even more issues, right? Why is the wrist pose not just set to the new root pose when the root pose is changed?