XRHands isTracked detection is unreliable

Target Platforms XR
,
1

It seems the XRHands relies on whether the the hand root pose or joints were updated to dictate whether the hands are tracking or not.

This results in not being able to differentiate whether we’re actually using hand tracking or controllers on quest, if the OpenXR runtime has some sort of joint emulation.

This is the case with the virtual desktop’s new OpenXR runtime (VDXR) and quest, as it says the hands are tracked even when using controllers (on XRHands package).

I basically have no way to tell whether the controllers are being used or the actual quest hand tracking. Since handIsTracked property from XRHandTrackingEvents will always be true

https://docs.unity3d.com/Packages/com.unity.xr.hands@1.4/api/UnityEngine.XR.Hands.XRHandTrackingEvents.html#UnityEngine_XR_Hands_XRHandTrackingEvents_handIsTracked

Reproduction Steps:

  • Use Virtual Desktop with the VDXR OpenXR runtime on Quest
  • Run the XRHands sample HandVisualizer scene
  • Notice the hands are tracked even when using the controllers, because it has emulated hand root pose and joins.
  • Issue: In the example scene the hands should only show when the hand tracking acquired event is triggered, but as we can see the hands are always tracked regardless.

You can see in the video:

  1. I start with controllers (emulated joints)
  2. Then swap to the actual hand tracking (actual hand tracking)
  3. Then switch back to controllers (emulated joints again)

4d6w08

This is an issue because controllers have different behavior from actual hand tracking on my use case (I have special interactions when using actual hand tracking). But in this case I have no way to differentiate whether I have actual hand tracking or using the controllers on quest.

2

Is this only a problem over virtual desktop, or does it work properly on an actual device?

3

The standalone android quest OpenXR runtime doesn’t emulate joints, so it doesn’t have this issue.

I’m not sure if there are other OpenXR runtimes that emulate joints when using controllers, but AFIK this behavior only exists in on Virtual Desktop’s OpenXR runtime.

4

All we can really do on our end is rely on what the runtime is reporting. You could report your desired behavior as a bug/feature request to whoever makes that runtime, or perhaps intercept the relevant call in OpenXR to change its behavior, but that’s a more involved.