Seeking Assistance on Integrating Valve Index Controllers with Unity OpenXR for Finger Tracking

[unity_7CF105CC8030CC58A856]

I am currently working on a VR project using Unity’s OpenXR plugin, primarily with a Varjo VR headset and Valve wand controllers. My goal is to upgrade our setup to incorporate Valve Index Controllers, taking advantage of their sophisticated finger tracking capabilities.

While the basic button interactions and controller functionalities transition smoothly, I’ve hit a snag with implementing finger tracking. The finger tracking data, which is not critical but would make a great addition to our application, doesn’t seem to function under OpenXR as it does with OpenVR/SteamVR.

Here’s a snippet of the code that works flawlessly when running under OpenVR, specifically retrieving the pinky finger curl data:

    public SteamVR_ActionSet myActionSet;
    public SteamVR_Action_Skeleton skeletonAction;

    private void Start()
    {
        myActionSet.Activate(SteamVR_Input_Sources.Any, 0, true);
    }


    private void Update() {
       ///
       PrintFingerCurl();
       //
    }

    private void PrintFingerCurl()
    {
        if (!skeletonAction.active)
        {
            Debug.Log("Skeleton action not active");
        }
        if (!skeletonAction.poseIsValid)
        {
            Debug.Log("Pose not valid");
        }

        if (skeletonAction.active && skeletonAction.poseIsValid)
        {
            float pinkyCurl = skeletonAction.fingerCurls[(int)SteamVR_Skeleton_FingerIndexEnum.pinky];
            Debug.Log($"Pinky Curl: {pinkyCurl}");
        }
    }

However, when attempting to operate within the OpenXR framework, the system reports that the “Skeleton action is not active” and the “Pose is not valid”, issues I never encounter in OpenVR. It seems like the skeleton data isn’t being activated or is invalid despite my efforts to ensure all action sets and bindings are correctly configured and activated.

Given that switching the entire project’s base from OpenXR to OpenVR isn’t a feasible solution, I’m looking for advice, insights, or any shared experiences on how to properly integrate Valve Index Controllers for finger tracking within the OpenXR environment in Unity.

Does anyone have experience with configuring the Index Controllers for detailed skeletal tracking using Unity’s OpenXR plugin? Any tips on ensuring that the skeleton actions activate correctly and that the pose data remains valid?

[VRDave_Unity]

Hey @unity_7CF105CC8030CC58A856 . The Index controllers should have support for the OpenXR XR_EXT_hand_tracking extension via SteamVR. I just tried it on our base-level sample and it worked for me with the Index Controllers. I was using Unity 2021, OpenXR 1.10.0 and XR Hands 1.5.0-pre.1 (1.4.1 should work as well). You can import the HandVisualizer sample from the XR Hands package to test it out without building your own scene. Here is a screenshot of the configuration, including OpenXR configuration:

One thing of note. I did not turn on the Index controller profile which may change how the OpenXR runtime responds and sends data to the app. You might try with and without to see what happens.