Accessing EyeTracking Device on Vive Pro Eye

Similar to this thread Accessing EyeTracking Device on Hololens 2 , I am trying to use the eye tracking API to get the fixation point. When i list the devices, like in the previously mentioned Thread, I get “name:Head Tracking - OpenXR, role:Generic, manufacturer:OpenXR, caracteristics:HeadMounted, TrackedDevice”.

The thing is that i can access the individual eye position and rotation by using:

        if (TryGetPosition(UnityEngine.XR.CommonUsages.leftEyePosition, XRNode.LeftEye, out var leftEyePosition)
            && TryGetRotation(UnityEngine.XR.CommonUsages.leftEyeRotation, XRNode.LeftEye, out var leftEyeRotation)
            && TryGetPosition(UnityEngine.XR.CommonUsages.rightEyePosition, XRNode.RightEye, out var rightEyePosition)
            && TryGetRotation(UnityEngine.XR.CommonUsages.rightEyeRotation, XRNode.RightEye, out var rightEyeRotation))
        {
            Debug.Log(string.Format("Left Eye Rotation2:{0} , Position2:{1}", leftEyeRotation, leftEyePosition));
            Debug.Log(string.Format("Right Eye Rotation2:{0} , Position2:{1}", rightEyeRotation, rightEyePosition));
        }

bool TryGetPosition(InputFeatureUsage<Vector3> inputFeatureUsage, XRNode node, out Vector3 position)
    {
        UnityEngine.XR.InputDevice device = InputDevices.GetDeviceAtXRNode(node);
        if (device.isValid)
        {
            if (device.TryGetFeatureValue(inputFeatureUsage, out position))
                return true;
        }
        position = default;
        return false;
    }

    bool TryGetRotation(InputFeatureUsage<Quaternion> inputFeatureUsage, XRNode node, out Quaternion rotation)
    {
        UnityEngine.XR.InputDevice device = InputDevices.GetDeviceAtXRNode(node);
        if (device.isValid)
        {
            if (device.TryGetFeatureValue(inputFeatureUsage, out rotation))
                return true;
        }
        rotation = default;
        return false;
    }

That is why i am wondering why i can still not access the Eyes Struct since TryGetFeatureValue in :

bool TryGetEyes(out Eyes eyes)
    {
        UnityEngine.XR.InputDevice device = InputDevices.GetDeviceAtXRNode(XRNode.CenterEye);
        if (device.isValid)
        {
            Debug.Log(string.Format("name:{0}, role:{1}, manufacturer:{2}, characteristics:{3}, serialNumber:{4},",
                device.name, device.role, device.manufacturer, device.characteristics, device.serialNumber));
            if (device.TryGetFeatureValue(UnityEngine.XR.CommonUsages.eyesData, out eyes))
                return true;
        }
        eyes = default;
        return false;
    }

always returns false.
Since there isn’t any indication in the documentation (see Unity - Manual: Unity XR Input under “Accessing eye-tracking data”) that this shouldn’t work I am kind of confused.

Is the Eyes Struct currently just not accessible and do I have to work with the individual eye position or am I perhaps missing something else to access the Eyes struct.

Thank you for the detailed information, we will take a look.

1 Like

Looked into this a bit closer and with OpenXR eye tracking does not work using the CenterEye XRNode as far as I can tell. Eye tracking for openxr requires that the EyeGazeInteraction be added to the interaction profiles and then you can make an action that binds to <EyeGaze>/pose. This of course only works if the runtime and device you are using supports tracking eyes.

Thank you for the suggestion and quick reply. I tried out the binding path but unfortunately with no success. I get left and right eye position, but after thinking about it again those probably have nothing to do with eye tracking.
Settings wise i activated openXR:


Selected these interaction Profiles:

I also tried the full position (/pose/position) and rotation (/pose/rotation) paths:

Steam VR uses the SRanipalRuntime so i am currently running that. Since I calibrated the Eye Tracking on my Device I am assuming that my Vive Pro Eye has working Eye Tracking capabilities. I also checked with the device of a colleague and it didn’t work on his either. So a device problem is probably unlikely. Not sure what else i can test to be honest. Any other suggestions would be highly appreciated.

Just to give more context these are the values i get in the XR Interaction Debugger for the OpenXR Head Tracking device (where i am assuming the gaze values should show up).

If you check your logs you will be able to see which OpenXR extensions were enabled. If the eye gaze extension is not there then it wont work unfortunately. If you want to paste your editor.log (assuming you are running it in the editor) we can take a look. I do not know if the Vive Pro Eye has the proper extensions implemented or not in the steam runtime.

I missed this part, the eye gaze values would show up in their own Eye Gaze device if everything is working correctly, they are not part of the Head device.

Here you go. But since it doesn’t show up as a separate device like you said it is probably just not supported yet for the Vive

7208284–865951–Editor.txt (1.42 MB)

I checked with valve and it unfortunately is not. No ETA as of yet as to when it will be.

ah what a shame. Thank you for your time though.

Good afternoon,

I stumbled across this threat, and was wondering if there is an update on this topic.
We are running into issues combining OpenXR & the Tobii SDK usign the HTC Vive Pro Eye, and wondered whether the Vive Pro Eye & OpenXR Gaze-Interactions are already supported.

We are using Unity 2022 & OpenXR 1.4.x, however we do not get any data, and would like to see if its a problem on our end, or if the device is still not supported.

Kind Regards

2 Likes

Hi everyone,

We are also using Unity 2022 with OpenXR, and it seems like eye tracking does not work.
Any update on the topic ?

Thanks

Hej :slight_smile:

I know this was 6 months ago but…
Any update on this?
Thanks !

Hey everyone, we are also trying to access eye tracking data from the VIVE Pro. We are using the OpenXR plugin and the XR Interaction Toolkit 2.3.x. Also we have been able to calibrate the eye tracking in the Vive dashboard.

I tried to access the position of the eyes through the EyePosePosition action reference with the binding “pose/rotation [Eye Gaze (OpenXR)]” which is preselected for the EyePosePosition action reference inside the action asset “XRI Default Input Actions” (coming from the Starter Assets from XR Interaction Toolkit) and then simply using the actionReference.action.ReadValue() method.
Although this concept works when applied to e.g. the controller position, this does not work for the eye tracking data for me.
Is maybe the Vive OpenXR Plugin required to access the eyetracking? Or is the OpenXR Plugin enough?
Maybe someone can help us out here.

One thing to check is if the testing device actually supports eye tracking feature.
I would recommend checking the player.log first and looking for Available Runtime Extensions: and see if XR_EXT_eye_gaze_interaction is listed there.
So far I know Hololens, Varjo and HTC vive cosmo supported eye gaze extension.

1 Like

Hey!
Vive Pro Eye does have eye tracking feature (listed below)
“[XR] [15772] [10:02:06.758][Info ] Available Runtime Extensions: (27)
[XR] [15772] [10:02:06.758][Info ] XR_EXT_hand_tracking: Version=4
[XR] [15772] [10:02:06.758][Info ] XR_EXT_eye_gaze_interaction: Version=1”

Do you know what we should do next? Just like @Virtual_Ross , I’ve tried a bit of everything but can’t make the OpenXR Eye-tracking work :frowning:

If anyone pass by, feel free to provide any tips!

if could try enabled Eye Gaze Interaction Profile in the OpenXR setting UI, like this:
8884413--1214184--upload_2023-3-17_13-32-59.png

and then you can make an action that binds to /pose.
Previous post with couple screenshots was the right way to use it.

Hey!
You’re right, got it working, thanks :slight_smile:
But since it’s not the gaze that is directly returned (no eyes struct)… I think I’ll stick with Vive solutions for now!

Thanks for your help !

@AxelGogoris I’m at the same point but still stuck. Could you please explain what you did (and did not?) install in Unity (2022) to get your Pro Eye working with the OpenXR and the Eye Gaze profile?
The GazeInputManager does not detect eye tracking on my Pro Eye (connected, eye tracking enable, calibrated).
I tried installing the Vive OpenXR extension, but that did not make any difference.
I noticed that OpenXR with Eye Gaze Profile does not automatically start SRanipal (it does start SteamVR). Not sure if we still need that though! SRanipal icon stays orange, so it seems that interface is not working without it being called upon fro Unity (using the legacy Vive Unity SDK).

When I look in my Editor Log I see the XR_EXT_eye_gaze_interaction ‘is not supported by runtime’. So something doesn’t work…

@Tech-Labs , @AxelGogoris , same issue here with Vive Pro, still looking for answer,
[XR] [23644] [16:26:15.115][Info ] Requested feature extensions not supported by runtime: (1)
[XR] [23644] [16:26:15.115][Info ] XR_EXT_eye_gaze_interaction: Feature=“Eye Gaze Interaction Profile”: Version=0.0.1, Company=“Unity”