We have a Unity VR environment running on Windows, and a HTC Vive XR Elite connected to PC. The headset also has the Full face tracker connected and tracking.
I need to just log the face tracking data (eye data in specific) from the headset to test.
I have the attached code snippet as a script added on the camera asset, to simply log the eye open/close data.
But I’m getting a “XR_ERROR_SESSION_LOST” when trying to access the data using GetFacialExpressions as shown in the code snippet below. And the log data always prints 0s for both eye and lip tracking data.
What could be the issue here? I’m new to Unity so it could also be the way I’m adding the script to the camera asset.
Using VIVE OpenXR Plugin for Unity (2022.3.44f1), with Facial Tracking feature enabled in the project settings.
Code:
public class FacialTrackingScript : MonoBehaviour
{
private static float[] eyeExps = new float[(int)XrEyeExpressionHTC.XR_EYE_EXPRESSION_MAX_ENUM_HTC];
private static float[] lipExps = new float[(int)XrLipExpressionHTC.XR_LIP_EXPRESSION_MAX_ENUM_HTC];
void Start()
{
Debug.Log("Script start running");
}
void Update()
{
Debug.Log("Script update running");
var feature = OpenXRSettings.Instance.GetFeature<ViveFacialTracking>();
if (feature != null)
{
{
//XR_ERROR_SESSION_LOST at the line below
if (feature.GetFacialExpressions(XrFacialTrackingTypeHTC.XR_FACIAL_TRACKING_TYPE_EYE_DEFAULT_HTC, out float[] exps))
{
eyeExps = exps;
}
}
{
if (feature.GetFacialExpressions(XrFacialTrackingTypeHTC.XR_FACIAL_TRACKING_TYPE_LIP_DEFAULT_HTC, out float[] exps))
{
lipExps = exps;
}
}
// How large is the user's mouth opening. 0 = closed, 1 = fully opened
Debug.Log("Jaw Open: " + lipExps[(int)XrLipExpressionHTC.XR_LIP_EXPRESSION_JAW_OPEN_HTC]);
// Is the user's left eye opening? 0 = opened, 1 = fully closed
Debug.Log("Left Eye Blink: " + eyeExps[(int)XrEyeExpressionHTC.XR_EYE_EXPRESSION_LEFT_BLINK_HTC]);
}
}
}
Hi. I have same issue.
I tried to log eye tracking data by the script below.
- Unity version is (2021.3.9f1).
- OpenXR Plugin for Unity version is 2.4.2.
- HMD is VIVE Pro eye.
using UnityEngine;
using VIVE.OpenXR;
using VIVE.OpenXR.EyeTracker;
public class EyeTracker : MonoBehaviour
{
void Update()
{
XrSingleEyeGazeDataHTC[] gazeData;
XR_HTC_eye_tracker.Interop.GetEyeGazeData(out gazeData);
if (gazeData != null)
{
XrSingleEyeGazeDataHTC leftGaze = gazeData[(int)XrEyePositionHTC.XR_EYE_POSITION_LEFT_HTC];
if (leftGaze.isValid)
{
Debug.Log($"Left Eye Gaze Position: {leftGaze.gazePose.position.ToUnityVector()}");
Debug.Log($"Left Eye Gaze Orientation: {leftGaze.gazePose.orientation.ToUnityQuaternion()}");
}
else
{
Debug.Log("Left Eye Gaze Data is Invalid.");
}
XrSingleEyeGazeDataHTC rightGaze = gazeData[(int)XrEyePositionHTC.XR_EYE_POSITION_RIGHT_HTC];
if (rightGaze.isValid)
{
Debug.Log($"Right Eye Gaze Position: {rightGaze.gazePose.position.ToUnityVector()}");
Debug.Log($"Right Eye Gaze Orientation: {rightGaze.gazePose.orientation.ToUnityQuaternion()}");
}
else
{
Debug.Log("Right Eye Gaze Data is Invalid.");
}
}
XrSingleEyeGeometricDataHTC[] geometricData;
XR_HTC_eye_tracker.Interop.GetEyeGeometricData(out geometricData);
if (geometricData != null)
{
XrSingleEyeGeometricDataHTC leftGeom = geometricData[(int)XrEyePositionHTC.XR_EYE_POSITION_LEFT_HTC];
if (leftGeom.isValid)
{
Debug.Log($"Left Eye Openness: {leftGeom.eyeOpenness}");
Debug.Log($"Left Eye Squeeze: {leftGeom.eyeSqueeze}");
Debug.Log($"Left Eye Wide: {leftGeom.eyeWide}");
}
else
{
Debug.Log("Left Eye Geometric Data is Invalid.");
}
XrSingleEyeGeometricDataHTC rightGeom = geometricData[(int)XrEyePositionHTC.XR_EYE_POSITION_RIGHT_HTC];
if (rightGeom.isValid)
{
Debug.Log($"Right Eye Openness: {rightGeom.eyeOpenness}");
Debug.Log($"Right Eye Squeeze: {rightGeom.eyeSqueeze}");
Debug.Log($"Right Eye Wide: {rightGeom.eyeWide}");
}
else
{
Debug.Log("Right Eye Geometric Data is Invalid.");
}
}
XrSingleEyePupilDataHTC[] pupilData;
XR_HTC_eye_tracker.Interop.GetEyePupilData(out pupilData);
if (pupilData != null)
{
XrSingleEyePupilDataHTC leftPupil = pupilData[(int)XrEyePositionHTC.XR_EYE_POSITION_LEFT_HTC];
if (leftPupil.isDiameterValid)
{
Debug.Log($"Left Eye Pupil Diameter: {leftPupil.pupilDiameter}");
}
else
{
Debug.Log("Left Eye Pupil Diameter is Invalid.");
}
if (leftPupil.isPositionValid)
{
Debug.Log($"Left Eye Pupil Position: {leftPupil.pupilPosition}");
}
else
{
Debug.Log("Left Eye Pupil Position is Invalid.");
}
XrSingleEyePupilDataHTC rightPupil = pupilData[(int)XrEyePositionHTC.XR_EYE_POSITION_RIGHT_HTC];
if (rightPupil.isDiameterValid)
{
Debug.Log($"Right Eye Pupil Diameter: {rightPupil.pupilDiameter}");
}
else
{
Debug.Log("Right Eye Pupil Diameter is Invalid.");
}
if (rightPupil.isPositionValid)
{
Debug.Log($"Right Eye Pupil Position: {rightPupil.pupilPosition}");
}
else
{
Debug.Log("Right Eye Pupil Position is Invalid.");
}
}
}
}
I also tried sample project instead, but same error occured.
Hi, I tried using your code and some other different settings too but its still the same error. Even I am trying to get the eye tracking data but no luck yet.
Let me know if you find any solution and I will do the same.
Thank you !
The GetFeature() function seems to return the Tracker correctly regardless of it being connected. I think this just identifies if this feature is enabled in the OpenXR settings.
OpenXRRuntime.IsExtensionEnabled() seems more accurate to identify if a certain extension is connected, I am using it to check if the eye tracker is enabled and it works. For the facial tracking, you would use OpenXRRuntime.IsExtensionEnabled("XR_HTC_facial_tracking") I believe.
This method is not flawless though: if I disconnect the headset but keep steam vr / vive hub opened, Unity will identify the extension as enabled if I run again.
