Set eye displacement to 0 with "Rotation Only" tracking

Unity Engine
,
1

Is there any way to set the eye displacement to 0 when setting the tracking type to “Rotation Only”?

I am using the OpenXR plugin (1.8.2) with Meta Quest support in Unity 2022.3.4f1.

I need to render a 3D character that is placed in the middle of a 360 video skybox. In order to prevent the character from moving relative to the skybox, I need fix the headset position and only allow head rotations. I can set the “Tracked Pose Driver” to track “Rotation Only” which kinda works to prevent the 3D character from moving relative to the skybox, but when you tilt your head, you can still see a minor offset in the character position relative to the background. After several hours of testing, I suspect this is due to the fact that there is still an offset between the left and right eyes which creates a change of relative position when tilting the head.

9747541--1394836--upload_2024-4-3_16-16-35.png
9747541--1394836--upload_2024-4-3_16-16-35.png860×657 382 KB

3D volumetric character standing in a 360 video of a living room.

My question is: Is there any way to set the eye displacement to 0 so that no translation occurs when tilting your head?

Note: I have some scenes with 3D characters in a 3D world. In this case, I need both position & rotation tracking. Only in the scenes with a 360 video skybox do I need to set the eye displacement to 0, so I would need to be able to configure it on a per-scene basis.

Thanks in advance!

2

I have tried to set Camera.stereoSeparation to 0, but nothing changed:

m_camera.stereoSeparation = 0.0f;
m_camera.ResetStereoViewMatrices();

I’ve tried overriding the left/right stereo view matrix like this:

var m = m_camera.worldToCameraMatrix;
m_camera.SetStereoViewMatrix(Camera.StereoscopicEye.Left, m);
m_camera.SetStereoViewMatrix(Camera.StereoscopicEye.Right, m);

But then the right eye doesn’t render (the right eye is just black and a bunch of errors are spewed to the console, at least when testing in the editor, I did not try to create a build with this):

Assertion failed on expression: 'std::abs(det) > FLT_MIN'
UnityEngine.GUIUtility:ProcessEvent (int,intptr,bool&)

Screen position out of view frustum (screen pos 0.000000, 0.000000, 1000.000000) (Camera rect 0 0 0 0)
UnityEngine.GUIUtility:ProcessEvent (int,intptr,bool&)

Even after setting the stereo view matrix according to the example here: Unity - Scripting API: Camera.SetStereoViewMatrix, the errors persist and the right eye is black.

Also, doing this also seems to override any kind of motion smoothing and makes the view extremely jittery.

Any ideas to get both the left and right eyes on the same spot using the OpenXR plugin?

3

I don’t have a direct answer to this but maybe moving the character very far away and scaling it up might make the parallax issue less apparent?

4

Thanks for your reply, but there are more objects in the scene (both 2D and 3D) that are not shown in the screenshot, making this solution problematic.

I was hoping there would be a way to fix the IDP distance when setting the tracking type to “Rotation only”.

For now, I’ll leave it as-is until I can find a better solution (might mean that I need to try to switch back to OVR SDK (not ideal since it’s been deprecated).