Creating a multi-camera SinglePass XR Plugin

Unity Engine
, , , , ,
1

Hello !

I am trying to achieve a very specific setup with Unity 6, but so far I haven’t been able to achieve something.

I have 2 stereoscopic screens, and I want to use them to display a 3D stereo image viewed by active shutter glasses. In Unity 2019, it was working well with the “Stereo Display (non head-mounted)”. Since it’s not available in recent versions, I am trying to implement it by hand.

I’ve succeeded in doing a stereo display by having 4 cameras (2 per “face”, did that in C++), but in the 2019 version of Unity, I only needed 1 camera per face (so 2 in total, since Unity was able to compute the stereo matrices). Using 4 cameras is highly costly in CPU time, because the culling is called twice per face (on each eye), the render graph is also called many times, etc… . So I am trying to implement what we can call Single Pass Multi-Camera Stereo, but so far, I haven’t got any positive result. Here is what I’ve tried so far :

  • Create an XR Plugin to try to gather the generated textures native pointer. Since this approach implies Texture2DArray, RenderDocs displays the left and right eye, but I haven’t been able to get the D3D11 subresources. I also don’t understand how to handle 2 cameras.
  • Creating in Unity a RenderTexture with a volumeDepth of 2, to see if Unity can generate XR slices, but it doesn’t. Apparently, an XR plugin must be used to have the stereo enabled on cameras. Rendering to a RenderTexture breaks the stereo.
  • Creating URP render feature, but I’ve read online that the Camera.Render() call order will prevent this approach to work.
  • Creating a Native Plugin using Unity API, but I am still facing the problem of getting texture pointers and generating a multi-camera stereo data.

The best approach will be the XR Plugin (C++ code), but I don’t know if it’s possible to use that with more than 1 camera, since this plugin is designed for HMDs.

Or maybe trying to achieve this kind of Single Pass stereo is not possible at all and I am wasting my time. It might be good to have a clear answer to that, because I saw some posts on GitHub or Unity about the possibility of porting the Stereo Display (non head-mounter) on newer Unity versions.

Any ideas?

Thanks!

2 Likes