Working on an AR project for the MagicLeap 2 (that runs on an android OS) using Unity (with Stereo Instanced Rendering) and the MRTK
I want to render the depth of an object into a render texture that I can read out in shaders later when rendering the normal view to screen.
For that, I have created a second camera as a child object of the main camera that renders only that object (via layer mask). When I render that second camera to the screen, I can distinguish between eyes in the shader using unity_StereoEyeIndex. But when I set the camera’s render target to a render texture, I cannot get the information from both eyes into that texture even when the target eye is set to “both” on the camera.
It is important to note that in the tests I made the rendering order seems to be
- left eye cam 1, right eye cam 2 2) left eye cam 2, right eye cam 2 so I need to write depth values for each eye into the tex and later read one of the two values respectively
I tried:
-
Using a 2D RGFloat render texture with size equal to the ML2 screen dimensions and writing the depth into color.r if unity_StereoEyeIndex == 0 or writing it into color.g otherwise. But when I read from the render texture when rendering with the main camera, I can only get a value from sampledColor.r, sampledColor.g is always 0. I have also tried doubling the render tex width to see if maybe both eyes get rendered next to each other. For testing purposes I have also tried simply displaying a quad with unlit/texture shader and the render tex as main texture so that I can see if something simply renders into the wrong coordinates but it also only has red shapes, no green.
-
Using a 2DArray render texture with 2 entries, declaring it with UNITY_DECLARE_TEX2DARRAY and sampling with UNITY_SAMPLE_TEX2DARRAY where uv.z = unity_StereoEyeIndex but with that approach I just see black.
Maybe using a RenderFeature is the better approach? Although I’m not really sure how these work with stereo instancing. It would also be okay if I can only get it to work with multipass rendering but so far I have had no luck with that either as unity_StereoEyeIndex is not set then and so I have no way of distinguishing between eyes in the shader.