Copy Depth Buffer to RAM

Unity Engine
1

Hello everyone,

I’m currently working on a project where I want to capture the depth buffer from the main camera and copy the contents to RAM, so that another application can access it and perform real-time computer vision.

I can’t seem to figure out how to do this. I’m able to write a shader that uses the depth buffer as the screen color and then read from the screen, but I want to be able to read the depth buffer behind the scenes without the user seeing it.

Any help or tips would be appreciated.

Thanks!

2

So far, I have tried running Unity in OpenGL mode and using glReadPixels in a render plugin with GL_DEPTH_COMPONENT as the format to grab the pixels from the rendered screen. However, it seems that no matter when I run the plugin, it always returns 1.0f for all pixels in the rendered area, indicating that the depth buffer is cleared, even when I set the camera to render a depth texture. This strategy works in Blender so I thought it might work in Unity. I am only able to use glReadPixels to read the color of the rendered pixels in Unity.

Any ideas for alternative strategies would be appreciated.

3

Copying to RAM will probably be slow, but you should at least be able to do it. I would make two RenderTextures. One color without a depth buffer and the other in depth format. Then you assign them to the camera using SetTargetBuffers. You should then be able to read the pixels from the depth RenderTexture after rendering.

RenderTexture color = new RenderTexture(width, height, 0, RenderTextureFormat.Default);
RenderTexture depth = new RenderTexture(width, height, 0, RenderTextureFormat.Depth);
camera.SetTargetBuffers(color.colorBuffer, depth.colorBuffer);
// Then after rendering you can read from the depth RenderTexture.