Change camera output on the fly on pixel level

Unity Engine
1

Hi

I have a very special use case where i need to access the output of several cameras as Color or byte arrays, merge these data and display the results in real time (imagine a mosaic of, lets say, 8 camera videos). reason for that is a projekt i am working on which uses this output to display it as stereo3d, just from different perspectives so i need more than 2 cameras to get switchable angles.

what is the best way to grab the final rendered image from multiple cameras, process it, and pass the result on to the screen?

so far i tried using OnRenderImage to get camera data, ReadPixels (of Texture2D Object) to get the Pixel Array and onGUI() to draw the result. But the result is far from even okayish because the result is not rendered constantly but maybe with 1fps while the actual camera output of the maincamera is still shown in the background. i need to change the output of the main camera to the result of the processing of the color-arrays.

so is there any way to accomplish this or does unity hide too much low lvl data behind read only variables so it cant be done?

2

ReadPixels is very very slow because the CPU has to go through each pixel 1 by 1 in a texture.

I recommend setting a “render texture” to your cameras so each camera outputs a texture instead of rendering directly to the screen. Use a command buffer to wait until all cameras are done rendering, then you can take all those textures and pass them into a shader using the commandBuffer.Blit() command.

Blit is way, way faster than readPixels because it happens on the GPU.