I have depth maps and color images (both of them are from a camera), and I create some virtual objects to interact with real objects in the scene. How to render both of them (in Unity)? Maybe a shader will work. But I am relatively new to Cg. In a pixel shader, how to compare depth values of different objects which are projected to one same pixel? Thanks.
So what in my mind is this: A color image with depth data at each pixel (get from a camera), and several virtual objects in Unity. I want to overlay them. I want a pixel shader to check depth buffer for every pixel. If the depth value of the pixel from color image is smaller, rendering that pixel. Or if the virtual object is closer, rendering the virtual object.