Pre-rendered depth doesn't work with perspective camera

Unity Engine
1

I am working on a project where I supply depth values to the depth buffer from depth values stored in a png file rendered out from Maya. It’s the effect they used a lot in the old days so they could get dynamic objects to cull with a pre-rendered 3D scene.

Anyway, I can render an orthographic scene and the depth values work as expected, but as soon as I try it with a perspective camera it’s a no go. It acts like the depth is way towards the front of the camera, but I’ve tried scaling and offsetting values and nothing works as predicted.

Is there something that I don’t understand regarding depth values? Aren’t they just values that range from 0 to 1? I’ve looked all over for answers and nothing seems to touch on this subject specifically.

The following shader is called by Graphics.Blit in my post effect that transfers the depth of my png into the depth buffer. Does anything look out of place? Do I need to adjust the values before transferring the depth from the image and into the buffer?

Shader "Hidden/DepthPainting"
{
    Properties
    {
        _MainTex ("Texture", 2D) = "white" {}
        _PaintingTex ("Painting", 2D) = "white" {}
        _PaintingDepthTex ("Painting Depth", 2D) = "white" {}
        _DepthScale ("Depth Scale", Float) = 1.0
        _DepthOffset ("Depth Offset", Float) = 0.0
        _DepthInvert ("Depth Invert", int) = 0.0
    }

    SubShader
    {
        Cull Off
        ZWrite On
        ZTest Off

        Blend SrcAlpha OneMinusSrcAlpha

        Pass
        {
            CGPROGRAM
            #pragma vertex vert
            #pragma fragment frag
           
            #include "UnityCG.cginc"

            sampler2D _MainTex;
            float4  _MainTex_TexelSize;
            sampler2D _PaintingTex;
            sampler2D _PaintingDepthTex;

            float _DepthScale;
            float _DepthOffset;
            int _DepthInvert;

            struct appdata
            {
                float4 vertex : POSITION;
                float2 uv : TEXCOORD0;
            };

            struct v2f
            {
                float2 uv : TEXCOORD0;
                float4 vertex : SV_POSITION;
            };

            struct fo
            {
                half4 color : COLOR;
                float depth : DEPTH;
            };

            v2f vert (appdata v)
            {
                v2f o;

                o.vertex = UnityObjectToClipPos(v.vertex);
                o.uv = v.uv;

                if(_MainTex_TexelSize.y < 0)
                {
                    o.uv.y = 1.0 - o.uv.y;
                }

                return o;
            }

            fo frag (v2f i)
            {
                // fragment out
                fo o;

                half4 p = tex2D(_PaintingTex, i.uv);
                float d = (tex2D(_PaintingDepthTex, i.uv).r);

                o.color = p;
                o.depth = d * _DepthScale + _DepthOffset;

                #ifdef UNITY_REVERSED_Z
                    o.depth = 1 - o.depth;
                #endif

                if(_DepthInvert == 1.0)
                {
                    o.depth = 1 - o.depth;
                }

                return o;
            }
            ENDCG
        }
    }
}
2

Here’s a picture that represents a debug scene I created showing the issue. White represents the far plan, and grey (0.5, 0.5, 0.5) should be exactly in between the near and far planes. It works for orthographic cameras, but not for perspective.

3162220--240611--culling behavior.png
3162220--240611--culling behavior.png1024×768 95.5 KB

3

Hi weatxyz,

the depth values are indeed in range [0,1], but the difference between orthographic and perspective projection is that in perspective depth is non-linear. There’s a lot of docs online about how exactly it’s calculated.

4

Hi aleksandrk,

From what I just read it sounds like non-linear depth is the distance from the camera/eye origin to the fragment, and linear is distance to the camera/eye plane. I thought it just stored depth one way. I will play around with some things and come back if I have any more questions.

Thanks

5

My hat goes off to the math guys who came up with all this stuff. I thought I had a pretty good understanding of the graphics pipeline, that is until I hit the depth buffer challenge :eyes:.

The depth buffer is still a little bit of a mystery to me, but, I did stumble upon a solution. My idea was to convert the linear depth value to view space, multiply it with the projection matrix, then store it into SV_Depth. Here was the conversion (which for those excited for a solution, this isn’t it):

float4 vertex = i.vertex; // initially assign entire incoming vertex pos

// assign my pre rendered z position (I was just guessing with the w at this point)
vertex.z = preRenderedZ * vertex.w;

vertex = mul(UNITY_MATRIX_P, vertex)

o.depth = vertex.z;

So yeah, that didn’t work :stuck_out_tongue:

Then all this got me wondering about what w was for in the first place, so I went on a quest to learn more about it. I found this site which explains it pretty well.

However, that still didn’t help me find what I was looking for, so I did a search for “unity pre rendered depth”. Duh right? Well, I did searches similar to that before but the sites were always about enabling depth in a render texture, which isn’t what I wanted. This time I found this post (it’s the answer at the bottom that is of interest):
https://gamedev.stackexchange.com/questions/97034/3d-game-with-pre-rendered-background

At the bottom they guy points everyone to this place, which was in fact what I was looking for:

It took a while to figure this one out, but well worth it, because now I know what w is used for :smile: and I know about linear and non-linear depth buffers.

6

I noticed that there is a depth sacle and a offset in your code. Why there is such a thing? I find that my orthographic view need a scale and offset to work, but I don’t know why.