How to do a depth pre-pass before a surface shader

Unity Engine
,
1

Hi,

I am trying to compute the depth of a transparent mesh before drawing it, so that I can prevent overdraw within the mesh. The mesh has a vertex shader and a shape that changes every frame.

I’ve been looking at compute buffers and I’ve messed around with them a bit to get an understanding of how they work. I think I should be able to use them to do what I want to do.

I was just wondering if anyone could outline a high level plan of action here because I’m not sure if I have the right idea.

My current plan would be something like this:

Before drawing the mesh:

  • Calculate the shape of the mesh using the vertex function of the shader.
  • Calculate the clip space depth of each vertex.
  • Run a fragment shader to interpolate vertex depth across triangles and write shallowest values to a texture.

When drawing the mesh:

  • Only draw the fragment if the depth is <= to the depth saved in the buffer.

I just don’t know the best way to do this. I think I can add a command to my command buffer to draw a mesh with a given material so I could write a shader to write to a depth texture but then I have to run the vertex program again when I actually want to draw the mesh.

I’m also not entirely sure if I can just turn on ZTest for a transparent surface shader and let it do the work but I haven’t really investigated that yet.

Does any of this make sense? Or am I just entirely confused? Is a command buffer what I need or am I overcomplicating it?

Thanks in advance!

2

If you don’t mind it writing to the existing depth buffer, then you can indeed just set ZWrite true in the shader for that. And ZTest LEqual to test against depth so only front most faces of the transparent mesh render. Keep in mind though, any other transparent meshes you might want seen behind it aren’t going to render behind, since they’ll also get depth culled.

Your mesh rendering twice for what you’re proposing is normal behavior, that’s how Unity does it for all your Opaque objects. A depth pass is run on all the objects, and then a regular shader pass if deferred, or potentially many more passes if Forward rendered.

If you were using URP instead, it would be simple to write a Renderer Feature that renders a specific layer of objects to a separate buffer so you can use that buffer to compare against manually.

Otherwise the command buffer approach is the way to go, yes.

3

Thanks a lot for your answer!

This makes sense and is reassuring to know.

Having thought about it a bit since reading your response, I don’t think I can use ZTest because the overdraw I’m trying to solve is only happening with a mesh drawing over itself. As I understand it the transparent mesh doesn’t appear in the camera depth texture (as expected) and the verts are drawn in the order that they are stored in the mesh, so verts which are further back are drawing over verts which are closer when seen from some angles.

My transparent mesh will need the opaque depth information to render fog so I’m going to try to use a command buffer to render my transparent mesh to a separate texture and then test against that texture when actually rendering the mesh in the surface shader. I’m not yet sure how to prevent the mesh from rendering in the surface shader when I detect that it would result in an overdraw though. If you have any information about that I’d be super grateful.

Thanks again!

4

You wouldn’t really try to “detect” that it’s an overdraw. You’d just always be rendering that command buffer when Transparent objects that need to use it are going to be rendered, they would write to it before the Transparent Render Queue happens, and then your surface shaders rendering normally would need to be comparing their depth to the RenderTexture used in that Command Buffer and checking if their current fragment depth is further than the value in that buffer, if so, discard;

5

Yeah what I meant by ‘detecting overdraw’ is that fragments would see that their depth is further than the value in the buffer I rendered earlier. I just meant that I would need to do that check myself and then discard as you said (thanks for reminding me of the command btw I could only remember Clip()).

So if I understand correctly, I can’t use the built in ZTest to have my surface shaders compare their depth to my custom depth texture (unless I blit that texture to the existing depth texture). So I need to do that comparison manually and discard where necessary.

6

Yeah. I would still have ZTest enabled though so it will also compare against the existing depth buffer too. So either it gets culled early by opaque depth buffer automatically or else you manually cull by your custom pass data.

7

I’ve made some good progress with your help but I’ve run into another obstacle and I would love to sanity check my understanding and my approach.

I’ve written a shader which creates the shape of my mesh and returns the screen depth of each fragment and then I’ve used a command buffer to draw my mesh to a render texture with that depth shader.

The issue I’m having is that the render texture that is generated by the command buffer doesn’t match what I see in the game view if I just attach my depth shader to my mesh.

I hate to post a load of code here but this is my depth pass shader and command buffer code:

Shader "Unlit/OceanDepthPass"
{
    SubShader
    {
        Tags
        {
            "RenderType" = "Opaque"
            "Queue" = "Geometry"
        }

        Pass
        {
            ZWrite On
            ZTest LEqual
            CGPROGRAM
            #pragma vertex vert
            #pragma fragment frag

            #include "UnityCG.cginc"
            #include "WaterWaveUtils.cginc"

            struct appdata
            {
                float4 vertex : POSITION;
                float4 normal : NORMAL;
            };

            struct v2f
            {
                float4 vertex : SV_POSITION;
                float depth : TEXCOORD0;
            };

            v2f vert (appdata v)
            {
                v2f o;
                AddWaterWaves(v.vertex, v.normal.xyz);
                o.vertex = UnityObjectToClipPos(v.vertex);
                o.depth = COMPUTE_DEPTH_01;
                return o;
            }

            fixed4 frag (v2f i) : SV_Target
            {
                return i.depth;
            }
            ENDCG
        }
    }
}

    private void InitialiseCommandBuffer()
    {
        _oceanRenderers = GetComponentsInChildren<Renderer>();

        CommandBuffer buffer = new CommandBuffer();

        int depthBufferID = Shader.PropertyToID("_DepthBufferID");
        buffer.GetTemporaryRT(depthBufferID, -1, -1);
        buffer.SetRenderTarget(depthBufferID);
        buffer.ClearRenderTarget(true, true, Color.white);

        foreach(Renderer renderer in _oceanRenderers)
        {
            buffer.DrawRenderer(renderer, _depthMaterial);
        }

        buffer.SetGlobalTexture("_OceanDepthPrePass", depthBufferID);

        _camera.RemoveAllCommandBuffers();
        _camera.AddCommandBuffer(CameraEvent.BeforeDepthTexture, buffer);
    }

This is the game view and render texture result (this is a single mesh):

9768564--1399617--ocean_overdraw.jpg
9768564--1399617--ocean_overdraw.jpg1697×674 46.4 KB

My understanding is that the vertices are writing to the texture in the order that they are stored in the mesh. I guess when I draw to the render texture I’m not correctly z-testing each fragment. I could manually do that check in the fragment shader but I’m expecting to have issues with fragments writing to pixels at the same time. So how does unity get perfect z buffers? Is there a way for me to use the normal flow that Unity uses without having to manually depth test?

Thanks a million times in advance!

8

Hard to say. Your ZTest and ZWrite modes are correct… So the culprit may be in your setup of RT. It may not be getting a temporary RT that actually has a depth buffer. I’m not sure what -1, -1 is doing in your GetTemporaryRT call, but it doesn’t sound correct… You want to make sure you’re setting the depthBuffer paramter to 16 or 24 bits depending on the precision you want.

The way ZTest works is that it’s a fixed-function math operation that happens at a low level in the GPU, and is able to compare the current output value of your fragment to whatever was last written to that pixel, even during the same frame/pass-invocation. (And depending on API, it can even do Early-Z, where it discards your Fragment program instead of calculating it, if ZTest fails.)

The Blend and BlendOp shader tags work on a similar principal, but for the color buffer data.

9

Wow, yeah that was it! The -1, -1 are the screen size params and -1 tells it to use the current screen size but I wasn’t specifying the depth buffer size. I didn’t make the connection that when rendering to that texture it would be using that depth buffer for depth culling.

Can’t thank you enough!

1 Like
10

I honestly didn’t expect to ever get it to work but I’ve finally managed to put all the pieces together

9770499--1399977--ocean_custom_depth.png
9770499--1399977--ocean_custom_depth.png1097×281 210 KB

I had to do something a bit hacky with the depth comparison and I don’t yet understand why. My surface shader has this check:

if ((IN.clipPos.z / IN.screenPos.w) < (SAMPLE_DEPTH_TEXTURE(_OceanDepthPrePass, screenUV) - 0.00001))
{
    discard;
}

If I don’t push the depth texture back a tiny bit with the -0.00001 it looks like this:
9770499--1399980--upload_2024-4-14_21-49-55.png

As a hunch I guess it’s something to do with precision. I think I saw a related post about something similar in the past week or so, so I’ll try to find that. But for now I’m happy with the result as this is just a passion project to learn a bit about shaders.

Thanks for all your help!

11

No problem, glad it worked out!

The issue there might be because your original depth write is LEqual (<=) based, but the comparison you’re doing in the frag here is only <. How does it look if you do <= and remove that tiny offset?

12

Yeah I tried different variations of <, <=, but I just get slightly different variations of the patchwork effect seen above.

I might try unifying the depth calculations in the command buffer shader and the surface shader. When rendering the depth to a texture I just calculate clip pos in the vertex shader and then return i.vertex.z in the frag shader which is rendered to the depth buffer. But when doing the calculation in the surface shader I have to use IN.clipPos.z / IN.screenPos.w for some reason.

Depth pass shader:

Shader "Unlit/OceanDepthPass"
{
    SubShader
    {
        Tags
        {
            "RenderType" = "Opaque"
            "Queue" = "Geometry"
        }

        Pass
        {
            ZWrite On
            ZTest LEqual

            CGPROGRAM
            #pragma vertex vert
            #pragma fragment frag

            #include "UnityCG.cginc"
            #include "WaterWaveUtils.cginc"

            struct appdata
            {
                float4 vertex : POSITION;
                float4 normal : NORMAL;
            };

            struct v2f
            {
                float4 vertex : SV_POSITION;
            };

            v2f vert (appdata v)
            {
                v2f o;
                AddWaterWaves(v.vertex, v.normal.xyz);
                o.vertex = UnityObjectToClipPos(v.vertex);
                return o;
            }

            fixed4 frag (v2f i) : SV_Target
            {
                return i.vertex.z;
            }
            ENDCG
        }
    }
}

Surface shader:

Shader "Custom/Ocean"
{
    Properties
    {
        _WaterColour ("Water Colour", Color) = (0, 0, 0, 0)
        _WaterFogColourLow ("Water Fog Colour Low", Color) = (0, 0, 0, 0)
        _WaterFogColourHigh ("Water Fog Colour High", Color) = (0, 0, 0, 0)
        _WaterMaxHeight ("Water High Colour End", float) = 10
        _WaterFogDensity ("Water Fog Density", Range(0, 2)) = 0.1
        _Glossiness ("Smoothness", Range(0,1)) = 0.5
        _Metallic ("Metallic", Range(0,1)) = 0.0
        _AtmosphereColour ("Atmosphere Colour", Color) = (0, 0, 0, 0)
    }
    SubShader
    {
        Tags { "RenderType" = "Transparent" "Queue" = "Transparent"}

        Cull OFF
       
        GrabPass { "_WaterBackground" } 

        CGPROGRAM
        #include "WaterShaderUtils.cginc"
        #include "CustomFog.cginc"
        #include "WaterWaveUtils.cginc"

        #pragma surface surf Standard alpha vertex:vert finalcolor:ResetAlpha
        #pragma target 5.0

        struct Input
        {
            float2 uv_MainTex;
            float4 screenPos;
            float3 worldPos;
            float4 clipPos;
        };

        float4 _WaterColour;
        half _Glossiness;
        half _Metallic;
        fixed4 _Color;
        float4 _AtmosphereColour;
        float _FullAtmosphereDepth;

        sampler2D _OceanDepthPrePass;

        void vert (inout appdata_full v, out Input i)
        {
            AddWaterWaves(v.vertex, v.normal);
            UNITY_INITIALIZE_OUTPUT(Input, i);
            i.screenPos = ComputeGrabScreenPos(UnityObjectToClipPos(v.vertex));
            i.clipPos = UnityObjectToClipPos(v.vertex);
        }

        void surf (Input IN, inout SurfaceOutputStandard o)
        {
            float2 screenUV = IN.screenPos.xy / IN.screenPos.w;
            if ((IN.clipPos.z / IN.screenPos.w) <= (SAMPLE_DEPTH_TEXTURE(_OceanDepthPrePass, screenUV) - 0.00001))
            {
                discard;
            }

            o.Metallic = _Metallic;
            o.Smoothness = _Glossiness;
            o.Albedo = _WaterColour.rgb;
            o.Alpha = _WaterColour.a;

            float3 emissionCol = ColorBelowWater(IN.screenPos, IN.worldPos) * (1 - _WaterColour.a);
            o.Emission = emissionCol;
        }

        void ResetAlpha (Input IN, SurfaceOutputStandard o, inout fixed4 colour)
        {
            float distance = length(_WorldSpaceCameraPos - IN.worldPos);
            float depth = distance / _ProjectionParams.z;

            colour.rgb = lerp(colour.rgb, _AtmosphereColour, saturate(depth / _FullAtmosphereDepth));

            float3 fogColour = LerpToFog(colour.rgb, distance);

            colour.rgb = fogColour;
            colour.a = 1;
        }
        ENDCG
    }
}
13

If I change the command buffer’s depth buffer to be 16 bits instead of 24 I get a much more uniform issue which I should probably use anyway because then it matches the 16 bit precision of the floats I’m using in my shaders.

9771531--1400370--Screenshot 2024-04-15 105633.png
9771531--1400370--Screenshot 2024-04-15 105633.png953×347 431 KB