Drawing objects based only on layer order with no overdraw. [solved]

Unity Engine
,
1

Hi everybody, my question is a bit complicated, but I’ll do my best to be as clear as possible:

I’m trying to write a shader for a 2.5D game that fulfills the following 2 requirements:

  • Draw opaque objects in front of each other based solely on the order they are rendered in, as set by their order in layer (which basically happens by default if ZTest is Off & you’re rendering back to front like in the transparent queue).
  • Produce no overdraw.

My first idea for avoiding overdraw was using ZTest & rendering Front To Back. The problem is: Adding “ZTest Less” gets rid of the overdraw, but the shader no longer draws objects solely based on their rendering order. Objects with lower ZDepth will always be drawn in front of others, no matter their order in layer.

Does anybody have an idea how the 2 requirements could be made to work together in a shader? Basically I’m looking for a way to tell a shader if a pixel has been rendered before and if so (regardless of ZDepth) to skip rendering it again.

One more constraint:
There’s going to be >255 objects on screen using this shader, so I think the stencil buffer wouldn’t work as a solution.

Here’s what the top of my shader-code looks like so far:

SubShader
    {
        Tags
        {
            "Queue"="Geometry"
            "IgnoreProjector"="True"
            "RenderType"="Opaque"
        }


        Cull Back
        Lighting Off
        ZTest Less
        ZWrite On
        Blend Off

        Pass
        {}

Thanks for reading, if anyone has an idea you’d really help me out! :slight_smile:

2

The stencil buffer should actually be helpful here. And your idea of drawing front to back is right.

You should essentially determine the entire draw order and then execute it in reversed order to remove overdraw. You can use a single bit in the stencil buffer to mark where you have already drawn something to later skip those pixels.

So it’s close to what you already did, but only uses a single bit in the stencil buffer instead of the entire depth value in the depth buffer.

1 Like
3

Oh god I am such an idiot. Thank you so much, you’re totally right. For some reason I thought I’d have to increase the stencil buffer for each object and so it’s 255-integer wouldn’t have been enough, but of course that’s not the case, just need to increase by 1 when the pixel is rendered for the first time and that’s it.

Again, thank you, I’ve been trying to figure this out for quite some time! :smile:

4

Hi Friedemann_A, I know I am a bit late but I am having a problem with overdraw that is killing performance for my Android game. Can you share the whole shader code if possible?

1 Like
5

Sure, here’s the full code, not sure how helpful it is in your case though. To my understanding, this will only be relevant if you only render fully opaque objects. You might have to modify the code, most importantly the ZTest stuff. The Stencil Variables are set in the inspector and I’m setting them to:
Mask Mode = Greater
Stencil Pass Operation = IncrementSaturate
Stencil Ref Value = 1

Shader "2D Meshes/Animated"
{
    Properties
    {
        [Enum(CompareFunction)] _StencilComp("Mask Mode", Int) = 5
        [Enum(StencilOp)] _StencilOperation("Stencil Pass Operation", Int) = 6
        _StencilRefValue("Stencil Ref Value", Int) = 1
    }
    SubShader
    {
        Tags
        {
            "Queue"="Geometry"
            "IgnoreProjector"="True"
            "RenderType"="Opaque"
            "DisableBatching" = "True"
        }

        // No Lighting, cull backsides
        Cull Back
        Lighting Off
        // No ZTest, because we want to sort meshes by their
        // sortingOrder only. This sortingORder is assigned
        // according to the mesh's position in the Unity-Hierarchy
        // by CustomMeshEditor.cs
        ZTest Off
        // ZWrite on, so objects that use ZTest can still be occluded by this
        ZWrite On
        // No transparency
        Blend Off

        Pass
        {
            // Stencil is used to make the Meshes occlude each other.
            // Only the first one that is drawn at a certain position is shown.
            Stencil
            {
                Ref [_StencilRefValue]
                Comp [_StencilComp]
                Pass [_StencilOperation]
                Fail Keep
            }

            Name "GEOMETRY"

            CGPROGRAM
            #pragma vertex vert
            #pragma fragment frag
            #include "UnityCG.cginc"

            struct appdata
            {
                float4 vertex : POSITION;
                fixed4 color : COLOR;
            };

            struct v2f
            {
                fixed4 color : COLOR;
                float4 vertex : SV_POSITION;
            };

            v2f vert (appdata IN)
            {
                v2f o;
                o.vertex = UnityObjectToClipPos(animatedVertex);
                o.color = IN.color;
                return o;
            }
          
            fixed4 frag (v2f IN) : SV_Target
            {
                return IN.color;
            }
            ENDCG
        }
    }
}

Best of luck!