UpdateDepthTexture breaks per instance matrixes with Vertex Displacement + GPU instancing

Here’s my setup:
I have some grass, and some trees. It sways with the wind by animating vertex positions.
I’m about to start generating a bunch of grass, and would like to get instancing working. The shader works perfectly fine without instancing.
When I turn it on, I have two issues:

  • I can see the shadows of object behind the grass sometimes.

  • When in front of my water, which reads from depth texture to determine color and foam, I can see that animated vertex positions for my grass differ between the depth texture write and the rest of the shader.

If I go into the frame debugger, I can clearly see that the grass animates the exact same way for all my meshes, during the depth pass. But in the actual drawing of the object, the animation is correct.

I guess the GPU instancing breaks the per instance world matrixes for the depth pass, resulting in different outputs.

I’ve been at this for some hours now and can’t find a solution. Anybody out there with some ideas?
Here’s the shader code. Note that the lighting function and wind function are in cgincludes. The wind function is in the bottom.

    SubShader
    {
        Pass
        {
            AlphaToMask On
            Cull[_Cull]
        
            Tags
            {
                "RenderType"="TransparentCutout"
                "LightMode"="ForwardBase"
                "IgnoreProjector"="True"
                "Queue"="AlphaTest"
            }

            CGPROGRAM

            #include "Assets/Shaders/Fog/Fog.cginc"
            #include "Assets/Shaders/Toon/ToonVegetation.cginc"
            #include "Assets/Shaders/Wind.cginc"
            #include "AutoLight.cginc"
            #include "UnityCG.cginc"
        
            #pragma vertex vert
            #pragma fragment frag
        
            #pragma shader_feature FOG_NOISE
            #pragma shader_feature SSS
            #pragma shader_feature RIM
            #pragma shader_feature_local WIND
            #pragma shader_feature_local WIND_UV

            #pragma multi_compile_fwdbase
            #pragma multi_compile_instancing

            sampler2D _MainTex;
            float4 _MainTex_ST;
            fixed _Cutoff;

            struct appdata
            {
                float4 vertex : POSITION;
                float3 normal : NORMAL;
                float2 uv : TEXCOORD0;
                UNITY_VERTEX_INPUT_INSTANCE_ID
            };
        
            struct v2f
            {
                float4 pos : SV_POSITION;
                float2 uv : TEXCOORD0;
                float3 worldPos : TEXCOORD1;
                float3 viewDir : TEXCOORD2;
                SHADOW_COORDS(3)
                float3 normal : NORMAL;
                float4 color : COLOR;
                UNITY_VERTEX_INPUT_INSTANCE_ID
            };

            inline float3 positionToNormal(float3 position)
            {
                float3 dpx = ddx( position );
                float3 dpy = ddy( position ) * _ProjectionParams.x;
                return normalize(cross(dpx, dpy));
            }
        
            float _WindScale;

            v2f vert (appdata v)
            {
                v2f o;
                UNITY_SETUP_INSTANCE_ID(v);
                UNITY_TRANSFER_INSTANCE_ID(v, o);
                float4 vert = v.vertex;
#if WIND
    #if WIND_UV
                vert = WindPosition(vert, _WindScale * step(0.1, v.uv.y));
    #else
                vert = WindPosition(vert, _WindScale);
    #endif
#endif

                o.pos = UnityObjectToClipPos(vert);
                o.worldPos = mul(unity_ObjectToWorld, vert);
                o.normal = mul(unity_ObjectToWorld, v.normal);
                o.uv = TRANSFORM_TEX(v.uv, _MainTex);
                o.color = lerp(UNITY_ACCESS_INSTANCED_PROP(Props, _BottomColor), UNITY_ACCESS_INSTANCED_PROP(Props, _TopColor), saturate(vert.y));
                o.viewDir = WorldSpaceViewDir(vert);
                TRANSFER_SHADOW(o);
                return o;
            }

            half4 frag(v2f i) : SV_Target
            {
                UNITY_SETUP_INSTANCE_ID(i);
                half4 col = i.color * tex2D(_MainTex, i.uv);
            
                float3 origNormal = positionToNormal(i.worldPos);
                float viewFactor = abs(dot(origNormal, -normalize(i.viewDir)));

                col.a = (col.a - _Cutoff) / max(fwidth(col.a), 0.0001) + 0.5;
                col.a *= viewFactor;

                col = LightingToonVegetation(col, i.normal, _WorldSpaceLightPos0.xyz, i.viewDir, SHADOW_ATTENUATION(i));
                col = ApplyFogAlphaBlend(i.worldPos, i.pos.w, col);

                return col;
            }

            ENDCG
        }

        Pass
        {
            Cull[_Cull]
            AlphaToMask On

            Tags { "LightMode" = "ShadowCaster"}
        
            CGPROGRAM

            #include "Assets/Shaders/Wind.cginc"
            #include "UnityCG.cginc"

            #pragma shader_feature_local WIND
            #pragma shader_feature_local WIND_UV
        
            #pragma vertex vert
            #pragma fragment frag

            #pragma multi_compile_shadowcaster
            #pragma multi_compile_instancing

            float _Cutoff;
            sampler2D _MainTex;
            float4 _MainTex_ST;

            struct appdata
            {
                float4 vertex : POSITION;
                float2 uv : TEXCOORD0;
                UNITY_VERTEX_INPUT_INSTANCE_ID
            };
        
            struct v2f
            {
                V2F_SHADOW_CASTER;
                UNITY_VERTEX_INPUT_INSTANCE_ID
                float3 worldPos : TEXCOORD2;
                float2 uv : TEXCOORD1;
                float3 viewDir : TEXCOORD03;
            };

            inline float3 positionToNormal(float3 position)
            {
                float3 dpx = ddx( position );
                float3 dpy = ddy( position ) * _ProjectionParams.x;
                return normalize(cross(dpx, dpy));
            }

            float _WindScale;

            v2f vert(appdata v)
            {
                v2f o;
                UNITY_SETUP_INSTANCE_ID(v);
                UNITY_TRANSFER_INSTANCE_ID(v, o);
                float4 vert = v.vertex;
#if WIND
    #if WIND_UV
                vert = WindPosition(vert, _WindScale * step(0.1, v.uv.y));
    #else
                vert = WindPosition(vert, _WindScale);
    #endif
#endif
            
                o.viewDir = WorldSpaceViewDir(v.vertex);
                o.worldPos = mul(unity_ObjectToWorld, vert.xyz);
                o.pos = UnityObjectToClipPos(vert);
                o.uv = TRANSFORM_TEX(v.uv, _MainTex);
                return o;
            }
        
            half4 frag(v2f i) : SV_Target
            {
                UNITY_SETUP_INSTANCE_ID(i);
                half4 col = tex2D(_MainTex, i.uv);

                float3 origNormal = positionToNormal(i.worldPos);
                float viewFactor = abs(dot(origNormal, -normalize(i.viewDir)));
            
                col.a = (col.a - _Cutoff) / max(fwidth(col.a), 0.0001) + 0.5;
                col.a *= viewFactor;
                return col;
            }

            ENDCG
        }
    }

Wind function:

inline float4 WindPosition(float4 vertex, half strength)
{
    float3 worldPos = mul(unity_ObjectToWorld, float4(vertex.xyz, 1)).xyz;
    float noise = tex2Dlod(_WindNoise, float4(( worldPos.xz * _WindFrequency - _WindDirection.xz * _WindSpeed * _Time.x)  ,0,0));
    float3 wind = noise * _WindDirection * strength * _WindStrength;
    return float4(mul(unity_WorldToObject, float4(worldPos + wind, 1)).xyz, vertex.w);
}

Thought I’d clarify a bit with a GIF.

You can see that the vertex displacement in depth for each grass object is the exact same (The bright outlines on the water), while the actual renders have the correct world space based vertex displacement.

5879966--626222--GrassDepth.gif

What I’ve tried so far:

  • Fallback Diffuse / Cutout, and no fallback (Shouldn’t matter since I have my own shadowcaster pass?)
  • Different rendertype tags in the shadowcaster pass
  • Different render queues (Geom/AlphaTest)
  • Transfering InstanceID to fragment shader and setting up the ID there was well, in shadowcaster pass
  • Turning AlphaToMask off for shadowcaster pass, and just clipping instead
  • Double checked that Unity is not somehow batching the meshes. Dynamic Batching is off, and DisableBatching tag added. Still no change.

So, after many hours spent I got it working.
I decided to just keep working on the “mass drawing” part and see if the problem might solve itself.
What I had to do was use Graphics.DrawMeshInstancedIndirect.(Graphics.DrawMeshInstanced had the same problem!). Now the depth values are correct, since I’m providing my own ObjectToWorld and WorldToObject matrixes.
I guess the benefit is that I have the flexibility of adding compute shader before rendering, and this would work for any object I need to mass spawn, not just grass.

I would still however, like to get the other approach working, so if someone has ideas, please let me know!
It’s strange to me that the depth pass wouldn’t support the world matrixes properly when just doing scene placed meshes and turning on instancing.
If I can’t get that working, and must always use DrawMeshInstancedIndirect, it basically means that I have to also write something that takes in transforms and makes a computebuffer from those, and apply that to every single tree-crown that I manually place in the world. (or some other placement tool), which breaks the normal unity workflow.

Here is a couple gifs with ~20.000 grass instances. Now I just need to disallow grass spawning on the water covered part of the mesh :slight_smile:

5886992--627359--GrassInstancedIndirect.gif 5886992--627362--GrassInstancedIndirect.gif

1 Like