Tessellation not writing depth properly when using custom UVs

I’ve used tessellation a lot recently, and had no real issues with it until today.

What am I doing: I am passing a rendertexture to my terrain material with footsteps on it. The rt displaces the tessellated mesh in the shader. The tessellation is working fine in all cases. The difference is that depth writing is behaving strangely.

  • When using a mesh UV (v.texcoord), tessellation works fine, and depth is correctly written.
  • When using a custom UV, tessellation works fine BUT depth is not written correctly?!

Example 1
footsteps writing depth properly in all these cases.

tex2Dlod (_FootstepTex, float4(v.texcoord,0,0)).r
tex2Dlod (_FootstepTex, float4(v.texcoord*5,0,0)).r
tex2Dlod (_FootstepTex, float4(v.texcoord*float(2,3),0,0)).r

Example 2
footsteps not writing depth… but the other tessellation heightmaps ARE all still working normally.

tex2Dlod (_FootstepTex, float4(screenUV.xy/screenUV.w,0,0)).r

The reason I want to use ClipSpace UV in the second example, is so that I can use a second camera to capture the footsteps in a render texture.

Things that I have tried without making a difference:

  • Every variation of render texture format.
  • Switching camera forward/deferred on both the RTcamera and the MainCamera
  • Vert Frag and Surface Shader for the tessellated mesh all behave the same.
  • Passing render texture via script (shader.setglobaltexture)
  • Changing rendertexture to texture2D (using get/set pixels)

If anyone can help point me in the direction of a solution, please let me know. Here is the code for my most recent shader.

// Dave Lindsay

Shader "Opaque_Tess_Feet" {
    Properties {

        _DistMin ("Distance, Min", Float) = 10
        _DistMax ("Distance, Max", Float) = 25
        _Tess ("Tessellation", Range(1,64)) = 4
        _Displacement ("Displacement", Range(0, 1.0)) = 0.3
        _FootstepTex ("Footstep Render Texture", 2D) = "white" {}
        _Tiling1 ("R: Tiling", Float) = 1
        _HeightTex1 ("R: Height", 2D) = "white" {}
    }
    SubShader {
        Tags { "RenderType"="Opaque" "Queue"="Geometry" }
      
        CGPROGRAM
        #pragma surface surf Standard addshadow fullforwardshadows vertex:disp tessellate:tessDistance
        #pragma target 5.0
        #include "Tessellation.cginc"
        #include "UnityCG.cginc"

        struct vertIn {
            float4 vertex : POSITION;
            float4 tangent : TANGENT;
            float3 normal : NORMAL;
            float4 color : COLOR;
            float2 texcoord : TEXCOORD0;
            float2 texcoord1 : TEXCOORD1;
            float2 texcoord2 : TEXCOORD2;
            float2 texcoord3 : TEXCOORD3;
        };

        float _Tess;
        float _DistMin;
        float _DistMax;
        float _Displacement;

        sampler2D _FootstepTex;

        float _Tiling1;
        sampler2D _HeightTex1;

        float4 tessDistance (vertIn v0, vertIn v1, vertIn v2) {
            return UnityDistanceBasedTess(v0.vertex, v1.vertex, v2.vertex, _DistMin, _DistMax, _Tess);
        }

        void disp (inout vertIn v)
        {
            float4 screenUV = ComputeScreenPos(mul(UNITY_MATRIX_MVP,v.vertex));
            screenUV.xy /= screenUV.w;
            float footsteptex = tex2Dlod (_FootstepTex, float4(screenUV.xy,0,0)).b;
            float height = tex2Dlod (_HeightTex1, float4(v.texcoord.xy,0,0)).r - footsteptex;
            v.vertex.xyz += v.normal * height * _Displacement;
        }

        struct Input {
            float2 uv_HeightTex1;
        };

        void surf (Input IN, inout SurfaceOutputStandard o) {
            o.Albedo = float3(0.5,0.5,0.5);
        }
        ENDCG
    }
    FallBack "Diffuse"
}

UPDATE: Could be related to a slight error on the screenUV calculation. It seems to be off by just a fraction, but I’m not sure why.

It seems that the shadow collection is where things are going wrong.

It’s probably the shadow map itself, rather than the “shadow collection” (it’s the exact same shader pass used for both). When rendering the shadow maps the UNITY_MATRIX_MVP is going to be from the point of view of the light and not the final camera. I think the only solution is you have to calculate the camera’s UNITY_MATRIX_MVP, or at least the UNITY_MATRIX_VP, and pass it as a shader variable.

However you should be aware the UNITY_MATRIX_VP isn’t as simple as just multiplying the Camera.worldToCameraMatrix and Camera.projectionMatrix together. See:

Also, that’s somewhat out of date as I believe Unity - Scripting API: GL.GetGPUProjectionMatrix can be used now to get the “real” projection matrix.

Thanks for your advice -yes, I’m quite sure it’s the shadow map, and it makes total sense as the the matrix does indeed seem to grab the light direction instead of camera direction.

I will try that out and then update this thread!