Making a FP shader - weird z-fighting issues, unsolved by traditional methods

Unity Engine
,
1

Hi everyone,
So after being contacted on Reddit about it I’ve decided to delve again in my attempt to give Unity a proper viewmodel shader. I’m planning on building it for URP afterward, but for the moment I’m trying to make it work in built-in. Not gonna bother with HDRP as I have no interest in this pipeline and I feel like URP gives so much more freedom overall, I sincerely hope URP is the future for Unity, with a bit of work it could become the best tool for fully customizable graphics, as evidenced by my recent attempt at a Toon shader with more controls over lighting. Anyway on to the topic at hand.

Ideally, for this shader to be interesting for FPS devs, I’d want to be able to customize the FOV of the viewmodel (player’s weapon). Problem is, Unity gives you no choice but to execute an ObjectToClipPos with the surface vertex shader output data. As has been mentioned in the long past in this forum, the best way to go about this is to inverse what ObjectToClipPos does to the vertex data, so its action is nullified.

As such my current workflow has a script sending a global containing a perspective projection Matrix calculated using Matrix4x4.Perspective. To nullify Unity’s MVP matrix I’ve tried 2 different solutions :

v.vertex = mul(customMVP, mul(unity_WorldToObject, mul(_invViewProjMatrix, v.vertex)));

where _invViewProjMatrix is the inverse of the view projection matrix, as calculated in the same script that sends my custom projection matrix, then sent as a global.

 v.vertex = mul(customMVP, mul(transpose(UNITY_MATRIX_IT_MV), mul(unity_CameraInvProjection, v.vertex)));

Which as far as I understand should first nullify projection and then model and view.

Unfortunately, neither of these worked. I even tried replacing customMVP with UNITY_MATRIX_MVP, which should result in a completely unchanged visual result, but it does change. The model appears flattened, and when rotating in editor, moves in weird patterns.

This is the last roadblock for me in this endeavour and it’s really frustrating, particularly as it feels like it should work, using, as far as I know, the correct inverse of all the matrix that I need.

Thanks in advance to anyone who can help me with this and have a great day everyone :slight_smile:

Edit : This hasn’t been solved BUT I realized a big mistake from my part which is that I did the operations on the entire vertices. Now the model properly follows the camera, and actually sees some change when changing my FOV value. However, even using Unity’s MVP matrix, I’m getting a vertically (Y axis relative to the camera) squished model. Gonna inspect that some more but I don’t see where that could come from. For the record, I changed all v.vertex to v.vertex.xyz.

2

Ok so I’ve worked a bit on that and I’ve solved most of my matrices issues if not all ! Unfortunately I’m now getting into a Z-fighting issue that I can’t quite solve. My shader now looks like that :

Shader "Custom/FPViewShader"
{
    Properties
    {
        _Color ("Color", Color) = (1,1,1,1)
        _MainTex ("Albedo", 2D) = "white" {}
        _Normals ("Normals", 2D) = "blue" {}
        _GlossTex ("Gloss", 2D) = "gray" {}
        _MetalTex ("Metal", 2D) = "black" {}
        _OcclusionTex ("Occlusion", 2D) = "white" {}
    }

    SubShader
    {
        Pass {
            Tags { "Queue" = "Geometry+1" }
            ColorMask 0
            ZTest Always
            ZWrite On

            CGPROGRAM
                #pragma vertex vert
                #pragma fragment frag

                #include "UnityCG.cginc"

                float4x4 _viewmodelProjMatrix;

                struct v2f {
                    float4 pos          : POSITION;
                    float4 screenPos    : TEXCOORD0;
                };

                v2f vert (appdata_full v)
                {
                    v2f o;

                    float4x4 newMVP = mul(_viewmodelProjMatrix, UNITY_MATRIX_MV);
                    v.vertex = mul(newMVP, v.vertex);
                    //v.vertex = UnityObjectToClipPos(v.vertex);

                    o.pos=v.vertex;
                    o.screenPos = ComputeScreenPos(o.pos);

                    return o;
                }

                fixed4 frag(v2f i) : SV_TARGET
                {
                    return 0;
                }
            ENDCG
        }
     
        Tags { "Queue" = "Geometry+2" "RenderType" = "Opaque" }

        CGPROGRAM
            // Physically based Standard lighting model, and enable shadows on all light types
            #pragma surface surf Standard vertex:vert addshadow

            // Use shader model 3.0 target, to get nicer looking lighting
            #pragma target 3.0

            sampler2D _MainTex;
            sampler2D _Normals;
            sampler2D _GlossTex;
            sampler2D _MetalTex;
            sampler2D _OcclusionTex;

            struct Input
            {
                float3 worldNormal; INTERNAL_DATA
                float2 uv_Normals;

                float2 uv_MainTex;
                float2 uv_GlossTex;
                float2 uv_MetalTex;
                float2 uv_OcclusionTex;
            };

            fixed4 _Color;
            float4x4 _viewmodelProjMatrix;
            float4x4 _invProjMatrix;

            UNITY_INSTANCING_BUFFER_START(Props)
            UNITY_INSTANCING_BUFFER_END(Props)

            void vert(inout appdata_full v) {
                float4x4 newMVP = mul(_viewmodelProjMatrix, UNITY_MATRIX_MV);
                float4x4 MVP_I = mul(transpose(UNITY_MATRIX_IT_MV), _invProjMatrix);
                //v.vertex = UnityObjectToClipPos(v.vertex);
                v.vertex = mul(newMVP, v.vertex);
                v.vertex = mul(MVP_I, v.vertex);
            }

            void surf(Input IN, inout SurfaceOutputStandard o)
            {
                // Albedo comes from a texture tinted by color
                fixed4 c = tex2D (_MainTex, IN.uv_MainTex) * _Color;
                fixed4 s = tex2D (_GlossTex, IN.uv_GlossTex);
                fixed4 m = tex2D (_MetalTex, IN.uv_MetalTex);
                fixed4 oc = tex2D (_OcclusionTex, IN.uv_OcclusionTex);

                half3 n = UnpackNormal( tex2D(_Normals, IN.uv_Normals) );

                o.Albedo = c.rgb;
                o.Smoothness = m.a;
                o.Metallic = m.rgb;
                o.Occlusion = oc.rgb;
                o.Alpha = c.a;

                float3 worldInterpolatedNormalVector = WorldNormalVector (IN, float3( 0, 0, 1 ));
                o.Normal = n;
                //o.Normal = normalize(o.Normal);
            }
        ENDCG
    }
}

Ref :
-float4x4 _viewmodelProjMatrix : is my custom provided projection matrix for the viewmodel object. It’s calculated by asking the camera to change fov, getting its new matrix, then setting the fov back (done each update for debugging purposes, but would only be done when changing the setting further down the line)
-float4x4 _invProjMatrix : is the inverse of the camera’s projection matrix as calculated in-script

both of which are scaled in the Y axis by -1 as expected by Unity then passed through the GL.GetGPUProjectionMatrix method to get proper results.

I’ve also tried separating the passes and applying them both to the same object, but the results are the same (which tells me my pass separation seems to work, as well as the render queues selected). Unfortunately the result is this on video :

Clearly this looks like good old proper z-fighting. I’ve tried using Offset as this seems to be the best way to fight it, and while it definitely makes the issue better, it still pops up. Actually the second part of the video shows the use of an offset bringing vertices closer to the camera for the actual render, and farther away for the Z-Writing invisible model (I’m using Offset 1, 1 and Offset -1, -1 respectively and I’ve tried multiple values for the first number to play around).

Now the particularly weird thing you can notice, is that this z-fighting is completely gone when I look at no models (hence the skybox). At this point I expected to see the “Microsoft Solitaire” effect we usually get when moving a camera around areas that have models but also part that are not “cleared” (the weird ghost/echo effect). However we see the model as expected. I’ve tested the same with the free “AKM” model to see if it was pure luck from the model being unicolor, or if it would work regardless with textures, and it works. So somehow the Z-Fighting is solved when not looking at other models, as if it wasn’t those models z-fighting.

For “science” I did test setting ColorMask to RGB and RGBA for the first part of the shader, and when doing that, the z-fighting is somehow still going on even when looking at the skybox, so clearly it’s only my 2 passes z-fighting each other but I have no idea why…

Does anyone have any idea ?

Thanks in advance :slight_smile: