Difficulty using normal maps to transform vertices in vertex shader

Graphics Shaders & Materials
1

Hi, I’m attempting to use a normal map to displace vertices on the ZX plane (in object space). The reason I’m doing this is to eventually be able to pass in a dynamically generated normal map for effects like grass being pushed in a specific direction, or water ripples. Here is what I currently have:

Shader "Unlit/Grass"
{
    Properties
    {
        _MainTex ("Texture", 2D) = "white" {}
        _Cutoff("Alpha cutoff", Range(0,1)) = 0.5

        _ForceMap("Force Map", 2D) = "black" {}
    }
    SubShader
    {
        Tags { "RenderType"="Opaque" }
        LOD 100
        Cull Off

        Pass
        {
            CGPROGRAM
            #pragma vertex vert
            #pragma fragment frag
           
            #include "UnityCG.cginc"

            struct appdata
            {
                float4 vertex : POSITION;
                float2 uv : TEXCOORD0;
            };           

            struct v2f
            {
                float2 uv : TEXCOORD0;
                float4 local : TEXCOORD1;
                float4 vertex : SV_POSITION;
            };

            sampler2D _MainTex;
            float4 _MainTex_ST;

            sampler2D _ForceMap;
            float4 _ForceMap_ST;
           
            v2f vert (appdata v)
            {
                v2f o;               

                o.local = v.vertex;

                float2 coords = float2(v.vertex.x, v.vertex.z) / 10 + 0.5;
                float3 normal = UnpackNormal(tex2Dlod(_ForceMap, float4(coords, 0, 0)));
                normal.y = 0;

                float4 transformed = v.vertex;
                transformed.xyz += normal;

                o.vertex = UnityObjectToClipPos(transformed);
                o.uv = TRANSFORM_TEX(v.uv, _MainTex);
                return o;
            }

            float _Cutoff;
           
            fixed4 frag (v2f i) : SV_Target
            {
                // sample the texture
                fixed4 col = tex2D(_MainTex, i.uv);
                clip(col.a - _Cutoff);

                float2 coords = float2(i.local.x, i.local.z) / 10 + 0.5;
                fixed4 normalColor = tex2D(_ForceMap, float4(coords, 0, 0));

                return fixed4(coords.x, coords.y, 0, 0);
            }
            ENDCG
        }
    }
}

I transform the vertex’s ZX position into UV coords by first dividing by the mesh’s bounds (I am using the default Unity plane to test, and it is 10x10) and then adding 0.5, as Unity’s UV coords are in the range of 0-1 (I think), and the mesh is centered around the 0 (being -5 to 5). I then sample the Force map texture at these coords and unpack them into a normal. Here is the normal map I’m using to test:

My assumption is that for tangent space normals, the pastel blue colors are straight up, while the red, dark blue and greens would be pointing towards the edges of the texture. I don’t use normal maps much, so please correct any assumptions of mine that are not on the mark!

So I zero out the Y of the normal, add the normal to the vertex and then transform it. My expectation would be to have the mesh stay centered around the 0, the vertices being pushed out into a circle (and the verts on the edge unchanged).

716×437 112 KB

It’s kind of there, but two main issues: it’s sort of a messy circle, which I don’t entirely understand. As well, the entire plane is translated 1 unit up the z axis (I have a copy of the plane with the default material on it below for comparison), which again I’m confused about.

Side note: Does anyone have any general advice on how to debug shaders properly? Outside of coloring normals and UVs, I can’t seem to find any handy tools like Debug.Log or stepping the shader.

Thanks for any help,
Erik

2

Figured it out! When you unpack a normal, the tangent normal is X/Y for left/up, and Z for coming out of the screen…which seems pretty obvious in retrospect, but I guess I was super in Unity mode and didn’t think enough about the coordinates system.