Issue calculating tangent space normals for procedural planet shader

I’m working on a procedural planet generator based on fractal 3D noise. I was previously using Godot and had a working shader in GLSL but am moving over to Unity so I have been porting my shader code over to HLSL. I basically got it working but am having issues calculating the tangent space normals from my 3D noise function. I must be making an incorrect assumption somewhere since the shadows in my model are not consistent with the light direction.

The attached image below shows the odd behavior. Some parts of the sphere have correct light but I am noticing a weird “pole” effect at other parts of the sphere. In the scene I have a single directional light shining directly down on the sphere.

I know my approach to calculating the surface normals is correct since it worked fine in Godot, so I’m thinking there’s some issue with the transform to tangent space. I did the spherical to Cartesian coordinate transform derivations by hand to make sure my angles made sense. I am calculating the tangent and bi-tangent vectors in object space by taking the partials of the Cartesian coordinates with respect to theta and phi. After I calculate my surface normals (in object space) from my noise map, I project the normal onto each axis of the tangent vectors in object space to get the normal in tangent space.

Also, I’m open to any feedback/suggestions. I haven’t use surface shaders before so if there is a more efficient way of doing things I would be happy to learn!

// AUTHOR:         Dan Greenheck
// DESCRIPTION:    Procedural planet generator
// DATE CREATED:   2020-07-25
//
// --------------------------------------------------------------------------------

Shader "Custom/Planet" {
    Properties {
        _OceanColor ("Ocean Color", Color) = (0.08, 0.25, 0.41, 1.0)
        _BeachColor ("Beach Color", Color) = (0.89, 0.80, 0.61, 1.0)
        _PlainsColor ("Plains Color", Color) = (0.17, 0.29, 0.16, 1.0)
        _TreeColor ("Tree Color", Color) = (0.10, 0.20, 0.10, 1.0)
        _MountainColor ("Mountain Color", Color) = (0.75, 0.75, 0.75, 1.0)
        _BumpStrength ("Bump Strength", Range(0.01, 1.0)) = 0.5
        _Scale ("Scale", Range(0.0, 2.0)) = 1.06
        _Period ("Period", Range(0.01, 0.5)) = 0.339
        _Octaves ("Octaves", Range(1, 10)) = 5
        _Lacunarity ("Lacunarity", Range(0.1, 10.0)) = 2.447
        _Persistence ("Persistence", Range(0.0, 1.0)) = 0.493
        _MainTex ("Albedo (RGB)", 2D) = "white" {}
    }

    SubShader {
        Tags { "RenderType"="Opaque" }
        LOD 200

        CGPROGRAM
        // Physically based Standard lighting model, and enable shadows on all light types
        #pragma surface surf Standard vertex:vert

        // Use shader model 3.0 target, to get nicer looking lighting
        #pragma target 3.0

        #include "UnityCG.cginc"

        // Magnitude of offset used for calculating tangent vectors
        static const float delta = 0.0001;

        // Thresholds controlling transitions between each terrain color
        static const float th_beach_min = 0.53;
        static const float th_beach_max = 0.55;
        static const float th_plains_min = 0.50;
        static const float th_plains_max = 0.60;
        static const float th_trees_min = 0.50;
        static const float th_trees_max = 0.62;
        static const float th_mountain_min = 0.69;
        static const float th_mountain_max = 0.75;

        // struct SurfaceOutputStandard {
        //     fixed3 Albedo;      // base (diffuse or specular) color
        //     fixed3 Normal;      // tangent space normal, if written
        //     half3 Emission;
        //     half Metallic;      // 0=non-metal, 1=metal
        //     half Smoothness;    // 0=rough, 1=smooth
        //     half Occlusion;     // occlusion (default 1)
        //     fixed Alpha;        // alpha for transparencies
        // };

        struct Input {
            float2 uv_MainTex;
            float3 pos;
        };

        fixed4 _OceanColor;
        fixed4 _BeachColor;
        fixed4 _PlainsColor;
        fixed4 _TreeColor;
        fixed4 _MountainColor;
        float _BumpStrength;
        float _Scale;
        float _Period;
        int _Octaves;
        float _Lacunarity;
        float _Persistence;

        // Author: Inigo Quilez
        // https://www.shadertoy.com/view/Xsl3Dl
        float3 hash(float3 p) {
            p = float3(dot(p, float3(127.1, 311.7, 74.7)),
                    dot(p, float3(269.5, 183.3, 246.1)),
                    dot(p, float3(113.5, 271.9, 124.6)));

            return -1.0 + 2.0 * frac(sin(p) * 43758.5453123);
        }

        // Author: Inigo Quilez
        // https://www.shadertoy.com/view/Xsl3Dl
        float noise(float3 p) {
            float3 i = floor(p);
            float3 f = frac(p);
            float3 u = f * f * (3.0 - 2.0 * f);
           
            float n = lerp(lerp(lerp(dot(hash(i + float3(0.0, 0.0, 0.0)), f - float3(0.0, 0.0, 0.0)),
                                dot(hash(i + float3(1.0, 0.0, 0.0)), f - float3(1.0, 0.0, 0.0)), u.x),
                            lerp(dot(hash(i + float3(0.0, 1.0, 0.0)), f - float3(0.0, 1.0, 0.0)),
                                dot(hash(i + float3(1.0, 1.0, 0.0)), f - float3(1.0, 1.0, 0.0)), u.x), u.y),
                        lerp(lerp(dot(hash(i + float3(0.0, 0.0, 1.0)), f - float3(0.0, 0.0, 1.0)),
                                dot(hash(i + float3(1.0, 0.0, 1.0)), f - float3(1.0, 0.0, 1.0)), u.x),
                            lerp(dot(hash(i + float3(0.0, 1.0, 1.0)), f - float3(0.0, 1.0, 1.0)),
                                dot(hash(i + float3(1.0, 1.0, 1.0)), f - float3(1.0, 1.0, 1.0)), u.x), u.y), u.z );
                           
            // Normalize to [0.0, 1.0]
            return 0.5*(n + 1.0);
        }

        // Converts spherical coordinates to Cartesian coordinates
        float3 sph2cart(float phi, float theta) {
            float3 unit = float3(0.0, 0.0, 0.0);
            unit.x = cos(phi) * cos(theta);
            unit.y = sin(phi) * cos(theta);
            unit.z = sin(theta);
            return normalize(unit);
        }

        // Calculates tangent vector for unit sphere
        float3 sph2tan(float phi) {
            float3 tangent = float3(0.0, 0.0, 0.0);
            tangent.x = -sin(phi);
            tangent.y = cos(phi);
            return normalize(tangent);
        }

        void vert (inout appdata_full v, out Input o) {
            UNITY_INITIALIZE_OUTPUT(Input,o);
            o.pos = normalize(v.normal);
        }

        void surf (Input IN, inout SurfaceOutputStandard o)  {     
           
            // Convert Cartesian position to spherical coordinates
            float phi = atan2(IN.pos.y, IN.pos.x);
            float theta = atan2(IN.pos.z, sqrt(IN.pos.x*IN.pos.x + IN.pos.y*IN.pos.y));
           
            // Normal vector
            float3 N = normalize(IN.pos);

            // Perturb the normal vector in the theta/phi directions. The 3D noise wil
            // be sampled at these vectors to later numerically calculate the surface normal
            float3 N_dx = sph2cart(phi + delta, theta);
            float3 N_dy = sph2cart(phi, theta + delta);
           
            // Get the tangent and bi-tangent vectors
            float3 T = sph2tan(phi);
            float3 B = cross(N, T);

            // Terrain height, sampled at N, N_dx, N_dy respectively
            float h = 0.0;
            float h_dx = 0.0;
            float h_dy = 0.0;
           
            // ----------- 3D fractal noise calculations ----------

            float a = 1.0; // Amplitude for current octave
            float max_amp = a; // Accumulate max amplitude so we can normalize after
            float p = _Period;  // Period for current octave
            for(int i = 0; i < _Octaves; i++) {
                // Sample the 3D noise
                h += a*noise(N/p);
                h_dx += a*noise(N_dx/p);
                h_dy += a*noise(N_dy/p);
               
                // Amplitude decay for higher octaves
                a *= _Persistence;
                max_amp += a;
                // Divide period by lacunarity
                p = p / _Lacunarity;
            }
           
            // --------------------------------------------------
           
            // Scale heights between 0.0 and 1.0 and then scale to
            // adjust land/water distribution
            h /= float(max_amp) / _Scale;
            h_dx /= float(max_amp) / _Scale;
            h_dy /= float(max_amp) / _Scale;
           
            // Exaggerate bump strength for mountains
            float bump_strength_scaled = _BumpStrength;
            if (h > th_mountain_min) {
                bump_strength_scaled *= 3.0;
            }

            // Threshold values for transitioning from each terrain type. Use
            // smoothstep to control transitions between terrain types. Large differences
            // between min/max values will result in smoother transitions.
            float th_ocean2beach = smoothstep(th_beach_min, th_beach_max, h);
            float th_beach2plains = smoothstep(th_plains_min, th_plains_max, h);
            float th_plains2trees = smoothstep(th_trees_min, th_trees_max, h);
            float th_trees2mountains = smoothstep(th_mountain_min, th_mountain_max, h);

           
            // Scale unit vectors on sphere by the noise multiplied by bump strength
            float3 R = N*(1.0 + bump_strength_scaled*h);
            float3 R_dx = N_dx*(1.0 + bump_strength_scaled*h_dx);
            float3 R_dy = N_dy*(1.0 + bump_strength_scaled*h_dy);

            // Calculate the object-space normal by taking differences between the
            // normal vector and the perturbed normal vectors
            float3 normal_obj = normalize(cross(R_dx - R, R_dy - R));
            // Interpolate between the flat surface normal and the terrain normal
            // This will cause the ocean areas to be shaded as a flat sphere.
            normal_obj = lerp(N, normal_obj, th_ocean2beach);
           
            // Project the normal defined in object space to tangent space
            o.Normal = normalize(float3(dot(normal_obj,T),
                                        dot(normal_obj,B),
                                        dot(normal_obj,N)));

            // ---------------- ALBEDO ----------------------------

            // lerp between the ocean color and the land color
            float3 color = lerp(_OceanColor.rgb, _BeachColor.rgb, th_ocean2beach);
            color = lerp(color.rgb, _PlainsColor.rgb, th_beach2plains);
            color = lerp(color.rgb, _TreeColor.rgb, th_plains2trees);
            color = lerp(color.rgb, _MountainColor.rgb, th_trees2mountains);
           
            o.Albedo = color.rgb;
            o.Smoothness = 1.0 -lerp(0.35, 1.0, th_ocean2beach);
        }
        ENDCG
    }
    FallBack "Diffuse"
}

Unity’s Surface Shaders assume any value set on the o.Normal is in the same space as the mesh’s vertex tangents. My guess is your sphere mesh’s tangents don’t match those you’re calculating.

Also, if your mesh does already have per vertex tangent data, there’s little need to calculate it manually in the shader afterwards. In fact you really shouldn’t if you can avoid it as it’s more important for the tangent space to match than it is for it to be correct.

I posted an example of how to convert a world space vector to a tangent space vector (approximately) here:

See the WorldToTangentNormalVector function on line 63. You’ll need the float3 worldNormal; and INTERNAL_DATA in the Input struct for this to work. You’ll also need to convert your object space normals into world space before using that function, but that’s a simple enough problem to solve.

// inverse transpose object to world
float normal_world = mul(normal_obj, (float3x3)unity_WorldToObject);
o.Normal = WorldToTangentNormalVector(normal_world);

The normal, tangent, and bitangent the Surface Shader has access to is by default in world space, hence the need to convert from object to world space. You could pass the vertex tangent from the vert function to avoid that, but it’s probably not a lot faster.

It should be noted that using an transposed matrix as an inverse, as the WorldToTangentNormalVector function and what your code is doing is only an approximation, but it’s close enough that most people won’t notice anything wrong. I also posted an example of an accurate inverse world to tangent matrix here:

1 Like