Multiple tangent spaces?

Unity Engine
1

Has anyone created a shader that deals with 2 tangent spaces to blend normal maps that are differently oriented?

We want to use UV0 for a base map, and UV1 to add details (decals/trim) that also have a normal map. The problem is that the orientation of the mapping isn’t guaranteed to be the same between the two layers.

Is there a standard nice for this sort of case? Or do people avoid it?

I guess I could use an AssetPostprocessor script to generate the second set of tangents, or some data derived from them (the 2D rotation required to rotate one normal into the other tangent space?).

(And on that subject, is there a better way than naming convention/string matching to tag models that require special postprocessing? - Is there any way to add custom data/UI to the import settings?)

2

Yes. I have.

Generally people avoid it, or they just YOLO it and use the UV0 tangents for everything (Unity’s own shaders do this!). None of Unity’s systems know how to handle more than one per vertex tangent, so things like batching or skinning will break if you try and generate a second set of tangents as they won’t get transformed properly. Though the idea of using a tangent that’s rotated from the base is a decent solution to that.

However what I do is I generate the tangent to world matrix in the fragment shader.

// Unity version of http://www.thetenthplanet.de/archives/1180
float3x3 cotangent_frame( float3 normal, float3 position, float2 uv )
{
    // get edge vectors of the pixel triangle
    float3 dp1 = ddx( position );
    float3 dp2 = ddy( position ) * _ProjectionParams.x;
    float2 duv1 = ddx( uv );
    float2 duv2 = ddy( uv ) * _ProjectionParams.x;
    // solve the linear system
    float3 dp2perp = cross( dp2, normal );
    float3 dp1perp = cross( normal, dp1 );
    float3 T = dp2perp * duv1.x + dp1perp * duv2.x;
    float3 B = dp2perp * duv1.y + dp1perp * duv2.y;
    // construct a scale-invariant frame
    float invmax = rsqrt( max( dot(T,T), dot(B,B) ) );
    return transpose(float3x3( T * invmax, B * invmax, normal ));
}
5 Likes
3

Thank you for the link to that post and for converting that snippet! For those (like me) who aren’t adept enough with graphics to immediately know how to utilize it in a context as described by @bluescrn :

  • The normal parameter is the mesh’s (interpolated) normal in world space
  • The position parameter is the world space position of the fragment minus the camera position. If you use HDRP with camera-relative rendering, this will simply be the world space position (the camera is at (0,0,0) then.)
  • You utilize your normal map vector by multiplying it with the cotangent frame matrix. By doing so, you perturb the mesh normal via your normal map and receive a normal in world space.
  • If you use shader graph or if you want to blend normals together from maps with different UV layouts (as was asked above with trim sheets) , you’ll want to transform this normal from world to tangent space.
1 Like
4

I have been banging my head against the wall for the past 3 hours, and for the life of me, I can’t figure out how to properly integrate this into my shader. I have no clue what I’m doing wrong here, and I hate to ask, but would you mind looking at my code and pointing out my mistakes?

Shader "Custom/Trim sheets"
{
    Properties
    {
        _MainTex ("Base color", 2D) = "white" {}
[Normal]_MainNorm ("Base normal", 2D) = "bump" {}
    _MainRM ("Base Roughness (R) and Metalic (G)", 2D) = "grey" {}
    _Trim ("Trim color", 2D) = "white" {}
[Normal]_TrimNorm ("Trim normal", 2D) = "bump" {}
    _MainRM ("Trim Roughness (R) and Metalic (G)", 2D) = "grey" {}
  
    }
    SubShader
    {
        Tags { "RenderType"="Opaque" "PerformanceChecks"="False"}
        LOD 200

        CGPROGRAM
        #pragma surface surf Standard fullforwardshadows vertex:vert
        #pragma target 3.5

        sampler2D _MainTex;
        sampler2D _MainNorm;
        sampler2D _MainRM;
    sampler2D _Trim;
        sampler2D _TrimNorm;
        sampler2D _TrimRM;

        struct Input
        {
                float2 uv_MainTex : TEXCOORD0;
        float2 uv3_Trim : TEXCOORD2;
        float4 pos;
        float3 norm;
        };

void vert (inout appdata_full v, out Input o) {
    UNITY_INITIALIZE_OUTPUT(Input,o);
    o.norm = v.normal;
    o.pos = UnityObjectToClipPos(v.vertex);

}
        UNITY_INSTANCING_BUFFER_START(Props)
        UNITY_INSTANCING_BUFFER_END(Props)



        void surf (Input IN, inout SurfaceOutputStandard o)
        {


//your code starts here
    float3 dp1 = ddx( IN.pos );
    float3 dp2 = ddy( IN.pos ) * _ProjectionParams.x;
    float2 duv1 = ddx( IN.uv3_Trim );
    float2 duv2 = ddy( IN.uv3_Trim ) * _ProjectionParams.x;
    // solve the linear system
    float3 dp2perp = cross( dp2, IN.norm );
    float3 dp1perp = cross( IN.norm, dp1 );
    float3 T = dp2perp * duv1.x + dp1perp * duv2.x;
    float3 B = dp2perp * duv1.y + dp1perp * duv2.y;
    // construct a scale-invariant frame
    float invmax = rsqrt( max( dot(T,T), dot(B,B) ) );

    float3x3 cotangent_frame = transpose(float3x3( T * invmax, B * invmax, IN.norm ));
//your code ends here


    float3 twonorm = tex2D (_TrimNorm, IN.uv3_Trim);
    twonorm*=2;
    twonorm-=1;
    float3 Norm2 =normalize(mul(cotangent_frame,twonorm));
    o.Normal = Norm2;



            fixed4 c = tex2D (_MainTex, IN.uv_MainTex);
        o.Albedo = c.rgb;
               o.Alpha = c.a;
        }
        ENDCG
    }
    FallBack "Diffuse"
}

Here’s how it works currently. my shader is on the left, and a reference using the standard shader using one uv map is on the right. I think it might be misidentifying the normals, because the error gets worse when I rotate the object.

5

The position should be the world position, not the clip space position. You could use float3 worldPos; in the Input struct, or to avoid some precision issues that will cause noise, use this in the vert program:

o.pos = mul(unity_ObjectToWorld, float4(v.vertex.xyz, 1)).xyz - _WorldSpaceCameraPos;

The other problem is this will calculate the tangent to world transform for an arbitrary UV. But the o.Normals expects the tangent space normals using the tangents for the mesh’s base UVs. So you still need to transform the world normals back into the’s mesh’s tangent space.

For that you’ll want to use this function:

6

Thank you, it works perfectly now. For anyone in the future that finds this thread and needs something similar, here’s the final shader:

edit: It works fine so long as you don’t rotate the model in question. If anyone can figure out how to fix that, please do share.

Shader "Custom/Trim sheets"
{
    Properties
    {
        _MainTex ("Base Color", 2D) = "white" {}
[Normal]_MainNorm ("Base Normal", 2D) = "bump" {}
    _MainR ("Base Roughness", 2D) = "grey" {}
    _MainM ("Base Metalic", 2D) = "black" {}
    _Trim ("Trim Color", 2D) = "black" {}
[Normal]_TrimNorm ("Trim Normal", 2D) = "bump" {}
    _TrimR ("Trim Roughness", 2D) = "grey" {}
    _TrimM ("Trim Metalic", 2D) = "black" {}
  
    }
    SubShader
    {
        Tags { "RenderType"="Opaque" "PerformanceChecks"="False"}
        LOD 200

        CGPROGRAM
        // Physically based Standard lighting model, and enable shadows on all light types
        #pragma surface surf Standard fullforwardshadows vertex:vert

        // Use shader model 3.5 target, because otherwise there won't be enough texture space.
        #pragma target 3.5

    #include "UnityStandardUtils.cginc"


        sampler2D _MainTex;
        sampler2D _MainNorm;
        sampler2D _MainR;
    sampler2D _MainM;
    sampler2D _Trim;
        sampler2D _TrimNorm;
        sampler2D _TrimR;
    sampler2D _TrimM;

        struct Input
        {
                float2 uv_MainTex : TEXCOORD0;
        float2 uv3_Trim : TEXCOORD2;
        float3 pos;
        float3 norm;
        float3 worldNormal;
INTERNAL_DATA
        };

float3 WorldToTangentNormalVector(Input IN, float3 normal) {
            float3 t2w0 = WorldNormalVector(IN, float3(1,0,0));
            float3 t2w1 = WorldNormalVector(IN, float3(0,1,0));
            float3 t2w2 = WorldNormalVector(IN, float3(0,0,1));
            float3x3 t2w = float3x3(t2w0, t2w1, t2w2);
            return normalize(mul(t2w, normal));
        }

void vert (inout appdata_full v, out Input o) {
    UNITY_INITIALIZE_OUTPUT(Input,o);
    o.norm = v.normal;
o.pos = mul(unity_ObjectToWorld, float4(v.vertex.xyz, 1)).xyz - _WorldSpaceCameraPos;


}
        // Add instancing support for this shader. You need to check 'Enable Instancing' on materials that use the shader.
        // See https://docs.unity3d.com/Manual/GPUInstancing.html for more information about instancing.
        // #pragma instancing_options assumeuniformscaling
        UNITY_INSTANCING_BUFFER_START(Props)
            // put more per-instance properties here
        UNITY_INSTANCING_BUFFER_END(Props)



        void surf (Input IN, inout SurfaceOutputStandard o)
        {
    IN.worldNormal = WorldNormalVector(IN, float3(0,0,1));
    fixed4 c;

float4 trim = tex2D (_Trim, IN.uv3_Trim);

//if you need to use a different normal map, switch the IN.uv variable
    float3 dp1 = ddx( IN.pos );
    float3 dp2 = ddy( IN.pos ) * _ProjectionParams.x;
    float2 duv1 = ddx( IN.uv3_Trim );
    float2 duv2 = ddy( IN.uv3_Trim ) * _ProjectionParams.x;
    // solve the linear system
    float3 dp2perp = cross( dp2, IN.norm );
    float3 dp1perp = cross( IN.norm, dp1 );
    float3 T = dp2perp * duv1.x + dp1perp * duv2.x;
    float3 B = dp2perp * duv1.y + dp1perp * duv2.y;
    // construct a scale-invariant frame
    float invmax = rsqrt( max( dot(T,T), dot(B,B) ) );

    float3x3 cotangent_frame = transpose(float3x3( T * invmax, B * invmax, IN.norm ));


    float3 twonorm = tex2D (_TrimNorm, IN.uv3_Trim);
    twonorm*=2;
    twonorm-=1;
    float3 Norm2 =normalize(mul(cotangent_frame,twonorm));

o.Normal=normalize(lerp(UnpackNormal(tex2D(_MainNorm, IN.uv_MainTex)),WorldToTangentNormalVector(IN, Norm2),trim.a));
c = lerp( tex2D (_MainTex, IN.uv_MainTex),trim.rgba,trim.a);
o.Smoothness = 1-lerp( tex2D (_MainR, IN.uv_MainTex), tex2D (_TrimR, IN.uv3_Trim),trim.a);
o.Metallic = lerp( tex2D (_MainM, IN.uv_MainTex), tex2D (_TrimM, IN.uv3_Trim),trim.a);

        o.Albedo = c.rgb;
               o.Alpha = c.a;
        }
        ENDCG
    }
    FallBack "Diffuse"
}
7

Is it possible to implement this in shader graph or Amplify Shader Editor?
Because I have the same question

8

For triplanar stuff in Shader Graph, you can set the output normal to Object or World space, or use the transform node to convert from world to tangent space. The built in Triplanar node also already outputs the normal in tangent space.

As for the cotangent_frame function, every one of those things can be done with the nodes Shader Graph and Amplify have. The main tweak is instead of rsqrt, you use sqrt (square root) and divide T and B instead of multiply.