I am seeing a very interesting problem on Metal API: when we compute a normalized world-space tangent in vertex shader, then pass it to fragment using a texcoord, the result is no longer normalized.

(2) somehow the model itself has wrong tangent. But I find this unlikely, as we are using mikktspace recalculation when importing, inspecting tangentOS between platforms, we didn’t see anything different.

(3) somehow the value is modified during rasterization step??

I am only able to reproduce this on some iOS device in combination with certain assets. The vertex-only normalization works on every other major platform, including Metal API on macOS, yet it fails on iOS.

This causes world-space normal calculation to be wrong, and affects the shading eventually.

I have no idea why, but for now it seems I have to normalize again at fragment stage to workaround this issue.

This is completely proper behaviour.
When you pass values from vertex to fragment shader, they get linearly interpolated between vertices of the triangle.
If the values are normalized in the vertex shader, you’re most likely not going to have a normalized vector in the fragment shader.
Example: you have two vertices, vectors are, respectively, (1, 1, 0) and (1, -1, 0). After normalization, they both get scaled by 1/sqrt(2). In the fragment shader, you’re going to get anything between 1/sqrt(2) and -1/sqrt(2) (including 0!) on the Y axis. So, the point in between those two vertices will get (1/sqrt(2), 0, 0).

So, to be accurate, we need to normalize tangent in both vertex and fragment shader, true? Or should we just skip computing tangent and binormal in vertex shader?

Second, why does this work on virtually all other platform, but only failed on a certain range of iOS devices? Were the working platforms just pure luck??

I see Unity Standard shader has “PerPixelWorldNormal”, which ortho-normalize tangent, and I have verified “UNITY_TANGENT_ORTHONORMALIZE” is 1 for Metal (on iOS and macOS), does it mean Unity always normalize tangent in fragment again?

Thx in advance!

float3 PerPixelWorldNormal(float4 i_tex, float4 tangentToWorld[3])
{
#ifdef _NORMALMAP
half3 tangent = tangentToWorld[0].xyz;
half3 binormal = tangentToWorld[1].xyz;
half3 normal = tangentToWorld[2].xyz;
#if UNITY_TANGENT_ORTHONORMALIZE
normal = NormalizePerPixelNormal(normal);
// ortho-normalize Tangent
tangent = normalize (tangent - normal * dot(tangent, normal));
// recalculate Binormal
half3 newB = cross(normal, tangent);
binormal = newB * sign (dot (newB, binormal));
#endif
half3 normalTangent = NormalInTangentSpace(i_tex);
float3 normalWorld = NormalizePerPixelNormal(tangent * normalTangent.x + binormal * normalTangent.y + normal * normalTangent.z); // @TODO: see if we can squeeze this normalize on SM2.0 as well
#else
float3 normalWorld = normalize(tangentToWorld[2].xyz);
#endif
return normalWorld;
}

In short: I agree with everything you said about the tangent being interpolated during rasterization, I am aware we can fix this by recalculating tangent in fragment stage.

My question is: why is a certain range of iOS device, interpolated the result so much different from the rest of platforms? In a way that we can’t ignore the errors in fragment, and HAVE TO recompute?

Very good idea, these are the results of “return half4(abs(normalize(tangentWS) - tangentWS), 1.0);”:

macOS (MacBook Air 2012)

iOS (iPhone 7)

We can reproduce the severe tangent mismatch on iPhone 7, iPhone 10 and iPad Pro 2018; but not on iPhone 11 Pro Max, or macOS.

And as you can see, not all assets are affected, the crates in the second building is using the same shader and texture, but doesn’t suffer from this problem. But AFAIK, the their import settings are the same, the main difference might be their scaling in prefab.

But on the other hand, macOS has no issue with it, so perhaps it’s indeed shader precision (we don’t see issues on Android though); I also see a few models with only 0.01 scaling, but they are also affected by this issue.

This isn’t a new bug. People have been complaining about the normals on iOS devices getting messed up when an object is scaled for years. The issue from the debugger screenshot above is pretty clear that the normalize in the vertex function is just being ignored. Not sure if this is a compiler issue or something with the device itself.

Fragment shader orthonormalization breaks MikkTSpace normal maps. One of the benefits of MikkTSpace is you don’t need to normalize those vectors, only the resulting normal. It seems like that bit of code was being used to work around the bug that the normalize is being skipped in the vertex shader, but ironically it’s doing so in a way that’s way more expensive than the proper fix of normalizing the tangent space normal after being transformed into world space, which is a common thing Unity skips as an optimization. Notice the todo comment…