Convert vector from tangent space to local space in C#

Unity Engine
1

Is there anything built in, or any info on how to convert from tangent space to local space in C#?

2

What do you mean by “tangent space”?

3

https://en.wikipedia.org/wiki/Tangent_space
Like a normal map is in tangent space for example (99.9% of the time)

I have a mesh, and want to sample a texture at each vertex’s uv coords, and convert that vector from tangent space to local space.

Most of that is trivial, but not sure how to convert from tangent space outside of a shader.

4

Ah, I see. So you need the local-space equivalent of the given UV coordinates on the mesh? No, nothing builtin exists for that.

One issue to start with is that there is no conceptual single answer to that question. A given UV coordinate might appear in many locations if the texture is reused, or it might not appear at all. So whatever this function is, it’d have to return a Vector3[ ].

The only way I can think of to accomplish this would be:

  1. Loop through all triangles
  2. Check the triangle’s UV coordinates and see if the desired UV is contained within it
  3. If so, find a sort of “local coordinate” relative to the UV triangle. (e.g. distance along side A by distance along side B)
  4. Apply this “local coordinate” to the triangle’s XYZ coordinates to get the XYZ coordinate that corresponds with this UV. Add this to the list of coordinates you’ll be returning.
  5. Repeat with all triangles
5

I think you misunderstood. I’m going to sample the texture at each vertex’s uv coordinates. After unpacking the normal from the RGB values I now have a tangent space normal. I just need to convert this normal from tangent space to local space. This is done in shaders all the time using things like Shader Graph’s Transform node for example.

6

OK, I gotcha. That is much simpler algorithmically and just involves vector math. I think this is about what you need to do:

Vector3 localNorm = mesh.normals[i];
Vector4 localTangent4 = mesh.tangents[i];
Vector3 localTangent = new Vector3(localTangent4.x, localTangent4.y, localTangent4.z); //The "U" direction
Vector3 localCross = Vector3.Cross(localNorm, localTangent) * localTangent4.w; //The "V" direction
Color normRGB = normalTexture.GetPixelBilinear(mesh.uv[i].x, mesh.uv[i].y);
Vector2 sampledNormal = new Vector2(normRGB.r * 2f - 1f, normRGB.g * 2f - 1f); // not 100% sure about this conversion?
return localTangent * sampledNormal.x + localCross * sampledNormal.y;
7

Sorry I’m not understanding your algorithm, because we already have the UVs and know where to sample the texture.

var uv = uvs[i];
var color = pixels[Utils.Convert2DTo1D(uv.x, uv.y, 512)];
var tangentNormal = new Vector3(
    color.r * 2.0f - 1.0f,
    color.g * 2.0f - 1.0f,
    color.b * 2.0f - 1.0f
);

// var localNormal =

We have the UVs, and we have the tangent space normal. Just need to convert the normal to local space. I assume the conversion process will involve some of those things you used, like the vert tagents, but no idea what.

8

OK, you can use your existing variables in that algorithm. tangentNormal is equivalent to sampledNormal (though I didn’t bother with the Z), “uv” is equivalent to mesh.uv*, and everything else in my code is the same.*

9

Oh okay, any tips for how to incorporate the z axis now? I’m not picking up on the pattern here :frowning:

That’s if it’s not already complete in this line?

return localTangent * sampledNormal.x + localCross * sampledNormal.y;

Btw I very much appreciate your help, I’m a little out of my depth here!

10

I’m not even sure that you do incorporate the Z axis at all? I’m not clear on what it represents in a normal map. It seems to be related to the normal so maybe adding in localNorm * normRGB.b would work?

11

Oh I think normals do use all 3 channels (though can be compressed down to 2), and anyway this is actually technically a vector displacement map which does require x, y, z.

I have the current algorithm in and it’s working enough to tell that we’re heading in the right direction, but missing something.

12

This thread is now the top result when searching for how to convert from tangent space to local space in C#, so if you stumble upon this in the future here is how to do it!

localTangent * sampledNormal.x + localCross * sampledNormal.y + localNormal * sampledNormal.z

Thanks again Ray, really appreciate the time you took to reply and help!

1 Like