How did unity convert a texture to normal

Unity Engine
1

I was currently woring on to convert some normal textures, which can only be used on PC with DXT5 compression, to be usable on android.
The basic concept of convertion is to reassign the value of z by the formula below:
— Unpack the normal as DXT5
r = r * 2 - 1;
g = g * 2 - 1;
b = sqrt(1 - saturate(r * r + g * g));
— Repack it in RGB
r = (r + 1) / 2;
g = (g + 1) / 2;
b = (b + 1) / 2;
and then normalize the rgb.

This formula worked for most of the textures. And their converted texture looks the same as the original one when set as normal type.
But some of them have pixels with (r * r + g * g) value greater than 1. Which cause the value of red and blue reduced after normalized and looked different from the original.
As I tested, it worked differently as well when assigned as normal.

So I think I might use the wrong formula for this.
Could anyone help me on the correct formula to convert the normal maps?

2

Sorry for my bad expression.
When you set a texture as normal type, the color is changed/fixed in Unity inspector. And the texture with changed color worked perfectly on mobile.
Some texture has no alpha channel, which means the editor cannot be using red and alpha to convert it (like DTX5).

I need to know what formula Unity is using for the color convertion, so I can convert/fix some normal map that cannot be used on mobile.

Anyone can help me?

3

I’m currently sure it’s only about red and green channel - I’ve tried to mess up with blue and alpha, which didn’t affect the result of the convert.

4

Never mind, it’s some kind of mistake on io or other stuff.
The correct formula is unpack the texture as DTX5 with r and b channels, normalize the result, then repack it in rgb.