[SOLVED] Standard Shader - Normal Maps strenght in iOS not working

[Pa0L0]

Hi,

So I have tested a bunch of different settings compiling for iOS, but the “strenght” of the Normal Map in a Standard Shader material is not being considered while running the app in the device (iPhone 6 or newer).

The device is displaying the Normal Map, but it always display as with Strenght = 1

Is it a device limitation? or we can consider this a bug?

Does anybody know a workaround for this? …or editing the normal map texture directly is the only option so far?

By the way, I found old threads complaining about the same thing, so it looks like this issue is there since long time ago , so I decided to start a new thread.

Thanks!

[bgolus]

It’s a conscious decision by Unity for optimization purposes.

It’s basically free for desktop because of the way normal maps are stored and have to be decoded, but normal maps on mobile aren’t stored in any special way so the process of doing the scaling would increase the shader cost by a not insignificant amount.

The mobile shader and desktop shader are actually quite different with the mobile shader being nearly an order of magnitude less complex. So something that is cheap for desktop isn’t so cheap for mobile.

[Pa0L0]

@bgolus

It’s a conscious decision by Unity for optimization purposes.

It’s basically free for desktop because of the way normal maps are stored and have to be decoded, but normal maps on mobile aren’t stored in any special way so the process of doing the scaling would increase the shader cost by a not insignificant amount.

The mobile shader and desktop shader are actually quite different with the mobile shader being nearly an order of magnitude less complex. So something that is cheap for desktop isn’t so cheap for mobile.

Thanks, but do you mean the normal maps strenght has to be processed in every single frame in mobiles??
I mean, the normal map is already being displayed and used correctly by the device, but it just doesn’t consider the strength setting from the editor, so … Couldn’t the shader modify the strength of the map just one time (when shader loads), and then just use it with the same calculations as it does now?

[bgolus]

@Pa0L0

Couldn’t the shader modify the strength of the map just one time (when shader loads), and then just use it with the same calculations as it does now?

That’s not how shaders work.

Shaders take data you give it and run ever frame, for every pixel on screen. The output is the image on screen, and all other data is thrown away and calculated again for each pixel, each frame.

That seems wasteful, but it works well with how GPUs function. They’re massively parallel processors that can be fed relatively small amounts of data and return a color value. The cost of moving data around can actually be more expensive (in terms of how long it takes) than recalculating something over and over again for every pixel, so they do that instead of calculating it once and then passing that data along. That’s especially true for mobile where the GPUs are quickly getting faster and faster at raw calculation, but the memory bandwidth isn’t increasing at the same speed. Now there’s the split between vertex and fragment shaders that does allow you to calculate something once and pass it to the fragment shaders, but that’s because this happens in different steps.

And yes, I know I just said it’s expensive to do normal map scaling in my previous post, and then said it’s faster to calculate everything every frame in this one. It’s a balancing act, send as little data as possible and do as little calculation as possible, and try to make smart decisions about which to use when you have to. In the case of a normal map the cost of sending one normal map or another that happens to be a scaled version is the same, so doing the work to scale it in the shader is almost always a loss.

If you want to know more you should read up on rasterization and shaders. This is a good place to start:
http://www.alanzucconi.com/2015/06/10/a-gentle-introduction-to-shaders-in-unity3d/

Now if you want to modify a normal map at runtime you can totally do that, even with a shader, but you’d need a shader that explicitly takes a normal map and outputs the modified version that you then save and store. But on mobile that can be expensive as textures are usually stored in a compressed form (again, because moving data around is slow), and if you use a shader the resulting texture is not compressed. Moreover to compress the texture on a PC can take several seconds to even minutes depending on the size and format; compressing on a mobile device can take hours(!) so they don’t even bother supporting that. Instead you’re left with an uncompressed copy of a texture you already have in memory taking up more memory and slowing down rendering.

So your best option is to modify the texture before hand. Either do it in your 2d graphics tool of choice, or you can write a tool to do it in the Unity Editor and output a new texture asset.

The last alternative is write a custom surface shader that does the normal scaling. The main thing that’s expensive about normal map scaling is actually when scaling it up and not when scaling it down / smoothing it out. If you only ever want to scale down normal maps that’s much cheaper.

o.Normal = lerp(half(0,0,1), UnpackNormal(tex2D(_BumpMap, IN.uv_MainTex)), _NormalMapScale);

[Pa0L0]

@bgolus

That’s not how shaders work.

Shaders take data you give it and run ever frame, for every pixel on screen. The output is the image on screen, and all other data is thrown away and calculated again for each pixel, each frame.

That seems wasteful, but it works well with how GPUs function. They’re massively parallel processors that can be fed relatively small amounts of data and return a color value. The cost of moving data around can actually be more expensive (in terms of how long it takes) than recalculating something over and over again for every pixel, so they do that instead of calculating it once and then passing that data along. That’s especially true for mobile where the GPUs are quickly getting faster and faster at raw calculation, but the memory bandwidth isn’t increasing at the same speed. Now there’s the split between vertex and fragment shaders that does allow you to calculate something once and pass it to the fragment shaders, but that’s because this happens in different steps.

And yes, I know I just said it’s expensive to do normal map scaling in my previous post, and then said it’s faster to calculate everything every frame in this one. It’s a balancing act, send as little data as possible and do as little calculation as possible, and try to make smart decisions about which to use when you have to. In the case of a normal map the cost of sending one normal map or another that happens to be a scaled version is the same, so doing the work to scale it in the shader is almost always a loss.

If you want to know more you should read up on rasterization and shaders. This is a good place to start:
http://www.alanzucconi.com/2015/06/10/a-gentle-introduction-to-shaders-in-unity3d/

Now if you want to modify a normal map at runtime you can totally do that, even with a shader, but you’d need a shader that explicitly takes a normal map and outputs the modified version that you then save and store. But on mobile that can be expensive as textures are usually stored in a compressed form (again, because moving data around is slow), and if you use a shader the resulting texture is not compressed. Moreover to compress the texture on a PC can take several seconds to even minutes depending on the size and format; compressing on a mobile device can take hours(!) so they don’t even bother supporting that. Instead you’re left with an uncompressed copy of a texture you already have in memory taking up more memory and slowing down rendering.

So your best option is to modify the texture before hand. Either do it in your 2d graphics tool of choice, or you can write a tool to do it in the Unity Editor and output a new texture asset.

The last alternative is write a custom surface shader that does the normal scaling. The main thing that’s expensive about normal map scaling is actually when scaling it up and not when scaling it down / smoothing it out. If you only ever want to scale down normal maps that’s much cheaper.

o.Normal = lerp(half(0,0,1), UnpackNormal(tex2D(_BumpMap, IN.uv_MainTex)), _NormalMapScale);

Thanks for taking the time with a detailed response

So, just to know if we are in the same page here:
I can see you said “scaling down/up the texture”,
but do you mean changing the size of the texture?
I never meant the texture size should change in runtime, and I don’t think that’s what the “strength” parameter does.
So in Unity I can see that the only difference is how “bumpy” the pixels are displayed for the material (based on what you selected as the “strength” parameter). But the normal map texture itself, remains with the same size (256x256 for example)

So I guess that multiplying this “bumpy” pixel value on every frame by a float number between
0…1, would be expensive for a mobile? Am I correct here?
Thanks!

[bgolus]

@Pa0L0

“scaling down/up the texture”

The “scaling up and down” I’m referring to is the scale value the Standard shader has, which is modifying the normals to increase or decrease the “strength” of the bumpy effect and not the texture size as you surmised. Also a normal map is just another texture that just happens to have normal data.

The problem is what you have to do to make a normal map look more bumpy. You can’t just multiply the normal, you have to change the direction encoded in the texture.

half3 UnpackScaleNormal(half4 packednormal, half bumpScale)
{
    #if defined(UNITY_NO_DXT5nm)
        return packednormal.xyz * 2 - 1;
    #else
        half3 normal;
        normal.xy = (packednormal.wy * 2 - 1);
        #if (SHADER_TARGET >= 30)
            // SM2.0: instruction count limitation
            // SM2.0: normal scaler is not supported
            normal.xy *= bumpScale;
        #endif
        normal.z = sqrt(1.0 - saturate(dot(normal.xy, normal.xy)));
        return normal;
    #endif
}

This is the function Unity uses to do normal scaling. At the top is why it doesn’t do anything on mobile, mobile generally doesn’t support DXT5nm so it just skips the code. Just after that is code to skip it if it’s a low end desktop or mobile platform that does support DXT textures. And lastly that normal.z = sqrt(1.0 - saturate(dot… etc. is actually a fairly expensive line. sqrt() isn’t cheap, and dot() isn’t free either (saturate() actually is free).

However because normal maps on platforms that support the DXT texture format only store the red and green channels (x & y) of the normal the blue (or more specifically the z) has to be reconstructed. On most mobile platforms you just have to do normal.xyz * 2 - 1 and you’re done so you can skip all of that math for reconstructing the z component that’s needed for scaling the “bumpiness”. Again, that complexity is only needed for scaling greater than 1, and for 0-1 the little snippet of code I posted above works just fine. It’s not free either, but it’s cheaper than the above function.

As for why on platforms where DXT5 is supported, but on a lower end system they skip the single multiply, it’s because they’re limited to 64 instructions in a shader, and they’re often right at that limit so every instruction counts. Just unpacking a DXT5 normal texture is 8~9 instructions by itself.

[Pa0L0]

Thanks again for the great explanation!

Then I think my easiest option at the moment is modifying the normal map directly in my graphics program.

In the code above, I wonder what would happen if I modify the initial if…else to be this way

    ...
    #if defined(UNITY_NO_DXT5nm)
        packednormal.xy *= bumpScale; // <- new line here ;)
        return packednormal.xyz * 2 - 1;
    #else
    ...

[bgolus]

That particular bit of code will cause all of your normal maps to skew up and right for >1 values and down and left for <1 values, and also have blown out / overly dark lighting.

packednormal.xyz * 2 - 1;
packednormal.xy *= bumpScale;
return normalize(packednormal);

That would work a little better, but that normalize() is actually just as expensive as the sqrt(1.0 - saturate(dot(normal.xy, normal.xy))) for mobile (and is actually potentially more expensive on desktop for complicated reasons), and the later will produce potentially better results.

[Pa0L0]

So, this is being clearly avoided by the Unity Team then.
They just decided to skip that logic in the shader for mobile performance.

So, it would have been really good to actually let the people know about, … I mean adding that bit of info into the documentation of the “material inspector”. If I knew that, then first I would have never started this thread, and second I would have created my normal maps with the correct strenght from the very first time with my graphics program, and not relying in the “Strenght” parameter of the inspector

I would like to mark your answers number #4 or #6 as the “solution” to this thread but I don’t find any option around (I’m new to unity and its stuff actually haha)

[bgolus]

Yeah, XenForo (the forum software Unity uses) doesn’t have a “mark as solved”. If you want to be friendly to future readers you can edit the thread title to say “[SOLVED] blah blah”.

Curiously there’s a lot of the standard shader that gets disabled when building for mobile, and there’s not a straightforward way to know what those are in the editor. You can preview some of it by using graphics emulation settings, but it still doesn’t warn you anywhere.

[Pa0L0]

Curiously there’s a lot of the standard shader that gets disabledwhen building for mobile, and there’s not a straightforwardway to know what those are in the editor

That’s sooo bad. I just hate the undocumented stuff when it requires just a few “warning” sentences in the docs.

I changed the title to SOLVED.

[Kritankboy]

@bgolus

The “scaling up and down” I’m referring to is the scale value the Standard shader has, which is modifying the normals to increase or decrease the “strength” of the bumpy effect and not the texture size as you surmised. Also a normal map is just another texture that just happens to have normal data.

The problem is what you have to do to make a normal map look more bumpy. You can’t just multiply the normal, you have to change the direction encoded in the texture.

half3 UnpackScaleNormal(half4 packednormal, half bumpScale)
{
    #if defined(UNITY_NO_DXT5nm)
        return packednormal.xyz * 2 - 1;
    #else
        half3 normal;
        normal.xy = (packednormal.wy * 2 - 1);
        #if (SHADER_TARGET >= 30)
            // SM2.0: instruction count limitation
            // SM2.0: normal scaler is not supported
            normal.xy *= bumpScale;
        #endif
        normal.z = sqrt(1.0 - saturate(dot(normal.xy, normal.xy)));
        return normal;
    #endif
}

This is the function Unity uses to do normal scaling. At the top is why it doesn’t do anything on mobile, mobile generally doesn’t support DXT5nm so it just skips the code. Just after that is code to skip it if it’s a low end desktop or mobile platform that does support DXT textures. And lastly that normal.z = sqrt(1.0 - saturate(dot… etc. is actually a fairly expensive line. sqrt() isn’t cheap, and dot() isn’t free either (saturate() actually is free).

However because normal maps on platforms that support the DXT texture format only store the red and green channels (x & y) of the normal the blue (or more specifically the z) has to be reconstructed. On most mobile platforms you just have to do normal.xyz * 2 - 1 and you’re done so you can skip all of that math for reconstructing the z component that’s needed for scaling the “bumpiness”. Again, that complexity is only needed for scaling greater than 1, and for 0-1 the little snippet of code I posted above works just fine. It’s not free either, but it’s cheaper than the above function.

As for why on platforms where DXT5 is supported, but on a lower end system they skip the single multiply, it’s because they’re limited to 64 instructions in a shader, and they’re often right at that limit so every instruction counts. Just unpacking a DXT5 normal texture is 8~9 instructions by itself.

Hey it’s 2023. I think mobile phones support the dxt5 format in today’s date. The standard unity shader still seems to keep bump scale disabled on the other hand the terrain seems to allow that for mobile. Do you think bump mapping on mobile is much of an issue after so many years? Thanks