Optimising Overlay

Graphics Shaders & Materials
1

Hey folks,

Got a shader in one project at the moment that’s causing a bit of a pain. There are 3 images going into it, a diffuse texture, a lightmap texture, and a baked ambient occlusion texture. In trying to get the right look as close to the pre-rendered concepts as possible, I was looking at using an overlay function for the diffuse and lightmap. On pc and mac it looks good, However I soon found out that this wouldn’t run on device.

Here is the shader as it stands at the moment:

Shader "Custom/Crate Shader"
{
	Properties 
	{
		_Color("_Color", Color) = (1,1,1,1)
		_MainTex ("Base (RGB) AO(A)", 2D) = "white" {}
		_LightMap ("LightMap", 2D) = "white" {}
		_AOTex ("Ao texture", 2D) = "white" {}
		_AOStrength ("AO Strength", Range(0,1)) = 0
		_LightMapStrength ("Light Map Strength", Range(0,1)) = 0
	}

	SubShader 
	{
		Tags {"Queue"="Transparent" "RenderType"="Transparent"}
		LOD 200
		Blend SrcAlpha OneMinusSrcAlpha

		Pass 
		{
			CGPROGRAM

				#pragma vertex vert
				#pragma fragment frag
				#include "UnityCG.cginc"
			
				struct v2f 
				{
					float4 pos : SV_POSITION;
					float2 uv : TEXCOORD0;
					float2 uv2 : TEXCOORD1;
				};

				struct appdata 
				{
					float4 vertex : SV_POSITION;
					float2 texcoord : TEXCOORD0;
					float2 texcoord1 : TEXCOORD1;
				};
			
				sampler2D _MainTex;
				sampler2D _LightMap;
				sampler2D _AOTex;

				float4 _MainTex_ST;
				float4 _LightMap_ST;
				float4 _AOTex_ST;

				half _AOStrength;
				half _LightMapStrength;
				fixed4 _Color;
			
				v2f vert(appdata v) 
				{
					v2f o;
					o.pos = mul(UNITY_MATRIX_MVP, v.vertex);
					o.uv = TRANSFORM_TEX(v.texcoord, _MainTex);
					o.uv2 = TRANSFORM_TEX(v.texcoord1, _MainTex);
					return o;
				}
			
				half Overlay(half a, half b)
				{
					if(a < 0.5)
						return 2*a*b;
					else
						return 1-(2*(1-a)*(1-b));
				}
			
				fixed4 frag(v2f IN) : COLOR 
				{
					fixed4 diffuse = tex2D (_MainTex, IN.uv);
					fixed4 lightmap = tex2D (_LightMap, IN.uv2);
										
					diffuse.r = Overlay(diffuse.r, lightmap.r);
					diffuse.g = Overlay(diffuse.g, lightmap.g);
					diffuse.b = Overlay(diffuse.b, lightmap.b);
					
					diffuse *= 1 - (_AOStrength * (1 - (tex2D(_AOTex, IN.uv2))));
					diffuse.a = 1;

					return diffuse * _Color;
				}


			ENDCG
		}
	}
    FallBack "Transparent/Diffuse"
}

Now when run like this on device, the mesh itself just disappears completely and the frame rate drops far below acceptable levels. However if i comment out just one of the lines that calls the overlay, regardless of which RG or B channel, the box draws (albeit with undesired colour) and the framerate is happy to sit back up at 30.

Now I’m no shader programmer, I get by with shaders just about. I’m hoping that someone will read this and be able to make a suggestion as to how to make the shader cheaper so it will run on device overlaying all 3 channels, or perhaps knows from experience that this wont work and have an alternative approach to suggest.

Note: I have tried removing the AO to see if that helps at all, but the behaviour is still the same, broken using 3 channels, and working when only using 2. And the strength values are only there for development, when we settle on values they will of course be baked into the shader.

Thanks.

2

Well I’m no shader expert either,
but glancing at your code, I see some things that could be better:
I’ve always been told that if-statements are a bad idea in shaders, supposedly they can be lots slower than just doing both calculations and then adding both, using sign() and saturate() and the like to have the same result.
Also, you can do stuff like diffuse.rgb += 0.5 which is supposedly just as fast as diffuse.r += 0.5, but does this for each channel, making it 3 times as fast as dooing it seperatly.
Anyway, honestly though, I just quickly glanced at your shader, so I’m sure there’s more than that.

3

Thanks Steven, although in the end we have decided to ditch the light map in favour of using the vert colours to let us paint on some lighting. However I’ will definitely take your advice and try make some optimizations for the shader as it might be useful later on.

I know if statements aren’t great, but I just couldn’t wrap my head around how to emulate the behaviour just with a fancy formula (using the if statements to check against 0.5 being why I had to do it per channel instead of on the whole rgb at once). But I’ll see what happens.

Cheers again.

4

yeah, I know, If-statements are a lot more logical/readable. I generaly also use if-statements when developing a shader, and only change that when the shader is otherwise finished.

and yeah, I see what you mean about why you had to do it seperately for each channel, didn’t notice that before, hadn’t read it properly, sorry about that

5

outside of answering your question im wondering why you dont combine your diffuse and AO in your graphics editor as this will always look the same regardless of lighting … its only the lightmap/'s texture that can be unique for each crate? so then you only have to handle 2 textures in the shader.

6

You’re not really giving us enough information. You say it fails on a device, but don’t mention which device or platform or 3D API. If it does fail I would expect some errors either for the shader compilation or whilst running. I’m also surprised the model simply disappeared when you have a fallback shader included, not to mention that its a transparent shader yet uses lightmaps and AO?

It might be useful to know why you have to compare each component separately as that seems unusual especially the way you are using it. I could imagine it having some value if each component had some unique representation of some data (e.g. if you were packing 3x8bit maps), but you’re just using diffuse and a lightmap.

Also you define LightMapStrength but its never used.

You mention commenting out any one of the Overlay lines both make it work and brings back performance. That might suggest you are running out of instructions (ALU, temporary registers etc), though its odd that the shader can compile if thats the case. Maybe youve found a strange edge case on the device you are targetting.

7

@shaderbytes : The reason the AO texture was separate is because the AO was done in the second UV channel. This is because the model reused parts of the diffuse channel in multiple areas, but the AO is unique to each part of the model. (so really it was just a texture real estate issue).

@noisecrime : Good point, Reading the post back I realise that there was a fair bit of information that should have been included. The device(s) in question were iPad3 and iPhone4S in this case. It definitely wasn’t an instruction count issue as those have stopped the shader compile before. The LightMapStrength is just an artefact from various modifications to the shader that I clearly forgot to remove when posting. Also the reason I was doing checks per channel is that each channel needed to be checked against a 0.5 threshold which as far as I could tell at the time, isn’t possible to do on all channels at once with independent results per channel. (although as Steven pointed out, similar behaviours can be emulated in other ways).

But since the issue was solved with a different approach, I’ not concerned by it any more, but still appreciate people providing feedback.

(Note: I also am not sure why with a fallback shader it didn’t render something, but I’m sure there is a simple enough reason. Like I said though, with it being a solved issue now, I’m not really dedicating any time to look into it more).

8

Without if statement:

float3 blend_overlay( float3 base, float3 blend )
{
    return lerp( (base*blend*2), (1.0-(2.0*(1.0-base)*(1.0-blend))), round(base));
}