Opaque on lighted parts shader

[anon_73729884]

Hello,

I need to simulate a light effect over an existing background picture, like on the following shot (this is a modified picture) :

To acheive that, I tried to make a shader which outputs a 0 alpha value on non-lighted parts of the surface, and a x>0 alpha value on lighted parts. The output alpha of this shader would basically be the rgb one of the following picture :

I now need to plug the rgb output value of the surface to its alpha channel, so that the black parts become transparent. As it is directly related to lighting, I created a custom lighting model from BlinnPhong, but no matter what value was passed to the alpha output of the lighting mode, it does not change anything visible :

// Based on BlinnPhong Lighting model
inline fixed4 LightingBlinnPhongMod (SurfaceOutput s, fixed3 lightDir, half3 viewDir, fixed atten)
{
	half3 h = normalize (lightDir + viewDir);
	
	fixed diff = max (0, dot (s.Normal, lightDir));
	
	float nh = max (0, dot (s.Normal, h));
	float spec = pow (nh, s.Specular*128.0) * s.Gloss;
	
	fixed4 c;
	c.rgb = (s.Albedo * _LightColor0.rgb * diff + _LightColor0.rgb * _SpecColor.rgb * spec) * (atten * 2);
        
    // the alpha value does not seem to change anything in the scene
	c.a = (s.Albedo * _LightColor0.rgb * diff + _LightColor0.rgb * _SpecColor.rgb * spec) * (atten * 2);
	return c;
}

I found this thread about a similar problem but it did not help me.

Does anyone know why this alpha output does not impact the scene ?

Alternatively, do you think is is the right way to achieve this “Opaque on lighted parts and transparent on non-lighted parts” effect ?

Thank you !

[anon_73729884]

Maybe I was not very clear, but basically what I want to make is a “Grayscale to alpha” shader, after ligthing has been computed.
This should not be very hard to achieve I think, but I’m struggling with that ! I’ve also tried using ColorMask A and blending tweaks, but no way to get what I want…

Any help would be very much appreciated.

[aubergine]

Is the background picture a plane mesh? or a skybox kind of thing?

[anon_73729884]

The background picture is a GUITexture displayed by a dedicated camera, and the 3D scene is showed over this background by another camera.

[aubergine]

I havent messed with surface shaders since some time. But if you can post a simple scene setup, i can try to make this thing work.

[anon_73729884]

Thank you Aubergine !

Here is the scene, with the following hierarchy :

3DScene
| Lamp object (with a spotlight inside)
| Light plane (the plane with the problematic shader attached, lighted only by the lamp’s spotlight)
| Main Cam
| Main Light
|
Background
| BackgroundCam
| BackgroundPicture (GUITexture)

Since last post, I changed the Base Lighting Model to Lambert as I don’t need specular.

Currently, when you change the rgb color of LightPlane’s material, it directly impacts the Alpha output (so black makes the plane totally transparent). Ideally, the lighting model would then modify the alpha of the plane’s lighted parts, but for some reason it does not change anything visible.

Thank you again, and if you need more information, just ask !

859222–32049–$gs2a_Shading.rar (412 KB)

[aubergine]

Hi, here is your shader:

Shader "Custom/GrayscaleToAlpha" {
	Properties {
		_MainTex ("Texture", 2D) = "white" {}
		_Color ("Main Color", Color) = (1,1,1,1)
	}
	SubShader {
		Tags { "RenderType"="Opaque" }
		LOD 200
		
		CGPROGRAM
		#pragma surface surf LambertMod

		half4 LightingLambertMod (SurfaceOutput s, half3 lightDir, half atten) {
			half NdotL = dot (s.Normal, lightDir);
			half4 c;
			clip(_LightColor0.a - 1);
			c.rgb = s.Albedo * _LightColor0.rgb * (NdotL * atten * 2);
			c.a = s.Alpha;
			return c;
		}

		struct Input {
			float2 uv_MainTex;
		};
		
		sampler2D _MainTex;
		float4 _Color;

		void surf (Input IN, inout SurfaceOutput o) {
			half4 c = tex2D (_MainTex, IN.uv_MainTex) * _Color;
			o.Albedo = c.rgb;
			//o.Alpha = c.a;
		}
		ENDCG
	} 
	FallBack "Diffuse"
}

You should use a plane object for the background rather than GUI texture, its faster.

[anon_73729884]

Amazing ! Thank you very much !

I have some questions about the shader you made (I’m a shader newbie so it can help me to understand better ) :

• Why does the “alpha” mode of the cg program have to be disabled, while the final output can have some transparency ? I’ve noticed that if I add the keyword to your shader, it does not work anymore…
• Why did you switch the RenderType to “Opaque” ?
• About the clip function : I thought it was not able to output this result, as it kills some pixels, so this creates alias :

Why in our case the output is not aliased ?

Thanks a lot !

[aubergine]

1-alpha doesnt have to be disabled, i made an opaque example thats it. You can add alpha to your pragma surface and then you can use your normal alpha as you want. But in your case you dont want this extra calculation because you only need to clip the black areas of your lighting.
Pragma alpha is only used in surf function, you can or must use seperate alpha calculation in lighting function as the pragma alpha doesnt get involved with lighting itself. Unfortunately, due to the way surface shaders work, blend functions wont work in lighting calculations in surface shaders.
Briefly, when you do this way of clipping in lighting function, you basically overwrite to the alpha calculation which was done in surf which is unnecessary.
2-Same reason as above. So, it could be faster.
3-Clipping is same as transparent-cutout, it will leave artifact pixels and will need antialiasing if your lighting isnt homogenous. Try changing the spotlights angle to 45 or more and you will see those undesired pixelation. By the way, it wont matter if you use _LightColor0.a or _LightColor0.r, its same.

Hope it helps.

Edit: If your lights will be static, i suggest you use a lightmap and just blend the texture on your plane object it will be faster if speed is concern.

[anon_73729884]

OK, I understand points 1 2 : in this case pragma alpha is not required because it’s not an alpha-based transparency and it’s not produced by the surf function.

But regarding the clipping, I tweaked the spotlight parameters and transform, I could not notice any aliasing…

However, I noticed that when I set the light color alpha under 32, the shader does not display anything (I’m pretty sure this is linked to the clip function, as it takes _LightColor0.a-1 as input). I’m still a little confused about why some pixels are shown with a transparency between 0 and 1 (vs. just totally visible pixels and totally transparent pixels, which produces aliasing).

[aubergine]

if it works, it aint broken. Then just dont bother

It is probably because the surface shader uses some blending operation and the result is the mixture of both blend and clip function after all.

Try looking at the compiled shader and see what happens there.

[anon_73729884]

OK, it must be some kind of blend as you say… I’ll maybe understand it when I’m more competent in shading
Anyway, thank you for your help !

[Orion]

I was looking for this kind of shader! Nice!
Although I do need it on particles and while it works fine on meshes, it doesn’t seem to show anything on particles. Do you know why that is?