Hi folks 
i´m working on a rts game and i am trying to do fog of war. i followed an tutorial where you create some shaders and do some sort of stencil thing. i got an quad in front of my camera that has an shader on that darkens the image. next i got an object in front of the quad with a shader on that blocks the other shader. making it appear. my issue here is that i tried to add alpha texture features to my stencil shader, making my object appear in any shape i choose.
here´s my shader code, i have no idea why it´s not working. i don´t really know much about how shaders work so i tried my bedst.
Shader "Custom/Stencil/Mask OneZLess"
{
Properties
{
_AlphaTex ("Alpha (A)", 2D) = "white" {}
_Cutoff ("Alpha cutoff",float) = 0.5
}
SubShader
{
Tags { "RenderType"="Tranparency" "Queue"="Geometry-1" "IgnoreProjector"="No"}
ColorMask 0
ZWrite off
Stencil
{
Ref 1
Comp always
Pass replace
}
Pass
{
Cull Back
ZTest Less
CGPROGRAM
#pragma vertex vert
#pragma fragment frag
uniform sampler2D _AlphaTex;
uniform float _Cutoff;
struct VertexInPut
{
float4 vertex : POSITION;
float4 texcoord : TEXCOORDO;
};
struct VertexOutPut
{
float4 pos : SV_POSITION;
float4 tex : TEXCOORDO;
};
VertexOutPut vert(VertexInPut input)
{
VertexOutPut output;
output.tex = VertexInPut.texcoord;
output.pos = mul(UNITY_MATRIX_MVP, input.vertex);
return output;
}
float4 frag(VertexOutPut input) : COLOR
{
float4 aTex = tex2D(_AlphaTex,input.tex.xy);
if(aTex.a <_Cutoff)
{
discard;
}
return aTex;
return float4(1,1,0,1);
}
ENDCG
}
}
}
Hm there are a few problems here:
- Tell us what the behaviour of the shader right now is and what it should be.
- You labeled the Thread Problems with Z-Buffer Shader. Even though I guess that this shader won’t write in the depth buffer I don’t think that this is really the problem you want to tackle.
- With the stencil shader you are writing in the stencil buffer. The values written in there can be used by other shaders to do simple conditional operations. As for my knowledge you can’t do a half transparency clipping effect with stencil buffers. What you could do is either fade the shader you just posted from something to transparent, or as it seems you have already chosen: Do a cutout shader.
Cutout shaders are quite different to transparent shaders. The only allow full transparency or no transparency for every pixel, nothing in between. If you want to go with a cutout shader, take a look at the cutout shaders from Unity itself. Here you can download the newest ones: http://download.unity3d.com/download_unity/38b4efef76f0/builtin_shaders-5.5.0f3.zip
Everything with alphatest might be interesting for you (e.g. DefaultResourcesExtra/Alpha-VertexLit.shader)´.
From a quick look at the shader:
The tags should be
Tags { “Queue”=“AlphaTest” “IgnoreProjector”=“True” “RenderType”=“TransparentCutout”}
In the frag you want to return twice?! The last one won’t be executed.
The frag should look more like this:
fixed4 frag(VertexOutPut input) : COLOR
{
fixed4 aTex = tex2D(_AlphaTex,input.tex.xy);
clip(aTex.a - _Cutoff);
return aTex;
}
Hi, thank you for your reply
well at the moment my shader goes complete pink. and i throws me an Token error, like in the pic.
it keeps saying unexpected token in line 53 and i have no idea why.
i changed my code like you said, however i can´t tell if it made any difference since it didn´t resolve my error.
to clarify what i am trying to do. this stencil shader will sort of cut out the geometry that is attached to the shader.
below here is the stencil fog shader that makes everything dark.
Shader "Custom/Transparent stencil_fog" {
Properties {
_Color ("Color", Color) = (1,1,1,1)
_MainTex ("Albedo (RGB)", 2D) = "white" {}
}
SubShader {
Tags { "RenderType"="Transparent" "Queue"="Transparent" "IgnoreProjector"="True"}
LOD 200
Stencil
{
Ref 1
Comp notequal
pass keep
}
CGPROGRAM
#pragma surface surf Standard alpha
sampler2D _MainTex;
fixed4 _Color;
struct Input
{
float2 uv_MainTex;
};
void surf (Input IN, inout SurfaceOutputStandard o) {
fixed4 c = tex2D (_MainTex, IN.uv_MainTex) * _Color;
o.Albedo = c.rgb;
o.Alpha = c.a;
}
ENDCG
}
FallBack "VertexLit"
the other shader (in which i got my errors) acts as an simple mask that removes the dark pixels. i was simply trying to add a texture feature so that i could mask out parts of my geometry to make other shapes than spheres and cubes etc.
below is the mask shader that i have modified based on your initial feedbag.
Shader "Custom/Stencil/Mask OneZLess"
{
Properties
{
_AlphaTex ("Alpha (A)", 2D) = "white" {}
_Cutoff ("Alpha cutoff",float) = 0.5
}
SubShader
{
Tags { "RenderType"="TransparentCutout" "Queue"="AlphaTest" "IgnoreProjector"="True"}
ColorMask 0
ZWrite off
Stencil
{
Ref 1
Comp always
Pass replace
}
Pass
{
Cull Back
ZTest Less
CGPROGRAM
#pragma vertex vert
#pragma fragment frag
uniform sampler2D _AlphaTex;
uniform float _Cutoff;
struct VertexInPut
{
float4 vertex : POSITION;
float4 texcoord : TEXCOORDO;
};
struct VertexOutPut
{
float4 pos : SV_POSITION;
float4 tex : TEXCOORDO;
};
VertexOutPut vert(VertexInPut input)
{
VertexOutPut output;
output.tex = VertexInPut.texcoord;
output.pos = mul(UNITY_MATRIX_MVP, input.vertex);
return output;
}
fixed4 frag(VertexOutPut input) : COLOR
{
fixed4 aTex = tex2D(_AlphaTex,input.tex.xy);
clip(aTex.a - _Cutoff);
return aTex;
}
ENDCG
}
}
}
i still got the unexpected token error, what could be wrong ?
Hey there,
you got three small typos in there (Sadly small typos are not acceptable in shaders 
)
At line 51 you tried to access the uv texture coordinates by calling VertexInPut.texcoord, but actually you need to get the data not from the data class but instance, so calling input.texcoord is the way to do it.
The other two typos are in your structs at line 39 and 45. It is not TEXCOORDO, but rather TEXCOORD0.
Here is the corrected shader (which does compile, I didn’t check it for functionality):
Shader "Custom/Stencil/Mask OneZLess"
{
Properties
{
_AlphaTex("Alpha (A)", 2D) = "white" {}
_Cutoff("Alpha cutoff",float) = 0.5
}
SubShader
{
Tags{ "RenderType" = "TransparentCutout" "Queue" = "AlphaTest" "IgnoreProjector" = "True" }
ColorMask 0
ZWrite off
Cull Back
ZTest Less
Stencil
{
Ref 1
Comp always
Pass replace
}
Pass
{
CGPROGRAM
#pragma vertex vert
#pragma fragment frag
uniform sampler2D _AlphaTex;
uniform float _Cutoff;
struct VertexInPut
{
float4 vertex : POSITION;
float4 texcoord : TEXCOORD0;
};
struct VertexOutPut
{
float4 pos : SV_POSITION;
float4 tex : TEXCOORD0;
};
VertexOutPut vert(VertexInPut input)
{
VertexOutPut output;
output.tex = input.texcoord;
output.pos = mul(UNITY_MATRIX_MVP, input.vertex);
return output;
}
fixed4 frag(VertexOutPut input) : COLOR
{
fixed4 aTex = tex2D(_AlphaTex,input.tex.xy);
clip(aTex.a - _Cutoff);
return aTex;
}
ENDCG
}
}
}
ah now it works !
thank you so much. it´s clear that i seriously need to learn how to code shaders. most of the stuff in there is in fact copy paste from an tutorial. i would love to learn how to write shaders but it´s difficult to find any good understandable tutorials. do you know any good way to learn it ? also i find it strange that there are no autocompletion when writing shaders.
but thank you very much for your help
Oh I feel you, the tools you can use for shader programming are lacking a lot of functionality. There is an extension for Visual Studio (Shader Unity Support - Visual Studio Marketplace) that might be interesting for you. Autocompletion is in there (ctrl + Space), so I guess it is a start. As for learning shaders, I really like to reffer to the unity shader examples to see how they did it (Surface Shaders, Vertex/Fragment Shaders).
