So I’m done trying to figure this out since there seems to be zero information about this anywhere on the web.
Does anyone know why setting a TilemapRenderer’s sorting layer to anything past default (or having it at default with an ‘Order in Layer’ greater than 0) just completely disables writing to the stencil buffer?
(if it’s not disabling it, it’s doing something weird to max out the Stencil ref values past 255, it would seem (because checking if the ‘Ref’ value is ‘Greater’ than 255 will make the pixels show)).
Edit with solution in quote (see below for shaders used):
- To get the effect working have some planes that cover all your tiles. (MAKE SURE THE PLANES’ LAYERS ARE SET TO ‘Ignore Raycast’)
- Apply the “Sprite Stencil Write” shader to your tilemap
- Set the stencil values on the tilemap material to | Stencil Ref = Anything above 0 (this should be unique per plane covering each tilemap) | Stencil Comparison = 8 (‘Always’ succeed) | Stencil Operation = 2 (‘Replace’ the value in the stencil buffer)
- Apply the “Unlit Color Stencil Read” shader to your planes
- Set the stencil values on your places to | Stencil Ref = The ‘Stencil Ref’ value on the material of the tilemap this plane is covering | Stencil Comparison = 3 (only draw the planes where the stencil buffer value is ‘Equal’ to the ‘Stencil Ref’ value on this material
- Finally if you want the fade effect just modify the alpha value (by changing the color) in a coroutine
IEnumerator ToggleFadeOverlay_cr(bool show, MeshRenderer planeMeshRenderer, float duration)
{
Color c = planeMeshRenderer.material.GetColor("_Color");
float time = show ? c.a : 1 - c.a;
while (time < 1)
{
time = Mathf.Clamp01(time + Time.deltaTime / duration);
planeMeshRenderer.material.SetColor("_Color", new Color(c.r, c.b, c.g, show ? Mathf.Lerp(0, 1, time) : Mathf.Lerp(1, 0, time)));
yield return null;
}
//Set the coroutine member variable to null, here, if you're tracking whether the coroutine is running not
}
- If you’re having issue with rendering order, drop the MeshRendererEditorOverride script from the solution quote in a folder called ‘Editor’ in your ‘Assets’ folder and set the “Order in Layer” value to something higher than what it is
Original Post
Some more info:
My shaders:
Writing to the stencil buffer -
Shader "Custom/Stencil/Sprite Stencil Write"
{
Properties
{
[PerRendererData] _MainTex ("Sprite Texture", 2D) = "white" {}
_Color ("Tint", Color) = (1,1,1,1)
[MaterialToggle] PixelSnap ("Pixel snap", Float) = 0
[HideInInspector] _RendererColor ("RendererColor", Color) = (1,1,1,1)
[HideInInspector] _Flip ("Flip", Vector) = (1,1,1,1)
[PerRendererData] _AlphaTex ("External Alpha", 2D) = "white" {}
[PerRendererData] _EnableExternalAlpha ("Enable External Alpha", Float) = 0
[IntSlider] _StencilRef ("Stencil Ref", Range(0,255)) = 0
[IntSlider] _StencilComparison ("Stencil Comparison", Range(1, 8)) = 3 //1 Never, 2 Less, 3 Equal, 4 LEqual, 5 Greater, 6 NotEqual, 7 GEqual, 8 Always
[IntSlider] _StencilOperation("Stencil Operation", Range(0,7)) = 0 //0 Keep, 1 Zero, 2 Replace, 3 IncrSar, 4 DecrSat, 5 Invert, 6 IncrWrap, 7 DecrWrap
}
SubShader
{
Tags
{
"Queue"="Transparent"
"IgnoreProjector"="True"
"RenderType"="Transparent"
"PreviewType"="Plane"
"CanUseSpriteAtlas"="True"
}
Cull Off
Lighting Off
ZWrite Off
Blend One OneMinusSrcAlpha
Pass
{
Stencil
{
Ref [_StencilRef]
Comp [_StencilComparison]
Pass [_StencilOperation]
}
CGPROGRAM
#pragma vertex SpriteVert
#pragma fragment frag
#pragma target 2.0
#pragma multi_compile_instancing
#pragma multi_compile _ PIXELSNAP_ON
#pragma multi_compile _ ETC1_EXTERNAL_ALPHA
#include "UnitySprites.cginc"
fixed4 frag(v2f IN) : SV_Target
{
fixed4 c = SampleSpriteTexture(IN.texcoord) * IN.color;
if (c.a < 0.00001) //This is here to prevent transparent pixels from writing to the stencil buffer
discard;
c.rgb *= c.a;
return c;
}
ENDCG
}
}
}
Reading from the stencil buffer -
Shader "Custom/Stencil/Unlit Color Stencil Read"
{
Properties
{
_Color("Color", Color) = (1,1,1,1)
[IntSlider] _StencilRef ("Stencil Ref", Range(0,255)) = 0
[IntSlider] _StencilComparison("Stencil Comparison", Range(1, 8)) = 3 //1 Never, 2 Less, 3 Equal, 4 LEqual, 5 Greater, 6 NotEqual, 7 GEqual, 8 Always
}
SubShader
{
Tags
{
"Queue" = "Transparent+10"
"IgnoreProjector" = "True"
"RenderType" = "Transparent"
"PreviewType" = "Plane"
"CanUseSpriteAtlas" = "True"
}
LOD 100
Stencil
{
Ref[_StencilRef]
Comp[_StencilComparison]
}
Pass
{
CGPROGRAM
#pragma vertex vert
#pragma fragment frag
// make fog work
#pragma multi_compile_fog
#include "UnityCG.cginc"
struct appdata
{
float4 vertex : POSITION;
};
struct v2f
{
UNITY_FOG_COORDS(1)
float4 vertex : SV_POSITION;
};
fixed4 _Color;
v2f vert (appdata v)
{
v2f o;
o.vertex = UnityObjectToClipPos(v.vertex);
UNITY_TRANSFER_FOG(o,o.vertex);
return o;
}
fixed4 frag (v2f i) : SV_Target
{
fixed4 col = _Color;
// apply fog
UNITY_APPLY_FOG(i.fogCoord, col);
return col;
}
ENDCG
}
}
}
-
Have some tiles in a tilemap.
-
Place the first shader as the tilemap material (in the TilemapRenderer on your tilemap gameobject)
-
Put the second shader on the plane that’s between the camera and the tilemap (facing the camera) and make sure the plane has all the tiles behind it.
-
Create a sorting layer below (lower down in the sorting layer list) ‘Default’ and assign this to your TilemapRenderer (or just set the “Order in Layer” on the TilemapRenderer to 1).
-
Now you’ll no longer be able to get the color from the second shader that’s on the plane to show up in front of the tiles; no matter what you change in the Stencil Buffers.
