please help with transparency issue (shadow casting/receiving sprite shader)

I need sprites to both cast and receive shadows, and I'm oh so close, as you can see in this test scene.

The multi-colored circle there is using my custom shader, and as you can see, it is receiving a shadow from the cylinder in front of it. It is also casting a shadow on the plane behind.

But the transparent areas of my sprites are doing weird things. In the image above, the purple sprite is a standard sprite (using the Sprites-Default shader) positioned behind my test sprite. But everywhere my test sprite (which is using "full rect" mode for its mesh) appears, the purple sprite behind it can't be seen... we see only the 3D plane and sky instead.

Moreover, the transparent areas are both casting and receiving shadows, which they should not do. And finally, I can't see the shadow on the plane behind the transparent areas.

If I switch my test sprite from "full rect" to "tight" mode, the problem is less apparent...

...but of course it's still there. And in my real game, I can't always use tight meshes. So I really need those transparent areas to be fully transparent — not cast shadows, not receive shadows, and not mess up the drawing of stuff behind them.

Here's my shader code.

// Unlit shader that nonetheless receives shadows.
// Adapted from: http://answers.unity3d.com/questions/1187379
Shader "Unlit/Custom/Unlit Transparent Shadow Receiver" {
     Properties {
         _Color ("Main Color", Color) = (1,1,1,1)
         _MainTex ("Base (RGB)", 2D) = "white" {}
     }
     SubShader {
         Tags {"Queue" = "Geometry" "IgnoreProjector"="True" "RenderType" = "Transparent"}
         Cull Off
         ZWrite On
         Blend SrcAlpha OneMinusSrcAlpha
// ------------------------------------------------------------------
//  Shadow rendering pass
Pass {
    Name "ShadowCaster"
    Tags { "LightMode" = "ShadowCaster" }

    ZWrite On ZTest LEqual Cull Off

    CGPROGRAM
    #pragma target 2.0

    #pragma shader_feature _ _ALPHATEST_ON _ALPHABLEND_ON _ALPHAPREMULTIPLY_ON
    #pragma skip_variants SHADOWS_SOFT
    #pragma multi_compile_shadowcaster

    #pragma vertex vertShadowCaster
    #pragma fragment fragShadowCaster

    #include "UnityStandardShadow.cginc"

    ENDCG
}
         Pass {
             Tags { "LightMode" = "ForwardBase" }    // handles main directional light, vertex/SH lights, and lightmaps
             CGPROGRAM
                 #pragma vertex vert
                 #pragma fragment frag
                 #pragma multi_compile_fwdbase
                 #pragma fragmentoption ARB_fog_exp2
                 #pragma fragmentoption ARB_precision_hint_fastest

                 #include "UnityCG.cginc"
                 #include "AutoLight.cginc"

                 struct v2f
                 {
                     float4    pos            : SV_POSITION;
                     float2    uv            : TEXCOORD0;
                     LIGHTING_COORDS(1,2)
                 };
                 float4 _MainTex_ST;
                 v2f vert (appdata_tan v)
                 {
                     v2f o;

                     o.pos = mul( UNITY_MATRIX_MVP, v.vertex);
                     o.uv = TRANSFORM_TEX (v.texcoord, _MainTex).xy;
                     TRANSFER_VERTEX_TO_FRAGMENT(o);
                     return o;
                 }
                 sampler2D _MainTex;
                 fixed4 frag(v2f i) : COLOR
                 {
                     //fixed atten = LIGHT_ATTENUATION(i);    // Light attenuation + shadows.
                     fixed atten = SHADOW_ATTENUATION(i); // Shadows ONLY.
                     fixed4 tex = tex2D(_MainTex, i.uv);
                     fixed4 c = tex * atten;    // attenuate color
                     c.a = tex.a;                // don't attenuate alpha
                     return c;
                 }
             ENDCG
         }
          Pass {
             Tags {"LightMode" = "ForwardAdd"}    // handles per-pixel additive lights (called once per such light)
             Blend One One
             CGPROGRAM
                 #pragma vertex vert
                 #pragma fragment frag
                 #pragma multi_compile_fwdadd_fullshadows
                 #pragma fragmentoption ARB_fog_exp2
                 #pragma fragmentoption ARB_precision_hint_fastest

                 #include "UnityCG.cginc"
                 #include "AutoLight.cginc"

                 struct v2f
                 {
                     float4    pos            : SV_POSITION;
                     float2    uv            : TEXCOORD0;
                     LIGHTING_COORDS(1,2)
                 };
                 float4 _MainTex_ST;
                 v2f vert (appdata_tan v)
                 {
                     v2f o;

                     o.pos = mul( UNITY_MATRIX_MVP, v.vertex);
                     o.uv = TRANSFORM_TEX (v.texcoord, _MainTex).xy;
                     TRANSFER_VERTEX_TO_FRAGMENT(o);
                     return o;
                 }
                 sampler2D _MainTex;
                 fixed4 frag(v2f i) : COLOR
                 {
                    // fixed atten = LIGHT_ATTENUATION(i);    // Light attenuation + shadows.
                     fixed atten = SHADOW_ATTENUATION(i); // Shadows ONLY.
                     fixed4 tex = tex2D(_MainTex, i.uv);
                     fixed4 c = tex * atten;    // attenuate color
                     c.a = tex.a;                // don't attenuate alpha
                     return c;
                 }
             ENDCG
         }
     }
     FallBack "VertexLit"
}

I'm hoping it's something simple — some way to check the alpha bail out without any effect at all, even on the depth buffer. I'm sure the problem is somewhere in this pass:

...but I just don't see how to fix it. Any pointers would be greatly appreciated.

Your shadowcaster pass is using this line:

#pragma shader_feature _ _ALPHATEST_ON _ALPHABLEND_ON _ALPHAPREMULTIPLY_ON

That tells a shader that it needs to compile a new variant if one of those keywords is enabled on the shader, but it doesn't actually enable or disable any of them. So instead that shadow caster pass is just compiling the default opaque version as nothing in the shader properties are allowing you to toggle on any of those keywords, and I doubt you're enabling it via script.

So you can enable alpha testing if you remove that line and replace it with:

#define _ALPHATEST_ON

And now it ... still won't work as you're missing a required property, _Cutoff, so add this to your properties:

[HideInInspector] _Cutoff ("", Float) = 0.5

It should also be noted there's really no need to add the custom shadowcaster pass, the real fix should be use the correct fallback shader. (You'll still need the _Cutoff).

Fallback "Legacy Shaders/Transparent/Cutout/VertexLit"

2 Likes

First, thanks so much for replying — this stuff is still very arcane to me.

These changes are definitely some progress. The shadows now cast (and are received) properly, on only the areas where the alpha is past the cutoff. But the remaining transparent areas are still somehow mucking up drawing of stuff behind it.

Stepping through the rendering with the Frame Debugger, I no longer see anything suspicious on step 3, or any other step, except the very last one — "Draw Dynamic" in the Render.TransparentGeometry pass — where the purple sprite is drawn, and it simply doesn't draw in the area of my custom sprite. But nothing in the previous steps gives me any clue why that would be.

Here's the revised shader code.

// Unlit shader that nonetheless receives shadows.
// Adapted from: http://answers.unity3d.com/questions/1187379
// With help from: http://forum.unity3d.com/threads/477358/
Shader "Unlit/Custom/Unlit Transparent Shadow Receiver" {
     Properties {
        _Color ("Main Color", Color) = (1,1,1,1)
        _MainTex ("Base (RGB)", 2D) = "white" {}
        _Cutoff ("Cutoff", Float) = 0.5
     }
     SubShader {
         Tags {"Queue" = "Geometry" "IgnoreProjector"="True" "RenderType" = "Transparent"}
         Cull Off
         ZWrite On
         Blend SrcAlpha OneMinusSrcAlpha
         Pass {
             Tags { "LightMode" = "ForwardBase" }    // handles main directional light, vertex/SH lights, and lightmaps
             CGPROGRAM
                 #pragma vertex vert
                 #pragma fragment frag
                 #pragma multi_compile_fwdbase
                 #pragma fragmentoption ARB_fog_exp2
                 #pragma fragmentoption ARB_precision_hint_fastest

                 #include "UnityCG.cginc"
                 #include "AutoLight.cginc"

                 struct v2f
                 {
                     float4    pos            : SV_POSITION;
                     float2    uv            : TEXCOORD0;
                     LIGHTING_COORDS(1,2)
                 };
                 float4 _MainTex_ST;
                 v2f vert (appdata_tan v)
                 {
                     v2f o;

                     o.pos = mul( UNITY_MATRIX_MVP, v.vertex);
                     o.uv = TRANSFORM_TEX (v.texcoord, _MainTex).xy;
                     TRANSFER_VERTEX_TO_FRAGMENT(o);
                     return o;
                 }
                 sampler2D _MainTex;
                 fixed4 frag(v2f i) : COLOR
                 {
                     //fixed atten = LIGHT_ATTENUATION(i);    // Light attenuation + shadows.
                     fixed atten = SHADOW_ATTENUATION(i); // Shadows ONLY.
                     fixed4 tex = tex2D(_MainTex, i.uv);
                     fixed4 c = tex * atten;    // attenuate color
                     c.a = tex.a;                // don't attenuate alpha
                     return c;
                 }
             ENDCG
         }
          Pass {
             Tags {"LightMode" = "ForwardAdd"}    // handles per-pixel additive lights (called once per such light)
             Blend One One
             CGPROGRAM
                 #pragma vertex vert
                 #pragma fragment frag
                 #pragma multi_compile_fwdadd_fullshadows
                 #pragma fragmentoption ARB_fog_exp2
                 #pragma fragmentoption ARB_precision_hint_fastest

                 #include "UnityCG.cginc"
                 #include "AutoLight.cginc"

                 struct v2f
                 {
                     float4    pos            : SV_POSITION;
                     float2    uv            : TEXCOORD0;
                     LIGHTING_COORDS(1,2)
                 };
                 float4 _MainTex_ST;
                 v2f vert (appdata_tan v)
                 {
                     v2f o;

                     o.pos = mul( UNITY_MATRIX_MVP, v.vertex);
                     o.uv = TRANSFORM_TEX (v.texcoord, _MainTex).xy;
                     TRANSFER_VERTEX_TO_FRAGMENT(o);
                     return o;
                 }
                 sampler2D _MainTex;
                 fixed4 frag(v2f i) : COLOR
                 {
                    // fixed atten = LIGHT_ATTENUATION(i);    // Light attenuation + shadows.
                     fixed atten = SHADOW_ATTENUATION(i); // Shadows ONLY.
                     fixed4 tex = tex2D(_MainTex, i.uv);
                     fixed4 c = tex * atten;    // attenuate color
                     c.a = tex.a;                // don't attenuate alpha
                     return c;
                 }
             ENDCG
         }
     }
    Fallback "Legacy Shaders/Transparent/Cutout/VertexLit"
}

Any idea what might still be awry?

Thanks again,
- Joe

I think I've got it! :smile:

The secret was to use the clip function in the fragment function, to bail out entirely on any pixels whose alpha is below the cutoff value. Documentation on this little trick appears to be weak, at least in the context of Unity, but my google-fu today was strong. ;)

So finally I got the desired result:

...and, here's the revised shader code:

// Unlit shader that nonetheless receives shadows.
// Adapted from: http://answers.unity3d.com/questions/1187379
// With help from: http://forum.unity3d.com/threads/477358/
Shader "Unlit/Custom/Unlit Cutoff Shadow Savvy" {
Properties {
        _Color ("Main Color", Color) = (1,1,1,1)
        _MainTex ("Base (RGB)", 2D) = "white" {}
        _Cutoff ("Cutoff", Float) = 0.5
    }
    SubShader {
        Tags {"Queue" = "Geometry" "IgnoreProjector"="True" "RenderType" = "Transparent"}
        Cull Off
        ZWrite On
        Blend SrcAlpha OneMinusSrcAlpha

        Pass {
            Tags { "LightMode" = "ForwardBase" }    // handles main directional light, vertex/SH lights, and lightmaps
            CGPROGRAM
            #pragma vertex vert
            #pragma fragment frag
            #pragma multi_compile_fwdbase
            #pragma fragmentoption ARB_fog_exp2
            #pragma fragmentoption ARB_precision_hint_fastest

            #include "UnityCG.cginc"
            #include "AutoLight.cginc"

            float4 _MainTex_ST;
            sampler2D _MainTex;
            float _Cutoff;

            struct v2f {
                float4    pos            : SV_POSITION;
                float2    uv            : TEXCOORD0;
                LIGHTING_COORDS(1,2)
            };

            v2f vert (appdata_tan v) {
                v2f o;

                o.pos = mul( UNITY_MATRIX_MVP, v.vertex);
                o.uv = TRANSFORM_TEX (v.texcoord, _MainTex).xy;
                TRANSFER_VERTEX_TO_FRAGMENT(o);
                return o;
            }

            fixed4 frag(v2f i) : COLOR {
                fixed4 tex = tex2D(_MainTex, i.uv);

                // If alpha is less than _Cutoff, then bail out on this pixel entirely
                // (don't even write to the depth buffer).
                clip(tex.a - _Cutoff);

                fixed atten = SHADOW_ATTENUATION(i); // Attenuation for shadows ONLY.
                fixed4 c = tex * atten;        // attenuate color
                c.a = tex.a;                // don't attenuate alpha
                return c;
            }
            ENDCG
        }
        Pass {
            Tags {"LightMode" = "ForwardAdd"}    // handles per-pixel additive lights (called once per such light)
            Blend One One
            CGPROGRAM
            #pragma vertex vert
            #pragma fragment frag
            #pragma multi_compile_fwdadd_fullshadows
            #pragma fragmentoption ARB_fog_exp2
            #pragma fragmentoption ARB_precision_hint_fastest

            #include "UnityCG.cginc"
            #include "AutoLight.cginc"

            struct v2f {
                float4    pos            : SV_POSITION;
                float2    uv            : TEXCOORD0;
                LIGHTING_COORDS(1,2)
            };

            float4 _MainTex_ST;
            sampler2D _MainTex;
            float _Cutoff;

            v2f vert (appdata_tan v) {
                v2f o;

                o.pos = mul( UNITY_MATRIX_MVP, v.vertex);
                o.uv = TRANSFORM_TEX (v.texcoord, _MainTex).xy;
                TRANSFER_VERTEX_TO_FRAGMENT(o);
                return o;
            }

            fixed4 frag(v2f i) : COLOR {
                fixed4 tex = tex2D(_MainTex, i.uv);

                // If alpha is less than _Cutoff, then bail out on this pixel entirely
                // (don't even write to the depth buffer).
                clip(tex.a - _Cutoff);

                fixed atten = SHADOW_ATTENUATION(i); // Attenuation for shadows ONLY.
                fixed4 c = tex * atten;        // attenuate color
                c.a = tex.a;                // don't attenuate alpha
                return c;

            }
            ENDCG
        }
    }
    Fallback "Legacy Shaders/Transparent/Cutout/VertexLit"
}

If anybody else needs sprites that cast and receive shadows, here you go! And of course if anybody can spot any further improvements to this shader, please don't be shy about saying so.

Thank you @bgolus for getting me unstuck today!

Oops. Not quite all happy now. There is still one issue remaining:

Shadows don't appear on my sprites when they are flipped.

They still cast shadows just fine (assuming I've set the renderer to cast double-sided shadows). But they don't receive shadows.

I'm hoping there is a simple shader property or pragma I can set that might fix this... but I already have "Cull off"... so I'm not sure what to try. Any ideas?

It is a simple as Cull Off, but only if you have a shadow caster pass in your shader. If you copy the code from the cutout vertex lit shader and add Cull Off it should fix the problem.

Shadow caster pass? Even though my problem is in receiving shadows, and not in casting them?

Yes. The main directional light’s shadows are actually “received” by the camera depth texture rather than during the forward pass. The camera depth texture is created using the shadow caster pass. In Unity 4 there were separate shadow caster and shadow receiver passes, but they’ve been merged into one for Unity 5.

1 Like

Awesome, thanks for the explanation.

Again in the spirit of giving back to the community, here's my updated shader... both casts and receives shadows, even when flipped.

// Unlit shader that nonetheless receives shadows.
// Adapted from: http://answers.unity3d.com/questions/1187379
// With help from: http://forum.unity3d.com/threads/477358/
Shader "Unlit/Custom/Unlit Cutoff Shadow Savvy" {
Properties {
        _Color ("Main Color", Color) = (1,1,1,1)
        _MainTex ("Base (RGB)", 2D) = "white" {}
        _Cutoff ("Cutoff", Float) = 0.5
    }
    SubShader {
        Tags {"Queue" = "Geometry" "IgnoreProjector"="True" "RenderType" = "Transparent"}
        Cull Off
        ZWrite On
        Blend SrcAlpha OneMinusSrcAlpha

        Pass {
            Tags { "LightMode" = "ForwardBase" }    // handles main directional light, vertex/SH lights, and lightmaps
            CGPROGRAM
            #pragma vertex vert
            #pragma fragment frag
            #pragma multi_compile_fwdbase
            #pragma fragmentoption ARB_fog_exp2
            #pragma fragmentoption ARB_precision_hint_fastest

            #include "UnityCG.cginc"
            #include "AutoLight.cginc"

            float4 _MainTex_ST;
            sampler2D _MainTex;
            float _Cutoff;

            struct v2f {
                float4    pos            : SV_POSITION;
                float2    uv            : TEXCOORD0;
                LIGHTING_COORDS(1,2)
            };

            v2f vert (appdata_tan v) {
                v2f o;

                o.pos = UnityObjectToClipPos( v.vertex);
                o.uv = TRANSFORM_TEX (v.texcoord, _MainTex).xy;
                TRANSFER_VERTEX_TO_FRAGMENT(o);
                return o;
            }

            fixed4 frag(v2f i) : COLOR {
                fixed4 tex = tex2D(_MainTex, i.uv);

                // If alpha is less than _Cutoff, then bail out on this pixel entirely
                // (don't even write to the depth buffer).
                clip(tex.a - _Cutoff);

                fixed atten = SHADOW_ATTENUATION(i); // Attenuation for shadows ONLY.
                fixed4 c = tex * atten;        // attenuate color
                c.a = tex.a;                // don't attenuate alpha
                return c;
            }
            ENDCG
        }
        Pass {
            Tags {"LightMode" = "ForwardAdd"}    // handles per-pixel additive lights (called once per such light)
            Blend One One
            CGPROGRAM
            #pragma vertex vert
            #pragma fragment frag
            #pragma multi_compile_fwdadd_fullshadows
            #pragma fragmentoption ARB_fog_exp2
            #pragma fragmentoption ARB_precision_hint_fastest

            #include "UnityCG.cginc"
            #include "AutoLight.cginc"

            struct v2f {
                float4    pos            : SV_POSITION;
                float2    uv            : TEXCOORD0;
                LIGHTING_COORDS(1,2)
            };

            float4 _MainTex_ST;
            sampler2D _MainTex;
            float _Cutoff;

            v2f vert (appdata_tan v) {
                v2f o;

                o.pos = UnityObjectToClipPos( v.vertex);
                o.uv = TRANSFORM_TEX (v.texcoord, _MainTex).xy;
                TRANSFER_VERTEX_TO_FRAGMENT(o);
                return o;
            }

            fixed4 frag(v2f i) : COLOR {
                fixed4 tex = tex2D(_MainTex, i.uv);

                // If alpha is less than _Cutoff, then bail out on this pixel entirely
                // (don't even write to the depth buffer).
                clip(tex.a - _Cutoff);

                fixed atten = SHADOW_ATTENUATION(i); // Attenuation for shadows ONLY.
                fixed4 c = tex * atten;        // attenuate color
                c.a = tex.a;                // don't attenuate alpha
                return c;

            }
            ENDCG
        }

        // Pass to render object as a shadow caster (and receiver)
        Pass {
            Name "Caster"
            Tags { "LightMode" = "ShadowCaster" }

            CGPROGRAM
            #pragma vertex vert
            #pragma fragment frag
            #pragma target 2.0
            #pragma multi_compile_shadowcaster
            #pragma multi_compile_instancing // allow instanced shadow pass for most of the shaders
            #include "UnityCG.cginc"

            struct v2f {
                V2F_SHADOW_CASTER;
                float2  uv : TEXCOORD1;
                UNITY_VERTEX_OUTPUT_STEREO
            };

            uniform float4 _MainTex_ST;

            v2f vert( appdata_base v )
            {
                v2f o;
                UNITY_SETUP_INSTANCE_ID(v);
                UNITY_INITIALIZE_VERTEX_OUTPUT_STEREO(o);
                TRANSFER_SHADOW_CASTER_NORMALOFFSET(o)
                o.uv = TRANSFORM_TEX(v.texcoord, _MainTex);
                return o;
            }

            uniform sampler2D _MainTex;
            uniform fixed _Cutoff;
            uniform fixed4 _Color;

            float4 frag( v2f i ) : SV_Target
            {
                fixed4 texcol = tex2D( _MainTex, i.uv );
                clip( texcol.a*_Color.a - _Cutoff );

                SHADOW_CASTER_FRAGMENT(i)
            }
            ENDCG

        }

    }
    Fallback "Legacy Shaders/Transparent/Cutout/VertexLit"
}
1 Like

That clip(tex.a - _Cutoff) thing just fixed an unrelated problem for me that was driving me crazy. Thanks for posting your code.

Thank you JoeStrout for sharing this shader. It works beautifully with 1 directional light. However, when I have 2 or more directional lights, the sprite actually starts to light up.

Below is an example with 1 directional light from the front, left. The sprite is rendering at its native brightness (i.e. it appears unlit) with correct shadows.

Below is the same as above except with a second directional light in the back. The shadows are rendering properly, but the sprite now appears much brighter.

Below is the same as above except with a third directional light added. The shadows are rendering properly, but the sprite now appears way brighter and is almost blown out in the back.

Can you help me to pinpoint the shader code that is causing this? Presumably it's in the "Tags {"LightMode" = "ForwardAdd"}" section of the shader.

Thanks for any help you can provide!
Dan

FYI: I found a bit of a workaround for now.

I replaced both instances of this line:
fixed4 tex = tex2D(_MainTex, i.uv);

with:
fixed4 tex = tex2D(_MainTex, i.uv) * _Color;

Now, I can use the shader's Main Color slider to turn down the brightness so that it matches its unlit level.

Hope this helps,
Dan

1 Like

Hello, i can use it it get me this error: " Parse error: syntax error, unexpected $end, expecting TOK_SHADER " can anybody help?(((