Why does my video alpha appear shiny with custom shader?

Questions & Answers
1

Hey team! I’ve built an AR tracking app and am beginning to implement 3D environments. When the app recognizes the image, the card becomes a window into a 3D room. (If you’re interested in some resources: here’s a great custom shader someone graciously shared that explains this antichamber effect).

SITUATION: I’ve ordered several 2D objects and have begun adding animated fires containing alpha channels. While I’ve been able to successfully make the alpha transparent, a shiny glow is reflected on each video plane. This makes the edges of the video planes visible and discolors all of the planes behind them. Below is a video of the effect in action.

Antichamber example

You’ll notices two lines near the top on the background that are a little darker (those are the video layers). They’re outlined in red below.

PROBLEM: I’m unsure how to get my custom shader to remove those lines so that the alpha channel on the animated fire videos are completely transparent. Utilizing the Standard shader (rendered to Cutout) will successfully remove this glare and restore the true color of the image (example below). However, it’s no use to me as it breaks the antichamber effect. The fire exists beyond my tracked object instead of existing within the “window.” I’d like to keep everything matted out.

QUESTION: I’ve struggled with multiple combinations of blend modes and passes to tweak the antichamber shader so that it will have the same alpha behavior as the “Cutout” render mode. How can I tweak this shader so that the alpha channel on the video layers remain fully transparent?

Here is the shader as it exists:

Upgrade NOTE: upgraded instancing buffer 'Props' to new syntax.

Shader "Custom/Stencil Object" {
    Properties {
        _Color ("Color", Color) = (1,1,1,1)
        _MainTex ("Albedo (RGB)", 2D) = "white" {}
        _Glossiness ("Smoothness", Range(0,1)) = 0.5
        _Metallic ("Metallic", Range(0,1)) = 0.0
        _RefNumber("Reference number", Int) = 1
        _Cutoff ("Alpha cutoff", Range(0,1)) = 0.5
    }
    SubShader {
        Tags {"RenderType"="Transparent" "ForceNoShadowCasting"="True"}
        LOD 200

        Stencil {
            Ref [_RefNumber]
            Comp equal
        }


        CGPROGRAM
        // Physically based Standard lighting model, and enable shadows on all light types
        #pragma surface surf Standard fullforwardshadows alpha


        // Use shader model 3.0 target, to get nicer looking lighting
        #pragma target 3.0

        sampler2D _MainTex;
        sampler2D _CutTex;

        struct Input {
            float2 uv_MainTex;
        };

        half _Glossiness;
        half _Metallic;
        fixed4 _Color;
        float _Cutoff;

        // Add instancing support for this shader. You need to check 'Enable Instancing' on materials that use the shader.
        // See https://docs.unity3d.com/Manual/GPUInstancing.html for more information about instancing.
        // #pragma instancing_options assumeuniformscaling
        UNITY_INSTANCING_BUFFER_START(Props)
            // put more per-instance properties here
        UNITY_INSTANCING_BUFFER_END(Props)



        void surf (Input IN, inout SurfaceOutputStandard o) {
            // Albedo comes from a texture tinted by color
            fixed4 c = tex2D (_MainTex, IN.uv_MainTex) * _Color;
            float ca = tex2D(_CutTex, IN.uv_MainTex).a;
            o.Albedo = c.rgb;
            // Metallic and smoothness come from slider variables
            o.Metallic = _Metallic;
            o.Smoothness = _Glossiness;
            o.Alpha = c.a;
        }
        ENDCG
    }
    FallBack "Diffuse"
}

Note: this question has also posted in the AR discussion forum.

2

SOLVED: change the optional parameter on line 24 from:

alpha
to decal:blend.