Adding cutout transparency to a shader

Questions & Answers
1

So I have this “planet” shader. It works fine, but with how I model/texture eyes, I need to somehow make it work like a cutout shader.

I’ve tried finding how to do this but all I’ve ever found is full shaders, not what I need to add to it to support cutout transparency.

Here’s the shader I’m working with:

Shader "BumpSpecRim" {

  Properties {
      _MainTex ("Diffuse(RGB) Spec(A)", 2D) = "white" {}
      _BumpMap ("Bumpmap", 2D) = "bump" {}
      _RimColor ("Rim Color", Color) = (0.26,0.19,0.16,0.0)
      _RimPower ("Rim Power", Range(0.5,8.0)) = 3.0
      _SpecColor ("Specular Color", Color) = (0.5,0.5,0.5,1)
   _Shininess ("Shininess", Range (0.01, 1)) = 0.078125 
    }
    SubShader {
      Tags { "RenderType" = "Opaque" }
      CGPROGRAM
      
       #pragma surface surf SimpleSpecular
   float _Shininess;
   
      half4 LightingSimpleSpecular (SurfaceOutput s, half3 lightDir, half3 viewDir, half atten) {
          half3 h = normalize (lightDir + viewDir);
          half diff = max (0, dot (s.Normal, lightDir));
          float nh = max (0, dot (s.Normal, h));
          float spec = pow (nh, 48.0);
          half4 c;
          c.rgb = (s.Albedo * _LightColor0.rgb * diff + _LightColor0.rgb * spec * s.Alpha * _Shininess * _SpecColor) * (atten * 2);
          c.a = s.Alpha;
          return c;
      }
            
      struct Input {
          float2 uv_MainTex;
          float2 uv_BumpMap;
          float3 viewDir;
      };
      sampler2D _MainTex;
      sampler2D _BumpMap;
      float4 _RimColor;
      float _RimPower;
      
      void surf (Input IN, inout SurfaceOutput o) {
          o.Albedo = tex2D (_MainTex, IN.uv_MainTex).rgb;
          o.Normal = UnpackNormal (tex2D (_BumpMap, IN.uv_BumpMap));
          half rim = 1.0 - saturate(dot (normalize(IN.viewDir), o.Normal));
          o.Emission = _RimColor.rgb * pow (rim, _RimPower);
          o.Alpha = tex2D (_MainTex, IN.uv_MainTex).a;
      }
      ENDCG
    } 
    Fallback "Diffuse"
  }

So I just need to know what to add to have this support cutout transparency.

2

Add a texture to the shader that represents the Mask:

_Mask ("Mask", 2D) = "white" {}

All that matters in this mask is alpha transparency. full transparency for the areas that you want to hide.

Then in the surf function (as well as just above it):

sampler2D _Mask;

void surf (Input IN, inout SurfaceOutput o) {
       o.Albedo = tex2D (_MainTex, IN.uv_MainTex).rgb;           
       o.Normal = UnpackNormal (tex2D (_BumpMap, IN.uv_BumpMap));
       half rim = 1.0 - saturate(dot (normalize(IN.viewDir), o.Normal));
       o.Emission = _RimColor.rgb * pow (rim, _RimPower);
       o.Alpha = tex2D (_Mask, IN.uv_MainTex).a;
   }

There might be a slicker way to do this, but I think this will get you started. The idea is that we use the transparency from the Mask, not the main texture. But, the alpha is inverted for the mask, so the areas you want to see will be opaque in the mask.

If you need the main texture’s alpha as well, you could do something like:

sampler2D _Mask;

void surf (Input IN, inout SurfaceOutput o) {
       o.Albedo = tex2D (_MainTex, IN.uv_MainTex).rgb;           
       o.Normal = UnpackNormal (tex2D (_BumpMap, IN.uv_BumpMap));
       half rim = 1.0 - saturate(dot (normalize(IN.viewDir), o.Normal));
       o.Emission = _RimColor.rgb * pow (rim, _RimPower);

       float maskAlpha =  tex2D (_Mask, IN.uv_MainTex).a;           
       if(maskAlpha == 0) {
           o.Alpha = 0;
       } else {
           o.Alpha = tex2D (_Mask, IN.uv_MainTex).a;
       }
   }

In this version, we simply use the mask’s alpha as a boolean to decide if it should be shown at all or not. I do hear that you should avoid conditionals in shaders though as much as possible. A single if block should be fine though.