How do I create an invisible mask that hides the 2nd closest object behind it?

Questions & Answers
1

Let’s say we’re in 2D view and there are 3 objects that have different z position values and partially overlap each other: InvisibleMask, ObjectA, and ObjectB. InvisibleMask is closest to the camera, then ObjectA, and then ObjectB. If I want to use InvisibleMask to only hide ObjectB where InivisibleMask overlaps ObjectB and not hide ObjectA at all, I can set the renderqueue of ObjectB to be higher than that of InvisibleMask while setting the renderqueue of ObjectA to be equal to or less than that of InvisibleMask. However, by doing so, ObjectB will appear in front of ObjectA where InvisibleMask does not overlap ObjectB, even though ObjectA is really in front of ObjectB. ObjectB now has a higher renderqueue than ObjectA, but I don’t understand why this would cause ObjectB to appear in front of ObjectA when ObjectB, which also has a higher renderqueue than InvisibleMask, does not appear in front of InvisibleMask.

InvisibleMask has to be closest to the camera for reasons, so given this constraint, is what I’m trying to do even possible? It seems that I have to somehow give the same renderqueue to both ObjectA and ObjectB except for where InvisibleMask overlaps ObjectB.

2

Hmm, so I think @BoredMormon is asking the right question, on second read this just seems like any other masking:

Mask shader:

Shader "Custom/mask" {
	Properties {
		_Color ("Main Color", Color) = (1,1,1,1)
		_MainTex ("Base (RGB) Trans (A)", 2D) = "white" {}
	}
	SubShader {
		Tags {"Queue"="Geometry+2" "IgnoreProjector"="True" "RenderType"="Transparent"}
		LOD 200

		// extra pass that renders to depth buffer only
		Pass {
			ZWrite On
			ColorMask 0
		}

		// paste in forward rendering passes from Transparent/Diffuse
		UsePass "Transparent/Diffuse/FORWARD"
	}
	Fallback "Transparent/VertexLit"
}

ObjectA shader:

Shader "Custom/first" {
	Properties {
		_MainTex ("Base (RGB)", 2D) = "white" {}
	}
	SubShader {
		Tags { "Queue" = "Geometry+1" "RenderType"="Opaque" }
		LOD 200
		
		CGPROGRAM
		#pragma surface surf Lambert

		sampler2D _MainTex;

		struct Input {
			float2 uv_MainTex;
		};

		void surf (Input IN, inout SurfaceOutput o) {
			half4 c = tex2D (_MainTex, IN.uv_MainTex);
			o.Albedo = c.rgb;
			o.Alpha = c.a;
		}
		ENDCG
	} 
	FallBack "Diffuse"
}

ObjectB shader:

Shader "Custom/second" {
	Properties {
		_MainTex ("Base (RGB)", 2D) = "white" {}
	}
	SubShader {
		Tags { "Queue" = "Geometry+3" "RenderType"="Opaque" }
		LOD 200
		
		CGPROGRAM
		#pragma surface surf Lambert

		sampler2D _MainTex;

		struct Input {
			float2 uv_MainTex;
		};

		void surf (Input IN, inout SurfaceOutput o) {
			half4 c = tex2D (_MainTex, IN.uv_MainTex);
			o.Albedo = c.rgb;
			o.Alpha = c.a;
		}
		ENDCG
	} 
	FallBack "Diffuse"
}

This seems to work fine for me, I just set the mask material color to have 0 alpha and it’s fine!