Render object behind others with ZTest Greater, but ignore self

Unity Engine
1

I’m attempting to make a shader (that is an additional pass on the Standard Shader) that will render an unlit silhouette when the object is occluded by others. Here is my current shader and an image of it.

Shader "Custom/StandardOccluded" {
    Properties {
        _Color ("Color", Color) = (1,1,1,1)
        _MainTex ("Albedo (RGB)", 2D) = "white" {}
        _Glossiness ("Smoothness", Range(0,1)) = 0.5
        _Metallic ("Metallic", Range(0,1)) = 0.0
        _OccludedColor("Occluded Color", Color) = (1,1,1,1)
    }
    SubShader {

        Tags { "RenderType"="Opaque" }
        LOD 200
        ZTest LEqual
       
        CGPROGRAM
        // Physically based Standard lighting model, and enable shadows on all light types
        #pragma surface surf Standard fullforwardshadows

        // Use shader model 3.0 target, to get nicer looking lighting
        #pragma target 3.0

        sampler2D _MainTex;

        struct Input {
            float2 uv_MainTex;
        };

        half _Glossiness;
        half _Metallic;
        fixed4 _Color;

        void surf (Input IN, inout SurfaceOutputStandard o) {
            // Albedo comes from a texture tinted by color
            fixed4 c = tex2D (_MainTex, IN.uv_MainTex) * _Color;
            o.Albedo = c.rgb;
            // Metallic and smoothness come from slider variables
            o.Metallic = _Metallic;
            o.Smoothness = _Glossiness;
            o.Alpha = c.a;
        }
        ENDCG

        Pass
        {
            ZTest Greater
            Cull Back

            CGPROGRAM
            #pragma vertex vert            
            #pragma fragment frag

            half4 _OccludedColor;

            float4 vert(float4 pos : POSITION) : SV_POSITION
            {
                float4 viewPos = mul(UNITY_MATRIX_MVP, pos);
                return viewPos;
            }

                half4 frag(float4 pos : SV_POSITION) : COLOR
            {
                return _OccludedColor;
            }

            ENDCG
        }
    }
    FallBack "Diffuse"
}

734×406 78.5 KB

As you can see it’s pretty baller, but the issue is that it’s rendering the pink parts when the object occludes itself. It makes sense that this would be the expected behaviour, but ideally I’d want to only render the silhouette when it’s being occluded by objects other than itself. Is this possible?

Thanks,
Erik

1 Like
2

There are several possible solutions.

Offset -
Push the outline geometry towards the camera some amount to reduce self overdraw. This is a pretty common method and easy to implement, but hard to get just right, especially on wide objects. Also will cause parts of it to not draw it too close to a wall as its getting pushed through it.

Stencil -
Stencils are a great option here, but won’t be an option if you’re using deferred rendering, and can be tricky to setup if you’re not familiar with them. The short version is you have your object write to a specific stencil mask, then the silhouette pass only draws where that mask isn’t. An advantage of this technique is if you later decide you want your silhouette to render over transparent objects that render later it’s a lot easier to get working.

Draw the silhouette first -
This is by far the simplest method. Just put the silhouette pass before the surface shader and it should render it first. That way your object won’t have been rendered yet, thus there won’t be anything for your silhouette to be behind to draw over. You may want to add ZWrite Off to your silhouette shader pass, otherwise it might stop the object from rendering! This is by far the simplest method, but you get the least control if you want it to sort with other transparencies.

Note all of these methods if implemented as a pass in the shader may randomly seem to fail if the order Unity decides to draw objects in ends up drawing a wall after your object. A simple fix is to have anything you want to have a highlight draw at a slightly later render queue, like “Queue”=“Geometry+1”

4 Likes
3

Too late bgolus! I already did your third suggestion mere seconds before you posted.

Shader "Custom/StandardOccluded" 
{
    Properties {
        _Color ("Color", Color) = (1,1,1,1)
        _MainTex ("Albedo (RGB)", 2D) = "white" {}
        _Glossiness ("Smoothness", Range(0,1)) = 0.5
        _Metallic ("Metallic", Range(0,1)) = 0.0
        _OccludedColor("Occluded Color", Color) = (1,1,1,1)
    }
    SubShader {
   
        Pass
        {
            Tags { "Queue"="Geometry+1" }
            ZTest Greater
            ZWrite Off

            CGPROGRAM
            #pragma vertex vert            
            #pragma fragment frag
            #pragma fragmentoption ARB_precision_hint_fastest

            half4 _OccludedColor;

            float4 vert(float4 pos : POSITION) : SV_POSITION
            {
                float4 viewPos = mul(UNITY_MATRIX_MVP, pos);
                return viewPos;
            }

                half4 frag(float4 pos : SV_POSITION) : COLOR
            {
                return _OccludedColor;
            }

            ENDCG
        }

        Tags { "RenderType"="Opaque" "Queue"="Geometry+1"}
        LOD 200
        ZWrite On
        ZTest LEqual
       
        CGPROGRAM
        // Physically based Standard lighting model, and enable shadows on all light types
        #pragma surface surf Standard fullforwardshadows

        // Use shader model 3.0 target, to get nicer looking lighting
        #pragma target 3.0

        sampler2D _MainTex;

        struct Input {
            float2 uv_MainTex;
        };

        half _Glossiness;
        half _Metallic;
        fixed4 _Color;

        void surf (Input IN, inout SurfaceOutputStandard o) {
            // Albedo comes from a texture tinted by color
            fixed4 c = tex2D (_MainTex, IN.uv_MainTex) * _Color;
            o.Albedo = c.rgb;
            // Metallic and smoothness come from slider variables
            o.Metallic = _Metallic;
            o.Smoothness = _Glossiness;
            o.Alpha = c.a;
        }
        ENDCG
    }
    FallBack "Diffuse"
}

I have to admit, reading your post it’s pretty impressive the amount of foresight going on there. Failing to put ZWrite Off does just draw my object over top of the silhouette, and if you don’t bump it up on the geometry queue is also does not render properly.

I did attempt using Offset, but like you said it seemed very tricky to get it to do what I wanted. The stencil buffer in general sounds really interesting, so I’ll take a look into it either way.

2 Likes
4

It’s possible I’ve made that effect before, several times, in multiple different games engines … it’s possible.

4 Likes
5

Just wanted to say Thank You to both of you. In my case I’m in unlit world (in my game) so changed the second pass to unlit. Sharing here in case anyone finds it beneficial.


Shader "Custom/UnlitOccluded"
{
Properties {
_MainTex ("Albedo (RGB)", 2D) = "white" {}
_OccludedColor("Occluded Color", Color) = (1,1,1,1)
}
SubShader {

Pass
{
Tags { "Queue"="Geometry+1" }
ZTest Greater
ZWrite Off

CGPROGRAM
#pragma vertex vert 
#pragma fragment frag
#pragma fragmentoption ARB_precision_hint_fastest

half4 _OccludedColor;

float4 vert(float4 pos : POSITION) : SV_POSITION
{
float4 viewPos = UnityObjectToClipPos(pos);
return viewPos;
}

half4 frag(float4 pos : SV_POSITION) : COLOR
{
return _OccludedColor;
}

ENDCG
}

Pass
{

Tags { "RenderType"="Opaque" "Queue"="Geometry+1"}
LOD 200
ZWrite On
ZTest LEqual

CGPROGRAM
// use "vert" function as the vertex shader
#pragma vertex vert
// use "frag" function as the pixel (fragment) shader
#pragma fragment frag

// vertex shader inputs
struct appdata
{
float4 vertex : POSITION; // vertex position
float2 uv : TEXCOORD0; // texture coordinate
};

// vertex shader outputs ("vertex to fragment")
struct v2f
{
float2 uv : TEXCOORD0; // texture coordinate
float4 vertex : SV_POSITION; // clip space position
};

// vertex shader
v2f vert (appdata v)
{
v2f o;
// transform position to clip space
// (multiply with model*view*projection matrix)
o.vertex = UnityObjectToClipPos(v.vertex);
// just pass the texture coordinate
o.uv = v.uv;
return o;
}

// texture we will sample
sampler2D _MainTex;

// pixel shader; returns low precision ("fixed4" type)
// color ("SV_Target" semantic)
fixed4 frag (v2f i) : SV_Target
{
// sample texture and return it
fixed4 col = tex2D(_MainTex, i.uv);
return col;
}
ENDCG
}
}
FallBack "Diffuse"
}```
2 Likes
6

Hey, I wonder what could be done if I have multiple materials in my renderer for different parts of the model. Say, one for the body with standard shader and one for armor and weapon with some metallic effect? Applying different materials with the same shader seem to work weird, as they try to overlap each other. Is there any way to tell shader to consider all materials on the renderer as one? I`m quite new in shaders so even if its generally obvious, its not that obvious for me)

7

There’s not a great way to do this. You need each pass of a material to run for the entire mesh / character before the next.

The work around is usually to split each pass up into separate shaders & materials and give them sequential material queues. Then you can apply both materials to all of the objects and it’ll act like it’s one. If an object has one material and you assign two materials to the object, it’ll render that object twice, once with each material.

The problem is when you have a single mesh with multiple materials, specifically a skinned mesh, there was a bug that every material past the number of materials the mesh has only draws the first submesh (the section the first material is applied to) over and over. If this is still broken you’ll need to make a duplicate of the mesh renderers, which isn’t ideal.

8

Thanks a lot!
I`ll see what I can do with it.

9

I was looking for an occlusion shader with an alpha, like this (Mario + Rabbids on Switch):

Iron Warrior’s shader just needed “Blend SrcAlpha OneMinusSrcAlpha” adding to the occlusion pass like this

Pass
        {
            Tags { "Queue"="Geometry+1" }
            ZTest Greater
            ZWrite Off
           Blend SrcAlpha OneMinusSrcAlpha

thanks for posting this stuff, really helped me out!

1 Like
10

Thank you guys for the nice work you did!

I’m just about to modify a toon shader and want it to look like the “Mario + Rabbids” example.
The “Blend SrcAlpha OneMinusSrcAlpha” does its trick well, but it’s not perfect because with the alpha you “look trough” the object.

Is there a way to prevent that?
(I’m new to shaders and only understand some basics ^^)

5762596--606814--ShaderExample.PNG
5762596--606814--ShaderExample.PNG725×604 16.4 KB

11

Use a stencil buffer

In the first pass, use

        Stencil {
                Ref 10
                Comp always
                Pass replace
            }

In the XRay pass, use

            Stencil {
                Ref 10
                Comp NotEqual
                Pass Replace
                Fail Keep
            }
12

No, that’s because the see-through shading should be a plain color, with no shadowing.
You can simply use a shader like this one :

Shader "Unlit/Color-SeeThroughOccluded"
{
    Properties
    {
        _Color("Color", Color) = (1,1,1)
    }

        SubShader
    {
        Tags
        {
            "RenderType" = "Opaque"
        }
        Color[_Color]
        Pass
        {
            Tags { "Queue" = "Geometry+1" }
            ZTest Greater
            ZWrite Off
            Blend SrcAlpha OneMinusSrcAlpha
        }
    }
}
1 Like
13

That’s not the solution to the issue mentioned by RiseBasti.

14

For anyone who comes across this page and is struggling the way I am, I achieved spakment’s effect using render textures. My characters are made of multiple meshes, so this approach has the added benefit of rendering all those meshes together into one sillhouette that doesn’t self-overlap. Here’s a copy/paste from our devblog:

We achieved this effect using a RawImage UI canvas element, with its size set to the canvas’ size. We added 3 cameras as children of the main camera (local rotation and position of zero), set their Clear Flags to “Solid Color”, making sure the Background color had alpha=0, and set them to each render to their own RenderTexture. Then, we set up their clear flags. To do this, we first set every level object prefab (walls, ramps, etc) to their own layer and the tank prefab to its own layer. Then, using the Camera’s “Culling Mask” property, we set one camera to render only tanks, one to render only the level, and one to render both. We then made a material that uses a shader with three texture properties and the below code for its fragment shader function. We set the material’s properties to the render textures we made earlier, set its color to have an alpha value of ~125, and set our RawImage’s material to it.

                fixed4 object = tex2D(_ObjectsOnly, i.uv);
                fixed4 cover = tex2D(_CoverOnly, i.uv);
                fixed4 objectAndCover = tex2D(_ObjectsAndCover, i.uv);
             
                float sillhouette = (objectAndCover == cover)*object.a;
             
                float4 col = sillhouette * _Color;
             
                return col;

This has a clear flaw in that if the tanks and the levels are ever the same color, the silhouette will fail or partially fail.

To fix this, we used replacement shaders. We first switched all level object prefabs to render using a shader that had a tag “OcclusionSilhouette”=”Occluder”, and all tank materials to a shader that had a tag “OcclusionSilhouette”=”Occludable1”. We then wrote a replacement shader that replaces shaders with the tag “OcclusionSilhouette”=”Occluder” with a subshader that renders a flat, unlit white, and “OcclusionSilhouette”=”Occludable1” with a subshader that renders a flat, unlit red. We then set each of the three special cameras to use this replacement shader. Now those cameras render tanks as a flat unlit red and the level as a flat unlit white, no matter what they really look like in game! No more problem with comparing colors!