How to cull transparent objects within the shader

Unity Engine
,
1

Hi, so im writing a shader for some engine plumes, and i want to add an opaque (transparent with cutoff) effect on top of it, but it would need ot be culled so that this doesnt happen

1034×668 110 KB

As you can see, its appearing a over the effect below it, when it should be culled…
I did some research and it seems like writing to the depth buffer might be the solution, but i dont see how i’d read it on the frag of my opaque shader…

the idea here is:

  • Transparent Multiplicative is on queue 3000 and writes to SV_DEPTH its closest depth
  • Transparent Cuttoff reads SV_DEPTH on current pixel and decides wheter it should cull or not

both shaders are volumetric, so i’m trying to cull just the density behind the other effects.

2

What are you expecting the SV_Depth value to be here? If you’re expecting the value that’s in the depth buffer, then that’s not what you’re getting. The input SV_Depth is the depth of that fragment as calculated by the GPU for that triangle. (It’s actually the same value the fragment shader gets from SV_Position’s z component.) There’s no way to read the current depth buffer value directly. If you want to clip against a specific depth, you need to make use of DepthTest and either output the wanted SV_Depth for your fragment, or preferably offset the vertex position’s clip space Z as modifying SV_Depth disables early depth rejection.

3

To put thing more clearly, both of these are on the Transparent Render queue, so its on 3000, so, at least afaik, all automatic depth rejection is already done.
What i’m currently doing is that I set the SV_DEPTH as the first instance of the volumetric marching’s depth (so i’m returning this value).
The problem is that I dont know how i can read this value outputed depth value. I need to read it so that i can cull (or not) other instances of the same (or similar) shaders.

Currently im doing this:

            fragOutput frag(v2f i) : SV_Target {
                if(_Opacity == 0)
                    discard;

                fragOutput o;
                o.depth = 1;
                o.color = 0;
                float3 rayPos = _WorldSpaceCameraPos - position;
                float viewLength = length(i.viewVector);
                float3 rayDir = normalize(i.worldPos - rayPos - position);
                float2 dstToBox = 0;
                float2 dstToBoxFar = 0;
             

                if(!LogBisect(orw(rayPos, 0), orw(rayDir,0), dstToBox, dstToBoxFar)){
                    return o;
                }

                dstToBox = max(0, dstToBox);

                float2 dstInsideBox = dstToBoxFar - dstToBox;
                dstInsideBox = max(0, dstInsideBox);

                // point of intersection with the cloud container
                float3 entryPoint = rayPos + position + rayDir * dstToBox.x;
                float3 exitPoint = rayPos +position + rayDir * dstInsideBox.x;

                float dstTravelled = 0;
                float dstLimit = dstInsideBox.x;
                int stepCount = (1+_Resolution)*32;
                stepCount *= _ResolutionMultiplier;

                stepCount = max(_MinimumResolution, stepCount);
                float stepSize = dstInsideBox.x/(stepCount); //Create multiplier for step count to allow for modder customization
                int failSafeCount = stepCount;

                float density = 0;
                float4 entryClipPos = UnityWorldToClipPos(float4(entryPoint, 1));
                float4 screenPos = ComputeScreenPos(entryClipPos);
                float2 uv = screenPos.xy/screenPos.w;
                float depthSolid = LinearEyeDepth(SAMPLE_DEPTH_TEXTURE(_CameraDepthTexture, uv));

                while(dstTravelled < dstLimit){
                    failSafeCount -= 1;
                    if(failSafeCount < 0)
                        break;
                 
                    float3 currentPos = entryPoint + rayDir * dstTravelled;
                    dstTravelled += stepSize;

                    #ifdef CUSTOM_ZTEST
                    float4 clipCurPos = UnityWorldToClipPos(float4(currentPos, 1));
                    float depthCurPos = LinearEyeDepth(clipCurPos.z/clipCurPos.w);

                    if(depthCurPos >= depthSolid)
                        continue;
                    else if (o.depth == 1)
                        o.depth = depthCurPos;
                    #endif
                    density += SampleDensity(currentPos - position)* (stepSize);
                }

                fixed4 col = density;
                o.color = col;

                return o;
            }

As you can see, I set o.depth to watever the depth on the current position is IF its not occluded by any geometry. What i’d like is for a way to read this o.depth on other shaders in a way that i can check if the depth on x position is greater or not, so that i can discard that march step.

4

Nope. If you write to SV_Depth, depth rejection is done after the fragment shader, meaning you’re paying the cost for rendering it when it’s within the camera frustum, even if it’s “not visible” because it’s behind something else. And it means when you modify the depth it changes what occludes it… as otherwise there wouldn’t be a real reason to use it.

As for what you’re trying to do, the short version is: you can’t do that. Reading from the _CameraDepthTexture is the only viable way to read the current depth buffer, and that’s not actually the current depth buffer as it’s a texture generated by rendering all opaque queue objects using their shadowcaster pass before the scene actually renders anything. Any transparent queue materials that write to the depth will not be included in the depth texture, and there’s no real way to update the depth texture after the fact to include new things, nor would you want to really. Even on mobile, which includes ways to read from the frame buffer directly, does not allow you to read the depth buffer.

So, what can you do? That’s harder to say. I honestly don’t think I fully understand what you’re trying to accomplish. If I were you, I’d render the alpha cutout effect you’re trying to add as an actual opaque alpha cutout effect with a shadowcaster pass so it renders to the camera depth texture and let the volumetric effect do the rest of the work. I wouldn’t have the volumetric effect do any depth writes at all if I can help it.

Alternatively if you really must be able to read the depth of another material, you’ll have to render that to a render texture manually yourself so you can sample it in the other shaders. There’s no other option.

5

to visualize better my issue i disabled the noise… Here’s whats happening, first from the side, which looks right:

but if i look at it from below, i can still see the whole “torus”, which should be culled as you can see here

I think that the alpha cutout might actually be a good idea, i’ll test that, any examples on how to properly do it? i’m guessing that i could just make a ColorMask 0 and write 1 where i want the alpha cutout to be? or should i write the actual depth?

6

Ah… hmm.

What you’re really looking for is order independent transparency. Which is a huge rabbit hole of expensive and/or incomplete solutions.

I don’t think any of the solutions you have in mind, nor the suggestion I had, will solve the issue you’re experiencing. The one solution I can think of is to breakup your flame into two parts. One that’s the same length as the “torus”, and explicitly renders behind it, and a second that renders the rest of the flame separately and makes use of Unity’s default transparency sorting to pick which renders over the other.