Apologies if this has been asked/posted somewhere else before, I did find lots of information seeming to relate to this but my limited understanding of graphics programming leaves me mostly clueless. And it’s late. And I’m sleepy.
Does anybody know if there’s any way to modify how the depth buffer gets rendered? I’m specifically looking at introducing some math that will clip pixels from getting drawn into it when they fall into a certain range. I’ve got the same logic already in place in a custom shader to produce the desired effect when drawing the objects themselves, but it seems that the object’s resulting pixels are still present in the depth buffer and is occluding Unity’s shadows from rendering as expected. That’s just my hypothesis; I’m trying to figure out a way of testing this.
I’d appreciate any help/guidance as my brain is quite fried at this point.
Cheers!
Hi colin, thanks for the reply!
So, is the depth buffer written to automatically by interpolated POSITION values coming from the vertex shader in the first pass or something? I was hoping that there was some way to specify the depth values getting written in a fragment shader so that I could clip the pixel according to my logic and avoid writing to the depth buffer for specific pixels.
I’m trying to write a shader that will hide occluding geometry between the camera and player but otherwise render the scene as normal, preserving the additive light pass, shadows, etc.
(Left is standard shader with camera above wall. Right is my shader with camera behind the wall - notice the shadows are not being fully rendered as intended underneath the red line)
I think I’m mostly there but am noticing that shadows are still getting occluded by clipped geometry and just assumed it was due to the geometry writing to the depth buffer in some internal pass before getting clipped in my fragment shader. I tried a quick test to muck with the outputted pos.z value in the vertex shader similar to your example but it didn’t seem to affect the shadow rendering at all (had other artefacts though), so maybe I’m approaching this the wrong way.
I think it’s beacuse the shadowcaster pass doesn’t do this clipping. (Not the same as the normal pass). Usually unity gets a shadowcaster pass thanks to fallbacks that have been set, however you can create a subshader for it yourself!.
Using a clip in a shader should discard the depth write as far as I know.
Hey Zicandar!
I based the shader I’m working on off of Unity 5’s Standard shader, removed the fallback, and added my clipping logic to the ForwardBase and ForwardAdd passes. I saw a ShadowCaster pass which I ignored at first because I do want my occluded geometry to cast shadows in my scene as normal, but I tried adding it just now by your suggestion - to no apparent effect. The Frame Debug shows the shadowmap being rendered as normal, surprisingly enough.
However, if I bypass all my logic and just immediately clip(-1) in the shadowcaster fragment shader, all the shadows do disappear as I would expect (so I know this pass is the one responsible for casting my shadows). But I don’t think it’s the shadow casting that I want to clip anyways, but the depth buffer which I’m pretty sure is being used in the shadowgathering phase to produce the on-screen shadows which later get attenuated into the final result during the ForwardBase pass.
You need to clip both, unless you want your clipped objects to cast shadows even when they are “gone”.
Ok, so I looked at the frame debugger in forward mode. (I’m normally in deferred, so havn’t done much research on forward).
Unity first does a depth pass, then it does the shadowcasters, and then gathers, lastly it does the normal rendering.
The first depth gather pass I’m gonna assume also works as a z-prepass.
Note that this is with DX11 mode on.
From what I can tell the depth pass and the shadowCasting pass use the same pass!
So summing up:
It IS the shadowcasting pass you need to modify!
Yes, I do want the clipped objects to cast their shadows as if they were still there.
I think that’s the pass that I need to clip, but I have no idea how to get to it. I’m afraid this might be a Unity internal pass that isn’t open to modification - was hoping somebody would prove that wrong.
What do you mean by a z-prepass? I’m not sure I’ve heard of that before.
You mean… that they… actually, sorry, what do you mean?
And thanks Zicandar. It helps alot to have another’s perspective on this - I was going nuts on my own.
Pass {
Name "ShadowCaster"
Tags { "LightMode" = "ShadowCaster" }
ZWrite On ZTest LEqual
CGPROGRAM
#pragma target 3.0
#pragma glsl
// TEMPORARY: GLES2.0 temporarily disabled to prevent errors spam on devices without textureCubeLodEXT
#pragma exclude_renderers gles
// -------------------------------------
#pragma shader_feature _ _ALPHATEST_ON _ALPHABLEND_ON _ALPHAPREMULTIPLY_ON
#pragma multi_compile_shadowcaster
#pragma vertex vertShadowCaster
#pragma fragment fragShadowCaster
#include "UnityStandardShadow.cginc"
ENDCG
}
That code that’s in the standard shader for generating depth values, you will need to replace the fragment shader with something that cuts.
Now this will be a problem, as if I’m correct, the pass I just mentioned is responsibe for both cutting AND for saying when stuff casts shadows!
IF it becomes a problem, one possible solution is to have a inivisible shadowcaster material.
Right, I’ve been messing with this pass to try and get it to do what I want, but nothing I try seems to work. The only thing I can do that seems to have an effect is to force it to clip, at which point any material using this shader stops casting shadows - as expected. Shadows being cast/received by the standard material will continue to work and will not be occluded by my clipped geometry the way I was seeing happen when only using my shader, which makes me think that you may be right regarding the function of this pass. But of course, those objects will not get clipped the way I need them to and continue casting/receiving their shadows correctly.
As you suggested, having an invisible shadow caster may be part of a solution, but I’m still left with the shadows cast being occluded by my clipped geometry, and adding my clipping algorithm to this pass doesn’t seem to do anything for some reason.
I may have to rethink my entire approach to this altogether.
But thanks again for the help!
Oh - are you thinking that way it would continue to cast its shadow while having “Receive Shadows” set to false would prevent it from occluding the other shadows rendered? That doesn’t seem to be the case, unfortunately. But even then, I think that means I would end up with my entire level having “Receive Shadows” set to false, which would result in no shadows rendering at all. If I understood what you were getting at.
If you (or anybody else) are really interested in figuring out what’s going on, then: awesome! I’m posting a minimal sample project that depicts the problem. I think I’m at the end of what I can do or hope to figure out. Maybe someone else will have better luck.
Just to sum up the desired outcome: I was trying to figure out a way I could, in a shader, hide geometry that stands between the camera and a target in the scene while otherwise having that scene continuing to render as expected: hidden geometry should continue to cast its shadow on the scene though the camera can see right through it.
I have up until now assumed you have a reason for not simply using nearClip plane for this?
(In addition to this code I changed all materials to a normal standard shader.)
[ExecuteInEditMode]
public class ShaderTarget : MonoBehaviour
{
private void OnWillRenderObject()
{
Shader.SetGlobalVector("_TargetPosition", transform.position);
Camera currentCamera = Camera.current;
float distanceToObject = Vector3.Distance (transform.position, currentCamera.transform.position);
currentCamera.nearClipPlane = distanceToObject - 0.5f;
}
}
Other then this I did managed to get your project to do what you want, HOWEVER I can only manage that by using deferred mode as it doesn’t do an extra depth pre-pass!
I added that code to the fragDeferred function.
Ofc your doing it differently by using the projection space w of the vertex, so to do it your way you’d have to dupe the vertex and fragment shaders for deferred.
(Btw, your way has the interesting property of cutting less stuff away when you go close!?)
And I found a solution for Forward mode! (Last solution I’m posting, I hope…)
So now we have solutions for deferred, forward AND the simple one with using camera nearclip
Command buffers and a global float >.<…
In your shader target script:
if (preShadows == null)
{
preShadows = new CommandBuffer ();
preShadows.SetGlobalFloat ("Shadowcasting", 0.0f);
}
if (postShadows == null) {
postShadows = new CommandBuffer ();
postShadows.SetGlobalFloat ("Shadowcasting", 1.0f);
}
Camera[] allCameras = new Camera[11];
Camera.GetAllCameras(allCameras);
foreach(Camera cam in allCameras)
{
if (cam == null) continue;
cam.RemoveAllCommandBuffers();
cam.AddCommandBuffer(CameraEvent.BeforeDepthTexture, postShadows);
cam.AddCommandBuffer(CameraEvent.AfterDepthTexture, preShadows);
}
Then in the shader I had to do a few more modifications to make it work. The order they are output from the vertex shader seems important:
void MyVertShadowCaster (VertexInput v,
out float dist : TEXCOORD0,
#ifdef UNITY_STANDARD_USE_SHADOW_OUTPUT_STRUCT
out VertexOutputShadowCaster o,
#endif
out float4 opos : SV_POSITION
)
{
Yes, in my actual use case I’m using different formulas to arrive at a shape other than a plane used for clipping. The example I provided is a simple sphere with it’s center on the camera and a radius length reaching to the target position, which is why you see the clipping effect become smaller the closer the camera is to the target.
O_O
Zicandar, you just made my week!
I was already knee deep into another approach (explicitly hiding meshes - blech) but kept thinking that there must be a way to get this working in a shader. And you figured it out! I can’t tell you how psyched I am! Now I’m gonna tear back into this and try getting it back to where I had it - with your magnificent fix in place.
So: You’re using CommandBuffers to flip a switch in such a way that the geometry is clipped before rendering the depth texture for the shadows, and then restoring it to render full shadows after the depth texture is rendered, did I understand that right?
It looks like everything works when adding the command buffers once on Awake - is there any reason I’m unaware of that I might need to do this every frame on Update instead (I wasn’t sure from your code where it was meant to go)?
And… I guess those are the only follow-up questions I have. I hope it won’t be a bother for you but I may need to hit you up for some related shader-business somewhere down the line.
I’d do the command buffers onEnable or similar, or at the VERY least on Start not awake.
Awake should never be used when modifying anything outside your own script.
Feel free to ask me questions!
If you have skype you can pm me your email/skype id and I’ll add you
