Hi friends,
it has come to my attention that the depht mask shader
http://www.unifycommunity.com/wiki/index.php?title=DepthMask does not work in unity 3.
Has anyone tried to fix it?
How can we mask geometry in unity 3?
I’m looking into this. You’ll notice that one of the comments refers to an undocumented argument to ColorMask and this could have been intentionally changed in 3.0.
thank you very much sir!
If you are referring to ColorMask 0, then I tried it and it did not work for me 
ColorMask 0 has been documented for a long time; that shader was actually the one that got me to read the ShaderLab manual, and by the time I got to it about a year ago, it was there.
I tried all the ColorMask combinations just now; they all work in the Editor, anyway. I’m assuming your problem lies elsewhere (i.e. the UsePass commands), but I’m not ready to tackle the new system yet.
Thanks Jessy,
indeed the color masks work fine. It seems to me it is the z test that fails. Perhaps, in unity 3, if something is invisible it is not written to the depth buffer? I don’t know how to check this.
Just place an object behind the ColorMask-using object, and have that behind-object use a shader that uses a later queue. (Adding Tags {Queue = Overlay} to any shader should do). I just did this, however, and had no problems. I only use my own fixed function shaders, so as I said, my inclination is to believe it has to do with the new surface shaders of 3.0.
thanks again Jessy. Yes, I can see that it is working when I do ColorMask RGB for example. It is rendered fine. But when I put it to 0 or A, I see the object behind it that is supposed to be culled. I also tried using custom shaders for the object that is being masked, but still no luck.
Would you please share a small project that works for you if you have the time?
Say the boat example found here:
http://www.unifycommunity.com/wiki/index.php?title=DepthMask
I would greatly appreciate it.
thanks
Certainly. Here’s the simplest possible case. See what you can glean from it and tell us if you need questions answered to further your understanding.
Thanks a lot Jessy, but the import doesn’t seem to be working. I am using unity 3 beta 4 version.
All I get is the list of items that are going to be imported and I can close that dialog but nothing gets in.
Can you please zip the full project folder?
Me too. I imported it myself and it worked perfectly. Maybe everything is just really screwed up on your end. You on Windows?
Sure. But that .unitypackage does represent the entire project.
352996–12276–$colormask_works_272.zip (58.9 KB)
It does indeed work. I will play around with the original to see why it fails. It should be because of the new surface shaders, like you said.
Thank you Jessy
Hey everyone,
I’m looking into this problem now. The example colormask project does work, but the DepthMask example doesn’t work.
The reason is the UsePass calls in Unity3 don’t work and cause the shader to go into Fallback if they are present
UsePass “Specular/BASE”
UsePass “Specular/PPL”
replacing them with Pass {} works, but obviously the material properties don’t come through.
Is it a matter of rebuilding the project in a way to get those calls to be recognized, or are those UsePass name’s no longer valid in unity 3?
Any updates on this? I think depth masking would really help me with my game…
what is your problem exactly? Depth masking can be made to work in unity 3 as discussed and shown above.
How do I make the overlay anything other than pure black?
Also, I think it interferes with lighting more than in 2.6.
Okay, so I’m trying to get Tags {Queue = Overlay} into my shader, which is otherwise the same code as the default diffuse shader since I downloaded the source for that. But every time I save that shader my changes disappear…
The Unity3 builtin shaders no longer have named passes, it seems. “UsePass”, however, requires both a shader name and a pass name.
My workaround for the DepthMask shader is, to copy&paste the appropriate builtin shaders into the Masked/* shaders, and remove the UsePass calls.
The drawback is, that any changes to the buildtin shaders in future Unity updates are not reflected here, and always must be updated manually.
There is another problem with the DepthMask shader: it no longer works correctly if the camera has a skybox.
Despite the builtin skybox shaders having their render queue set to “Background”, which is “Geometry-1000”, the skybox of any camera is actually rendered at “Geometry+500”, if the Camera’s ClearFlags are set to “Skybox”. As a result, the skybox will not be visible where the DepthMask shader is active (and the Game window will show the Editor’s(!) background color instead).
I could understand why they would render the skybox background after all opaque geometry - this way, the skybox is only rendered in places where it is actually visible (taking advantage of the z-test), instead of rendering it fullscreen every frame first, which would be a waste of pixel fill ressources.
However, if that’s the reason they changed it, the documentation is wrong.
To get the DepthMask shader to work with Skyboxes again, attach this script to the camera:
using UnityEngine;
[ExecuteInEditMode]
public class ManualSkyboxClear : MonoBehaviour {
void OnPreRender(){
GL.ClearWithSkybox(true,camera);
}
}
, and set the Camera’s ClearFlags to “Don’t Clear”, to prevent it from clearing it twice.
(note, the Camera Preview window won’t render the skybox correctly, but the Game window does).
I wondering this myself. Currently reading up on shader creation and experimenting, but no luck yet
I had the same problem of DepthMask shader not working in Unity3, though it worked in Unity2. The reason seemed to be because I was using deferred lighting in Unity3. The mask worked in Unity3 under forward rendering.
Like Jessy said, if the object that has to be culled is pushed later in render queue, then there is no problem. eg. “Queue” = “Overlay” or "Queue = “Geometry + 600” . This works in unity3 under deferred lighting.