how to programmatically allow Shader to control which object renders in front?

Hi,

I’ve only just started learning Unity, but because I come from a background of coding in C#, I’ve found the standard scripting to be very quick to learn. Unfortunately, I’ve now come across a problem for which I believe a custom shader is required and I’m completely lost when it comes to shaders.

Scenario:
I’m using a custom distance scaling process so that really big, far away objects are moved within a reasonable floating point precision range from the player. This works great and handles scaling of the objects based on their adjusted distance so they appear to actually be really far away. The problem occurs though when two of these objects pass close to eachother in game space (this would still be millions of units apart in real scale) because they visibly collide.

Attempted Solution 1:
I’ve looked into flattening the objects along the player’s view axis and this fixes the collision, but this affects shading and particle effects so wasn’t a good option

Attempted Solution 2:
I’ve tried changing the RenderOrder, but because sometimes one object is inside the mesh of another (though the centre of this object is still closer to the camera) it doesn’t fix the issue and particle effects are problematic again.

Attempted Solution 3:
I’ve tried moving the colliding objects to their own layer, spawning a new camera with a higher depth at the same position as my main camera and forcing the cameras to only see the items on their respective layers, but this caused lighting issues as some objects are lighting others and I had only a limited number of layers so this solution was quite limiting as it forced me to only have a low number of objects that could be overlapping at a time. NOTE: this solution is probably the closest I was able to come to what I need though.

Attempted Solution 4:
I’ve tried updating the Standard shader code by downloading it from Unity’s downloads page and creating my own, custom shader that allows me to modify the ZWrite and ZTest properties, but because I’ve no real understanding of how these work, I’m not getting anywhere.

Request:
I would greatly appreciate a Shader script code example of how I can programmatically force one object who’s mesh is either colliding with or completely inside another mesh to render in front of said mesh. I’m hoping I can then take that example and apply it to all the shaders that I’m currently using (Standard, Particle Additive) to achieve the effect I’m looking for. Thanks in advance for your help.

I could attempt to help, but I’m having a hard time visualizing what you’re looking for. Might help to draw out a diagram or show some scene view stuff or something, to help explain.

Here is an example video that shows the problem:
https://www.youtube.com/watch?v=KFnuQg4R8NQ

And here is the video showing how it should look (achieved via attempted solution 3):
https://www.youtube.com/watch?v=CyFDgimJ2-8

I’m still kind of confused. Is the orbiting object actually in front of the big star, or not? I don’t really understand how the positioning is working, and why things are going inside of other things when it’s just orbiting.

In the video, the bottom left shows an overhead view of the orbit. I’m scaling distances and sizes of the objects from their real-world sizes to sizes that fit within Unity’s 7 digit float precision as you move away from the player. Because of this, the distance that the objects are actually separated by (when viewed from close to the orbital plane) is so close that the star and planet intersect as the planet transits the star. If the player were far above the star, this wouldn’t be a problem, but when near the orbital plane, the distances and size scales work out such that the objects intersect

Ah okay. My first thought is to have two separate shaders, one with ZTest Always (when checking the depth, the shader will always evaluate as if the object is at the front) and one with ZTest LEqual (the default). Then, in script, when you need the object to be in front, set the material’s shader to the ZTest Always. When you need it to be behind, set it to the ZTest LEqual shader. The problem with this is that the object will then be drawn in front of other things as well, so it may not be right for this…

1 Like

My other idea would be to have the small object write to a stencil buffer when its in front, and then the big object would discard any pixels that share that stencil buffer.

Thanks for the suggestion. My only concern is that if I have multiple planets transiting at the same time the binary ZTest would only handle part of them and the rest would still appear to collide / intersect. Do you know if there is a way to give more fine-grained control over the ZTest depth so I could support several layers of transits?

This sounds promising as it could support multiple transiting objects as each could discard the shared pixels. Do you have an example of how this might be done? Just something to start me off would be greatly appreciated. Thanks for both the suggestions!

The only real way I can think of having totally fine grained control would be to set ZWrite Off on all the planets and stars and stuff. This way, they won’t write to the depth buffer at all, so render order should be all that matters. The problem with that is that if you use any shaders that you want to have normal depth sorting, they won’t work right with them.

Unity’s stencil example is pretty good. Near the bottom there are three shaders that interact with each other.

1 Like

Perfect! This looks like it might be the solution. I’ll have a go at it and see what I can accomplish. At a quick glance it seems it should solve the problem; now I just need to put together a functional modification to the Standard and Particle Additive Unity shaders so I can test it out. Thanks again for the help! :smile:

You’re welcome! Definitely post an update in here if you get it working :slight_smile:

Well, I tried a couple of other approaches first, just because they were easier to implement and didn’t require learning Stencils, and though they showed promise early on in a test scene, they didn’t work in my actual game so I decided to give the Stencils a go. I’m nearly there in terms of having something that works, but I can’t get the particles to behave properly.

In the following scene, all 3 spheres are located at the origin. The Green is scaled to 2, the Red is not scaled and the Blue is scaled to 0.5. I’m using stencils to allow the Red sphere to cut out it’s own shape from the Green and to allow the Blue to cut it’s own shape from both the Green and Red. This works perfectly, but I then added a particle system as a child of the Red sphere and made the same stencil addition to the standard Unity Particle Additive shader, but I can’t seem to get the particles to render over the Green sphere even though the settings for its stencil is the same as that of the Red sphere’s material. This is driving me nuts!

https://www.youtube.com/watch?v=mpsc_27aUys

Good news! It seems that the stencil additions that I made to the standard shaders works perfectly in my game (even though in my sample scene there were some issues with particles). Thanks again @wtrebella for the enormous help!

Here is the working system. I’m using a script to determine which object is furthest away in Real space and updating it’s Stencil Ref with 0 and then for each object that is closer to the player I increment the Stencil Ref value by 1. All Stencils are using a Comparison of GEqual and a Pass value of Replace. :smile:

Oh hell yes this is awesome. So glad you got this working with stencils!

1 Like