I’m trying to create a silhouette outline shader with the following requirements but failing pretty miserably at the minute. I’m really hoping someone can help as it’s keeping me up at night!
The entire character is outlined at its silhouette without any interior lines.
It needs to be an outline rather than an inline.
The outline must support characters made out of several separate Renderers as the characters have various accessory items that are equipped by the player.
The outline needs to appear to be in world space and must be able to be obscured by other objects or outlines that are in front of it.
The target platform is mobile so ideally this would be an object material approach rather than a post process solution.
Mario is made up of 2 models in this example, a main model plus a standalone hat model. In the “Desired Result” there is no outline around the rim of the hat where it would cover Mario’s face.
I have looked into a lot of different methods (inverted hull, toon shaders, stencil masks) and just cannot find a technique or combination of techniques where I can get everything working at once. At this stage I’m not even sure if this is possible using a per model shader approach so I would really appreciate if anyone can shed some light on this problem for me. I do think that there must be a way to do this that likely involves a depth pre-pass and the stencil buffer but I just haven’t hit on the magic combination yet!
@bgolus - I hope you don’t mind me tagging you but you have been active on a lot of the outline shader discussions I have been reading lately and you also have a pretty great blog post on outlines so I’m hoping maybe you can help here!
If you’re on mobile, then you’re going to want to use hull based outlines. What the built in toon shader does.
The easiest way to get outlines that don’t affect the interior of an object is to add a bit of depth bias.
Some side effects of using depth biasing are the outline won’t appear on the ground beneath the feet of your characters because the outline is being pushed back by some arbitrary distance, and that distance isn’t consistent if you’re modifying the pos.z directly as the clip space depth is non-linear, so a bias of “0.05” might not be enough in some cases and way too much in others depending on how close or far from the camera it is. But it is very inexpensive. You can fix this a little bit by offsetting the view space position away from the camera instead of directly modifying the clip space position. Whether or not an outline works around multiple separate objects is entirely determined by how far apart they are in distance and that bias setting still, so it’s a little finicky, but requires the least amount of initial setup.
What I like to do is use stencils, assuming you can afford a full 32 bit depth buffer w/ stencil support on mobile (which is the default unless you tell Unity otherwise). The basic idea is to have your character materials write to the stencil buffer a unique value per character. Then render the outline pass as a separate material per character that happens at the end of the opaque queue range or start of the transparency range that tests against that same stencil value per character and skips rendering for the pixels it’s been set. Usually when doing hull based outlines you render the outline pass using Cull Front, but instead you want to skip that and leave it as the default, which is equivalent to Cull Back if you leave that line out. You can even add a small bias towards the camera in that case to try to get the outlines to show where characters feet are near the ground. The nice advantage of this technique is that it works properly with depth sorting.
Alternatively if you don’t want to use stencils or biasing, you have to control the render order of your characters very carefully. Alternating between outline passes using ZWrite Off and the character. For something like the mario & hat case you’d need to render the outlines for both meshes, then render both mario and the hat. For overlapping characters you’d need to render the furthest outline > furthest character > next closest character outline > next closest character, etc. This is backwards from how you’d normally render opaque objects, and means you’ll likely need to be manually setting their materials’ render queue or using the renderer components’ .sortingOrder from a custom script to get this to work.
The stencil approach has worked like a charm! I knew there must be something I was missing, turns out I had been looking at the problem back to front and trying to draw the outlines first followed by the actual objects.
I also tried the bias approach but couldn’t make that work for our needs, if I made the bias large enough to hide the inner lines for one character, it also affected other characters close by.
