Hi all, copied out some of the Unity’s standard Toon Shader code to add to the Standard Shader for object highlighting. For the most part, this works flawlessly, but it seems to stumble quite a bit on renderers that have cloth components attached, such as the cape of our character.
To make sure I didn’t fubar anything, I changed the shader to just the “Toon Lit Outline” shader, and I am still seeing large ‘splotches’ of color in certain areas of the cape. If I disable the Cloth component, these go away entirely.
That being said, if I use the ‘Basic Outline’ toon shader, the splotches are significantly reduced, but still present in certain areas. Seeing as this seems to be a deficiency of the Toon Shader’s outline code itself (I have little to no experience with shaders), I’m wondering if anyone has encountered or found a workaround for this issue?
For what it’s worth, setting the Outline color’s alpha to 0 allows me to see entirely through the cape.
Here is an image of the problem in question:
Thanks!
No one has encountered this? I’m fairly certain it isn’t even exclusive to cloth either as I recently found another mesh that exhibits the same behavior. Basically, even with the outline width set to 0, I can still ‘see’ the outline. Quite frustrating.
Thanks for your response. Ignoring the cloth and after spending much of the day trying to wrap my mind around the unity shader system, I believe the problem is inherent to certain meshes themselves. Again, I am a complete noob when it comes to writing shaders in Unity and I had a lot of difficulty figuring out how this ‘outline’ shader wasn’t just overwriting the standard shader entirely. For reference, I took the original standard shader code and added a pass that adds the highlight. I am still somewhat confused about the ordering of certain shader passes, although I assume they execute as ordered in a particular shader file?
Nevermind that though, I realized that the outline shader is set to cull front faces and essentially extrude backfaces so that they ‘become’ front faces, IE, from the sides. As it turns out though, without doing anything at all to the shader, IE, writing out the transformed vertex and a ‘fake’ color, I can see that certain faces are shown on the other model, IE, they are rendering before I even do anything to try to show an outline, which is already a problem. Furthermore, I have removed all texturing from the equation, so we’re down to the mesh itself.
So for whatever reason, it’s rendering certain vertices/faces even though it shouldn’t and they always seem to form quadrilaterals. While I don’t know how this applies to the cloth directly, this is certainly the issue with this mesh and chances are that there might be duplicate vertices/faces in that spot, so perhaps it’s a form of z-fighting. Just kinda talking it out to myself and whoever else might be reading along that can maybe offer some more ideas.
If I’m not mistaken, there is a solution somewhere on the internet where a person has successfully merged an outline shader with the standard shader. Maybe you could search for this.
I havent played with Cloth component for long so I might be wrong, but maybe it somehow mess up the mesh normals, and since outline shader rely on these normals to extrude the face they extruded to wrong direction…
Maybe you can try using Image-Effect/Post-Proccessing toon shader… But seeing your screen shot, it might be not a best solution(or not a solution at all)…
Or maybe you can just scrap the cloth component and use physics joints & rigidbodies. I usually stay with those since mostly cloth give some erratic result…
That’s actually what started me down this path. I’ve already found and altered that shader, but really, the fault seems to be elsewhere and not with the shader itself. It has been an extremely valuable learning experience though!
I thought about using one of the screen-space edge detection effects, but I don’t know that it would be possible to apply it to a single object within the scene. I definitely agree with the cloth–it has given us a lot of headaches although I guess I assumed that internally the cloth was using joints and rigidbodies behind the scenes?
No, screen space effects apply to everything in the scene. There may be ways to mask it, but it sounds like a tremendous headache to mask for an animated character that may at times be partly obscured by other objects that are closer to the camera.
I don’t know if this would work, but have you considered applying a second material to your character? If you are using just one material for the character (using the standard shader), then you could try adding a second material that is just outlining for the full body. Probably the outliner would go first, and the standard shader would go second, writing over it (except for the outline part, I assume). You could try it with one of the free shaders I mentioned above, or maybe write your own if you have that skill.
A predictable side effect of this method would be that if you ever try to make the character invisible using alpha on the standard shader, the outline will remain unless you also switch that off.
FWIW, I too was trying to solve this sort of situation for my own project, and I’ve not completed my investigation into all the options. My current solution is to use Shader Forge, which does offer outline (plus control over outline width and color) in addition to PBR shading, but I’m not sure I’ll stick with it. If you try adding a second outline material and it works, I’d be interested to hear about that.
Using two materials is actually how I was previously doing it. Using two materials wasn’t too bad, but I never quite got them to ‘overlay’ correctly, despite different material ordering options, rendering modes (transparent, fade, etc) What I ended up doing was ‘switching’ materials by setting the highlight shader to null (messy, but I didn’t know of any other way to ‘disable’ a shader) during normal operation and setting it when the highlighting was desired.
However, I’ve learned a lot since then, so the approach could be worth a revisit. The main issue we had is that some of our assets have a large number of submeshes, and thus, a large number of ‘multi-material’ renderers, which was hurting us performance-wise. We wanted to try to handle everything in multiple passes instead of multiple materials.
Previously we were using a UAS highlight shader, which actually had built-in options for diffuse/normal mapping. For our early prototypes this was fine, but only normal/diffuse mapping leaves a lot to be desired. Since that shader, internally, was a bloated mess, I figured it’d be better to try to move away from it and combine the standard shader with another highlighting solution. And yes, the problem was still present even with that particular shader.
It is strange to me that this hasn’t been encountered by anyone though. Worth opening a ticket? It seems like an error immediately that a shader that is set to cull front faces would show anything that ‘should’ be obscured by another face. Admittedly, both the cloth submesh and the other mesh are quite thin, so I don’t know if there is some sort of z-fighting going on? I’ve been messing with other stuff ATM but I hope to get back to it and figure it out once and for all.
I’ve already found and altered that shader, but really, the fault seems to be elsewhere and not with the shader itself. It has been an extremely valuable learning experience though!