Outline shader for multiple meshes.

I want outline for object selection effect. But the object is consistent with multiple meshes.
I tried

  • Just scaling mesh, which was fine for single mesh but doesn’t work for multiple meshes. (top left)
  • Scaling in normal direction, because I want thick outline this looks weird.

And I have no idea how to apply edge detection method only for some(selected) objects.

I tried come combined mesh thing with naive expectation, didn’t work ofc.
So how can I properly apply such selection effect on multiple meshes? (outline of white cube is what I want)

5798542--612712--sad.png

I think you could try also custom pass which creates outlines more like a post-processing effect. These could probably use sobel-filter and depth/normal edge detection, too. There’s examples for URP/HDRP and probably similar approach is documented somewhere for built-in pipeline, too.

1 Like

The scaling method is perfectly acceptable, but it has it’s limitations. Mainly you need to make sure the object’s pivot is consistently centered for outlines from multiple objects to line up, and doesn’t handle concave setups well.

The normal based method is better for this as the pivot and convex objects work better, but it does require smooth normals, or some bridge geometry be added in seams. The other option is to store a second smoothed normal in the mesh’s vertex colors or an extra UV set. The Toony Colors Pro 2 asset on the store has a feature to add smoothed normals to an existing mesh like this, and shaders that make use of it. But I don’t remember if it handles removing interior lines or not, and multi-object lines will still be problematic.

A decent hack to get rid of interior lines would be to push the outline away from the camera. Like in this example:

This has some unfortunate side effects. One with the specific technique in the above link needs a different amount of offset depending on the camera distance. That can be worked around, but the bigger issue is the outline will be cutoff for anything near the ground or other objects even more so than the usual version of the technique.

The technique I prefer is to use is to render the objects twice and use stencils to mask out the outline. The order is important here. But basically you need to use two separate materials & shaders (and potentially renderer components) to ensure that things render in the order you need.

The trick is your objects are all rendered first normally with no outline, but they write to the stencil buffer. Then the outline material renders afterward (use a higher material queue) with a shader that skips any pixels that have the stencil value written to. It should also use Cull Off instead of Cull Front as is common for shell based outline shaders. This has the benefit of the outline showing even with sitting on the ground or near other objects more. It can also write to the stencil in the areas it does render to in case you want to support semi-transparent outlines, which is a nice bonus.

An alternative I used on a device without stencils was to render the outline material in a queue before the main material, but use ZWrite Off. Usually that requires a little more careful control over draw order in general since anything that renders after the outline will draw over it, so I also had to change the render queue of the objects main material to 2500 and the outline to 2499 so that no opaque objects rendered over it unintentionally. The stencil option is a lot nicer, and interacts with transparency better. With stencils you can even draw it really late if you want the outline to show up over all transparent objects.

The last option is more along the lines of what @Olmi suggested. If this is for highlights, use a post process based highlight system. Something like what Unity’s editor does when selecting objects. There are a number of assets on the store, as well as a few free ones floating about.

I think somewhere there’s a working version of the Unity Editor outline code as something you can actually use in the game, but I can never remember where that one is.

3 Likes