Hi, I use standard toony outline shaders in my game, but there’s problem with every object with sharp edges, like cube, cylinder, etc. See screenshot.
[21195-výstřižek.jpg|21195]
Is there way to fix it?
Hi, I use standard toony outline shaders in my game, but there’s problem with every object with sharp edges, like cube, cylinder, etc. See screenshot.
[21195-výstřižek.jpg|21195]
Is there way to fix it?
Yes, this happens because two vertices at the same position have different normals due to the smoothing settings on the model. If you have a model with no hard edges, such as the sphere, you won’t have problems.
You need to calculate alternative normals that completely smooth the entire hard edges. Write a script that iterates through the mesh vertices, finds all vertices at one location (i.e. distance less than some amount), average the existing normals, and saves it into a vertex field (i.e. color) for all those vertices.
Your normal rendering will use the regular normals, and you will have to write a custom outline shader to use vertex color (or wherever you saved the smoothed normals) instead of the regular normal.
It would be better if there were a way to do it all in the vertex shader, but I can’t think of a way to smooth the normals without having access to neighbouring vertices (which you won’t have in the vertex shader). There may be a better way I missed, but the mesh pre-processing should work.
I solved this problem by writing a second Shader for object that included hard surfaces/sharp edges.
For my assets, I baked the object space normal data into the vertex colors, and then I had a surface shader apply those vertex colors as the normal data. AS IT TURNS OUT: Blender and Unity don’t exactly speak the same language, and I had to rotate the normal data around so it was facing the correct direction. After kicking myself for hours trying to figure out how to do that in the surface shader (which works in tangent space normals), I discovered I could supply an extra vertex shader to do my transformations in object space.
void vert(inout appdata_full v) {
v.normal.x = 1 - v.color.r;
v.normal.y = 1 - v.color.b;
v.normal.z = v.color.g;
v.normal = v.normal * 2 - 1;
}
void surf(Input IN, inout SurfaceOutput o) {
o.Albedo = tex2D(_MainTex, IN.uv_MainTex).rgb;
o.Albedo *= _Color.rgb
}
Then my second Pass was the vertex and fragment shaders that created the outline.
v2f vert(appdata v)
{
v2f o;
v.pos.xyz += v.normal * _OutlineThickness;
o.vertex = UnityObjectToClipPos(v.pos);
return o;
}
fixed4 frag(v2f i) : SV_Target
{
fixed4 col = _OutlineColor.rgba;
return col;
}
If all of your objects have smooth normals throughout, then this method is unnecessary, but this is the best solution I have come up with so far.