Text outline effect for text

Unity Engine
,
1

Hello there.

I wanted to add opaque font outline to the TMP’s pixel font outline shader. So my implementation idea was simple:

  • in first pass i render TMP’s pixel glyph:
Pass
{
    Stencil
    {
        Ref [_Stencil]
        Comp [_StencilComp]
        Pass [_StencilOp]
        ReadMask [_StencilReadMask]
        WriteMask [_StencilWriteMask]
    }
    ColorMask[_ColorMask]
   
    CGPROGRAM
   
    float4 _MainTex_TexelSize;
    uniform float4 _OutlineColor;
   
    v2f vert(appdata_t i)
    {
        float4 vert = i.vertex;
        vert.xy += (vert.w * 0.5) / _ScreenParams.xy;
        float4 vPosition = UnityPixelSnap(UnityObjectToClipPos(vert));
        fixed4 faceColor = i.color;

        v2f o;
        UNITY_INITIALIZE_OUTPUT(v2f,o);
        o.vertex = vPosition;
        o.color = faceColor;
        o.texcoord0 = i.texcoord0;
        o.texcoord1 = TRANSFORM_TEX(UnpackUV(i.texcoord1), _FaceTex);   
           
        float2 pixelSize = vPosition.w;
        pixelSize /= abs(float2(_ScreenParams.x * UNITY_MATRIX_P[0][0], _ScreenParams.y * UNITY_MATRIX_P[1][1]));

        float4 clampedRect = clamp(_ClipRect, -2e10, 2e10);
        o.mask = float4(vert.xy * 2 - clampedRect.xy - clampedRect.zw, 0.25 / (0.25  + pixelSize.xy));

        return o;
    }

    float4 frag(v2f i) : COLOR
    {
        fixed4 c = tex2D(_MainTex, i.texcoord0);
        c = fixed4(tex2D(_FaceTex, i.texcoord1).rgb * i.color.rgb, i.color.a * c.a);
        half2 m = saturate((_ClipRect.zw - _ClipRect.xy - abs(i.mask.xy)) * i.mask.zw);
        c *= m.x * m.y;
        return c;
    }
    ENDCG
}
  • in second pass i render outline with using stencil buffer to prevent overlapping opaque pixels:

924×346 13.1 KB

Pass
{       

    Stencil{
        Ref [_Stencil]
        Comp Equal
        Pass IncrSat
    }

    CGPROGRAM

    float4 _MainTex_TexelSize;
    uniform float4 _OutlineColor;
   
    v2f vert(appdata_t i)
    {
        v2f o;
        UNITY_INITIALIZE_OUTPUT(v2f,o);
        o.vertex = UnityPixelSnap(UnityObjectToClipPos(i.vertex));
        o.color = i.color;
        o.texcoord0 = i.texcoord0;
        return o;
    }

    float4 frag(v2f i) : COLOR
    {
        float4 c = tex2D(_MainTex, i.texcoord0) * i.color;

        float2 texelSize = _MainTex_TexelSize * _Outline;
        texelSize = float2(1.0 / 256.0, 1.0 / 256.0) * _Outline;
       
        float pL = tex2D(_MainTex, i.texcoord0 + texelSize * float2(-1, 0)).a;
        float pR = tex2D(_MainTex, i.texcoord0 + texelSize * float2(1, 0)).a;
        float pU = tex2D(_MainTex, i.texcoord0 + texelSize * float2(0, 1)).a;
        float pD = tex2D(_MainTex, i.texcoord0 + texelSize * float2(0, -1)).a;

        half outline = saturate(pL + pR + pD + pU);

        float4 outlineColor = lerp(0, _OutlineColor, outline);

        clip(outlineColor.a - 0.01);

        outlineColor = lerp(_OutlineColor, 0, c.a);
       
        return outlineColor;
    }

    ENDCG
}

So all was alright until I found that my shader affects now on Unity masking mechanics:

522×555 19.2 KB

I tried to find any solution that might prevent this problem, but I couldn’t, now I have no idea. Can know how to handle this? I’m new in shader writing.
Thank’s in advance.

2

Unity’s masking system uses stencils. You’re writing to the stencil buffer too, so it’s interfering with the masking system.

My recommendation would be to use a hard coded stencil ref that’s unlikely to be used by Unity’s own masking system. Though this will mean you cannot use a mask to limit outlined text.

Stencil {
    Ref 128
    Comp NotEqual
    Pass Replace
}
1 Like
3

Thanks for your reply!
I remarked Unity UI uses one type of masks: 0…01 → if there’s content in first mask, 0…11 → if there’s content in second mask which nested in first one, etc. But I don’t understand how Unity UI responds to rest stencil values. I would guess rest values should using in user’s shader code to implement own logic within Unity UI system.

4

In my project it’s important outlined text to be masked,so when shader has stencil > 0 next code will work correctly:

Stencil{
    Ref [_Stencil]
    Comp Equal
    Pass DecrSat
}

I thinking now about what to do when stencil buffer value is equal to zero. :slight_smile:

5

Actually, I think there is a way to solve this with DecrSat, but not in the way you’re thinking.

Keep your existing pass as is using _Stencil and IncrSat, but add another pass that does the DecrSat.

Stencil {
    Ref [_Stencil]
    Comp Greater
    Pass DecrSat
}

ColorMask 0

float4 vert(appdata_t i) : SV_Position
{
    return UnityPixelSnap(UnityObjectToClipPos(i.vertex));
}

fixed4 frag() : SV_Target { return 0.0; }

That third pass should decrement the stencil value back to what it was before. And because Unity’s masking system will never draw an object with a _Stencil value greater than the max value that is already in the stencil buffer from masks, there shouldn’t be any other issues. Unity’s masks do something similar after drawing all child objects under them, resetting the stencil to a lower value. The only issue I can think of is if you have 8 chained layers of masks, the _Stencil reference value will be 255, so IncrSat won’t do anything. So don’t have that many masks in a single hierarchy.

1 Like
6

Thanks for you reply again!
But in your code snippet there shouldn’t be Less instead of Greater? As I understood comparison function works as follows: refValue compFunction bufferValue. It seems replacing Greater function to the Less works well as you explained.

7

Oh, yep, you’re right. It’s comparing the ref to the stencil, not the stencil to the ref.

8

Can I ask you about one moment? I have not enough understanding how shader applies to the TMP meshes: per pass for every glyph(mesh)(i.e performing 1st pass from shader for every mesh, next 2nd pass, etc.) or all glyphs batched in one mesh and then shader applies to it? Thanks in advance!

9

If a TMP mesh is using a single font / sprite atlas for all of the glyphs, and you’re not using rich text to change the material mid line, it’s drawn as a single mesh with multiple glyphs in it. As far as Unity’s rendering system is concerned each TMP component is a single mesh.