"Normal Extrusion" on sprites

[Peritectoid]

I am making a shader that distorts sprites, but I need to increase the area of the sprite because otherwise distorted parts go outside of the rendered area.
I have tried using this code example provided by Unity (see “Normal Extrusion with Vertex Modifier”).

float _Amount;
void vert (inout appdata_full v)
{
  v.vertex.xyz += v.normal * _Amount;
}

Unfortunately sprites’ normals seem to point in the z-direction, so the area is not increased but rather the sprite is moved forward (or backward, depending on the value of _Amount).

I think if I knew the position of the sprites’ center (within the vertex shader code) I could pull this off.

Thank you in advance.

[bgolus]

@Peritectoid

I think if I knew the position of the sprites’ center (within the vertex shader code) I could pull this off.

You are correct here … and Unity’s sprite system does not have a way for you to get the data you want / need for this. Even if you did, assuming your sprites are atlased, there’s not a lot of room to expand to (maybe just 2 pixels).

You’re better off doing all your distortions to the sprite vertices if you can.

[Peritectoid]

I gave up on automatically finding the normals and decided to just let the user manually set the pivot position. Here is the shader I came up with:

Shader "Sprites/Diffuse-Distortion"
{
   Properties
   {
       [PerRendererData] _MainTex ("Sprite Texture", 2D) = "white" {}
       _Color ("Tint", Color) = (1,1,1,1)
       _Cutoff ("Shadow alpha cutoff", Range(0,1)) = 0.5
       //Distortion map; Only the red and green channel are read
       _Distortion ("Distortion map (RGB)", 2D) = "white" {}
       _SpeedX("Speed X", range(-1, 1))=0.1
       _SpeedY("Speed Y", range(-1, 1))=0.1
       //Scale of the distortion map
       _Scale("Scale", range(0.0001, 10))=1
       //How many pixels are in one unit in world space
       _Pixel ("Pixel per unit", int)=16
       //The maximum displacement in pixels
       _DisplacementX("Displacement X", range(0, 128))=1
       _DisplacementY("Displacement Y", range(0, 128))=1
       //For sprites that are not centered
       _PivotOffsetX ("Pivot offset X", Range(-1,1))=0
       _PivotOffsetY ("Pivot offset Y", Range(-1,1))=0
       //Optional parameters that are to be manipulated with external scripts to, for instance, offset the additional accrued distortion of moving objects
       _PositionOffsetX ("Position offset X", float)=0
       _PositionOffsetY ("Position offset Y", float)=0
       //Interpolation of the pixels from the distortion map
       [MaterialToggle] _Smooth("Smooth", Float) = 1
   }
         
   SubShader
   {
       Tags
       {
           "Queue"="Transparent"
           "IgnoreProjector"="True"
           "RenderType"="Transparent"
           "PreviewType"="Plane"
           "CanUseSpriteAtlas"="True"
           "DisableBatching"="True"
       }
       LOD 200
       
       Cull Off
       Lighting On
       ZWrite Off
       Blend One OneMinusSrcAlpha
           
       CGPROGRAM
       #pragma surface surf Ramp vertex:vert alphatest:_Cutoff
     
       sampler2D _MainTex;
       sampler2D _Distortion;
     
       float _SpeedX;
       float _SpeedY;
       float _Scale;
       float _DisplacementX;
       float _DisplacementY;
     
       struct Input
       {
           float2 uv_MainTex;
           float2 uv_Distortion;
           float3 worldPos;
       };

       half4 LightingRamp (SurfaceOutput s, half3 lightDir, half atten)
       {
           half4 c;
           c.rgb = s.Albedo;
           c.a = s.Alpha;
           return c;
       }

       float _Pixel;
       float _PivotOffsetX;
       float _PivotOffsetY;
       void vert (inout appdata_full v)
       {
           const float2 displacement=float2(_DisplacementX,_DisplacementY);
           float3 direction=sign(v.vertex).xyz;
           if(direction.x==0 && _PivotOffsetX!=0)
           {
               direction.x=_PivotOffsetX;
           }
           if(direction.y==0 && _PivotOffsetY!=0)
           {
               direction.y=-_PivotOffsetY;
           }
           v.vertex.xy += direction*displacement.xy/_Pixel;
       }

       float Repeat (float value, float ceiling)
       {
           while(value<0)
           {
               value+=ceiling;
           }
           while(value>=ceiling)
           {
               value-=ceiling;
           }
           return value;
       }

       float2 Repeat (float2 value, float ceiling)
       {
           return float2(Repeat(value.x,ceiling),Repeat(value.y,ceiling));
       }

       float2 Repeat (float2 value, float2 ceiling)
       {
           return float2(Repeat(value.x,ceiling.x),Repeat(value.y,ceiling.y));
       }

       float4 Point(sampler2D tex, float2 uv, float4 texelSize)
       {
           float2 halfPixel=0.5*texelSize.xy;

           uv*=texelSize.zw;
           uv=floor(uv)+0.5;
           uv*=texelSize.xy;

           return tex2D(_Distortion, uv);
       }

       float4 Bilinear(sampler2D tex, float2 uv, float4 texelSize)
       {
           float2 halfPixel=0.5*texelSize.xy;

           uv*=texelSize.zw;
           float2 weight=uv;
           uv=round(uv);
           weight-=uv;
           weight+=0.5;
           uv*=texelSize.xy;

           float4 pixel1=tex2D(_Distortion, Repeat(uv+float2(-halfPixel.x,-halfPixel.y),1));
           float4 pixel2=tex2D(_Distortion, Repeat(uv+float2(halfPixel.x,-halfPixel.y),1));
           float4 pixel3=tex2D(_Distortion, Repeat(uv+float2(-halfPixel.x,halfPixel.y),1));
           float4 pixel4=tex2D(_Distortion, Repeat(uv+float2(halfPixel.x,halfPixel.y),1));
           float4 pixel12=lerp(pixel1,pixel2,weight.x);
           float4 pixel34=lerp(pixel3,pixel4,weight.x);

           return lerp(pixel12,pixel34,weight.y);
       }

       float4 BilinearAtBorders(sampler2D tex, float2 uv, float4 texelSize)
       {
           float2 halfPixel=0.5*texelSize.xy;

           if(uv.x<=halfPixel.x || uv.y<=halfPixel.y || uv.x>=1-halfPixel.x || uv.y>=1-halfPixel.y)
           {
               return Bilinear(tex,uv,texelSize);
           }
           return(tex2D(tex,uv));
       }

       float4 _Color;
       float _Alpha;

       float4 _MainTex_TexelSize;
       float4 _Distortion_TexelSize;


       float _PositionOffsetX;
       float _PositionOffsetY;
       float _Smooth;

       void surf (Input IN, inout SurfaceOutput o)
       {
           const float2 displacement=float2(_DisplacementX,_DisplacementY);
           float2 distortionSeed=float2(0,0);
           float2 finalScale=_Scale*_Pixel*_Distortion_TexelSize.xy;
           distortionSeed.x+=(_Time.g*_SpeedX)-_PositionOffsetX*finalScale.x;
           distortionSeed.y+=(_Time.g*_SpeedY)-_PositionOffsetY*finalScale.y;
           distortionSeed+=IN.worldPos.xy*finalScale;

           distortionSeed=Repeat(distortionSeed,1);

           float2 offset;
           if(_Smooth!=0)
           {
               offset=BilinearAtBorders(_Distortion,distortionSeed,_Distortion_TexelSize).rg;
           }
           else
           {
               offset=Point(_Distortion,distortionSeed,_Distortion_TexelSize).rg;
           }

           offset-=0.5;
           offset*=2*displacement*_MainTex_TexelSize.xy;

           float2 uv=IN.uv_MainTex;
           uv-=0.5;
           uv*=_MainTex_TexelSize.xy*(_MainTex_TexelSize.zw+displacement*2);
           uv+=0.5;
           uv+=offset;

           fixed4 c = tex2D (_MainTex, uv);
           o.Albedo=c.rgb*_Color;
           o.Alpha=c.a*_Color.a;
           if(uv.x<0 || uv.y<0 || uv.x>1 || uv.y>1)
           {
               o.Alpha=0;
           }
       }
       ENDCG
   }
   Fallback "Sprites/Diffuse"
}

Here’s the distortion map I use:

There’s still a little problem that is bugging me; I had to implement the bilinear interpolation for the borders of the distortion map myself and my solution causes ugly, “infinitely small” artifacts around them.
I have tried for a long time and I still don’t know how to get rid of them.

This error is not immediately apparent, but once you know it’s there you can’t help but notice.

If somebody is interested in trying to fix this (which I would be very grateful for) insert the following code snippet to better visualize this error:

        void surf (Input IN, inout SurfaceOutput o)
        {
           const float2 displacement=float2(_DisplacementX,_DisplacementY);
           float2 distortionSeed=float2(0,0);
           float2 finalScale=_Scale*_Pixel*_Distortion_TexelSize.xy;
           distortionSeed.x+=(_Time.g*_SpeedX)-_PositionOffsetX*finalScale.x;
           distortionSeed.y+=(_Time.g*_SpeedY)-_PositionOffsetY*finalScale.y;
           distortionSeed+=IN.worldPos.xy*finalScale;

            distortionSeed=Repeat(distortionSeed,1);

            float2 offset;
            if(_Smooth!=0)
            {
                offset=BilinearAtBorders(_Distortion,distortionSeed,_Distortion_TexelSize).rg;
            }
            else
            {
                offset=Point(_Distortion,distortionSeed,_Distortion_TexelSize).rg;
            }

            float2 expose=offset;

            offset-=0.5;
            offset*=2*displacement*_MainTex_TexelSize.xy;

            offset=0.5;

            float2 uv=IN.uv_MainTex;
            uv-=0.5;
            uv*=_MainTex_TexelSize.xy*(_MainTex_TexelSize.zw+displacement*2);
            uv+=0.5;
            uv+=offset;

            fixed4 c = tex2D (_MainTex, uv);
            o.Albedo=c.rgb*_Color;
            o.Alpha=c.a*_Color.a;
            if(uv.x<0 || uv.y<0 || uv.x>1 || uv.y>1)
            {
                o.Alpha=0;
            }
            o.Albedo=expose.x;
        }

Furthermore, use this (very tiny image) as the distortion map and enable the “Smooth” parameter in the shader.

If anyone has any suggestions for any further improvements, again, I’d be grateful, and if you have any questions, feel free to ask them.

[bgolus]

@Peritectoid

There’s still a little problem that is bugging me; I had to implement the bilinear interpolation for the borders of the distortion map myself and my solution causes ugly, “infinitely small” artifacts around them.

You need to use manual UV derivatives.
https://forum.unity3d.com/threads/texture2d-array-mipmap-troubles.416799/#post-2715201

[Peritectoid]

@bgolus

You need to use manual UV derivatives.
https://forum.unity3d.com/threads/texture2d-array-mipmap-troubles.416799/#post-2715201

Sorry if I sound stupid, I am a bit tired right now.
I have played around with tex2Dgrad a bit now and I am not getting the results I expected; it got even worse on a few tries.
What do I put into ddx() and ddy()? The world position of the fragment? All the documentation I can find on it is too vague for me to grasp.

Lastly, would this solution remove my need to manually bilinearly interpolate the border values myself?

Thanks in advance.

[bgolus]

On second thought, that might not be the solution, not exactly. I haven’t run the shader itself to check, but basically I’m assuming the problem you’re seeing is from mip mapping.

To explain tex2Dgrad and ddx, basically behind the scenes this line of code:
tex2D(_Tex, uv);
Is actually doing the equivalent of:
tex2Dgrad(_Tex, uv, ddx(uv), ddy(uv));

The short explanation of what ddx and ddy are is “how much does this value change between screen pixels on the x or y axis”. There’s some finer details there, but that’s just the short version. The important part is this is how the GPU knows what mip map level to use for a texture. Does the value change a lot, use a smaller mip. Barely changes at all, use the largest mip. When you’re doing UV manipulation that value can change significantly between pixels, like from your “repeat” function (which, FYI, is what the fmod() function is for). The reason to use tex2Dgrad is to tell the GPU, “ignore the derivatives of the UV I’m passing in, and use these instead to choose the mip level”.

So my updated suggestion for you is either turn off mip maps on the texture, and / or use tex2Dlod and see if that fixes the issue. If it does you’ll need to do a “bit” more work if you need those mipmaps.

If you think about this long enough and think, “wait, there’s an if statement there, Bilinear() doesn’t get called every pixel, so those UVs don’t get calculated for every pixel, so they shouldn’t be changing!”, you’re wrong. Because tex2D needs to know those derivatives the Bilinear() function does get run for every pixel! This is one of the situations that cause people to say “don’t use if statements, they’re expensive!” It’s not that they’re expensive, it’s that they don’t always work like people expect. If you switch to tex2Dlod, or use tex2Dgrad and calculate the ddx and ddy from the unmodified UVs prior to the Bilinear() function call the GPU can actually skip the unneeded work entirely.