# Using Soble filter to create normal map from height map creates vertical and horizontal artifacts

I’m calculating a height map curvature in a shader,
for small gradients it works nicely but for larger gradients it creates jittering artifacts.

the first step in this shader is using a 3x3 Soble filter to create the normal from the height map,
at this point you can see that with large gradients it starts showing the jittering artifacts, which leads to the artifacts in the curvature map as well.

is there a way of getting a smoother normal maps? without passing the normal map through another blur filter?
am I missing something in my implementation?

The filtered texture is a 1024x1024 render texture - ARGB64 (16 bit per channle)

``````fixed4 frag(v2f i) : SV_Target
{
float3 normal = sobleFilter3x3(_MainTex, _MainTex_TexelSize.x, i.uv, _Scale, _SobleSampleDist);

// calculate curvature
float3 derX = ddx(normal);
float3 derY = ddy(normal);
float3 xn = normal - derX;
float3 xp = normal + derX;
float3 yn = normal - derY;
float3 yp = normal + derY;

float s = tex2D(_MainTex, i.uv);
float3 pos = float3(i.uv.x, i.uv.y, s);
float mag = length(pos);

float curvature = (cross(xn, xp).y - cross(yn, yp).x) * _Scale / mag;

return fixed4((curvature + _Concave).xxx, 1);
}

float3 sobleFilter3x3(sampler2D tex, float texalSize, float2 uv, float scale, float sampleDist)
{
/*
[6][7][8]
[3][4][5]
*/
float ts = texalSize;
float s[9];
float2 off = {ts * sampleDist, -ts * sampleDist };// offsets
s[0] = tex2D(tex, uv + float2(off.y, off.y));
s[1] = tex2D(tex, uv + float2(0    , off.y));
s[2] = tex2D(tex, uv + float2(off.x, off.y));

s[3] = tex2D(tex, uv + float2(off.y, 0));
s[5] = tex2D(tex, uv + float2(off.x, 0));

s[6] = tex2D(tex, uv + float2(off.y, off.x));
s[7] = tex2D(tex, uv + float2(0    , off.x));
s[8] = tex2D(tex, uv + float2(off.x, off.x));

float3 normal;
normal.x = scale * -(s[2] - s[0] + 2 * (s[5] - s[3]) + s[8] - s[6]);
normal.y = scale * -(s[6] - s[0] + 2 * (s[7] - s[1]) + s[8] - s[2]);
normal.z = 1.0;
return normalize(normal) * 0.5 + 0.5;
}
``````

@NanushTol - Instead of using the Sobel filter, you can try using the Scharr filter, which is known to produce smoother results for edge detection. It has a similar structure to the Sobel filter but uses different kernel values. Replace the Sobel filter code in your shader with the following Scharr filter code:

``````float3 scharrFilter3x3(sampler2D tex, float texelSize, float2 uv, float scale, float sampleDist)
{
/*
[6][7][8]
[3][4][5]
[0][1][2]
*/
float ts = texelSize;
float s[9];
float2 off = {ts * sampleDist, -ts * sampleDist}; // offsets
s[0] = tex2D(tex, uv + float2(off.y, off.y));
s[1] = tex2D(tex, uv + float2(0, off.y));
s[2] = tex2D(tex, uv + float2(off.x, off.y));
s[3] = tex2D(tex, uv + float2(off.y, 0));
s[5] = tex2D(tex, uv + float2(off.x, 0));

s[6] = tex2D(tex, uv + float2(off.y, off.x));
s[7] = tex2D(tex, uv + float2(0, off.x));
s[8] = tex2D(tex, uv + float2(off.x, off.x));

float3 normal;
normal.x = scale * -(3 * (s[2] - s[0]) + 10 * (s[5] - s[3]) + 3 * (s[8] - s[6]));
normal.y = scale * -(3 * (s[6] - s[0]) + 10 * (s[7] - s[1]) + 3 * (s[8] - s[2]));
normal.z = 1.0;
return normalize(normal) * 0.5 + 0.5;
}
``````

Now, replace the call to sobleFilter3x3 with scharrFilter3x3 in your frag function, like this:

``````float3 normal = scharrFilter3x3(_MainTex, _MainTex_TexelSize.x, i.uv, _Scale, _SobleSampleDist);
``````

By using the Scharr filter instead of the Sobel filter, you should be able to get smoother normal maps without the need for an additional blur filter. Give it a try and let me know if this gets you closer to an answer.