Precision issues when subtracting values of adjacent texels

[trzy]

I’m attempting to learn about URP shaders by converting Splatoonity to URP. It’s essentially working but I’ve encountered a strange artifact that I suspect is caused by a subtraction. Interestingly, this is not present in the original non-URP code base.

The splat decal ought to be smooth but looks like this instead:

This is caused by the bump mapping calculation that attempts to create a bump at the edge of the splat by computing per-fragment normals from the intensity map that forms the splat. The relevant code is:

// Sample splat map texture (the world texture with all splats, indexed by the lightmap UVs) with offsets
float4 splatSDF = SAMPLE_TEXTURE2D(_SplatTex, sampler_SplatTex, uv_SplatTex);
float4 splatSDFx = SAMPLE_TEXTURE2D(_SplatTex, sampler_SplatTex, uv_SplatTex + float2(_SplatTex_TexelSize.x, 0));
float4 splatSDFy = SAMPLE_TEXTURE2D(_SplatTex, sampler_SplatTex, uv_SplatTex + float2(0, _SplatTex_TexelSize.y));

...

// Create normal offset for each splat channel, in the plane of the surface
float4 offsetSplatX = splatSDF - splatSDFx;
float4 offsetSplatY = splatSDF - splatSDFy;

The problem appears to be caused by those last two subtractions. If we draw offsetSplatX.x, we see:

But in the original this quantity looks more like:

I’m completely stumped by what could be causing this. My code is using HLSL and the original is older Cg ShaderLab code. Same version of Unity running both (2020.2). Graphics settings are essentially the same.

Because the offsets are normalized later anyway, I’ve tried computing the sign() as well as “splatSDF/splatSDFx-1” to no avail.

Shader debugging in MSVC and RenderDoc isn’t working for me (I can’t get either to show the shader source or even a disassembly). But I’m pretty confident the subtraction is where the problem is. Every other quantity appears to be the same between the Cg and HLSL/URP versions of the shaders.

Any pointers on how to debug or resolve this would be much appreciated!

[aleksandrk]

What happens if you visualise splatSDF, splatSDF.x and splatSDF.y? Do they differ from what was produced by the old shader?

[trzy]

I don’t think so. Note that we only care about the .x component here (xyzw correspond to four possible splat colors). Visually, the SDFs look smooth with no artifacts in both cases. SDFx and SDFy (which are the SDF texture offset by one texel in x and y) look about the same.

I can’t rule out an issue with texture sampling. The SplatTex is an 8192x8192 map created at run time (same exact code in both versions) that corresponds exactly to a lightmap texture — it contains all static geometry unwrapped the same way.

But 1.0/8192.0 should not cause any precision issues. Replacing _SplatTex_TexelSize.x with a literal 1/8192 has no effect.

Lastly, as a test, I tried sampling out 10 or even more texels out. This only slightly reduced the artifacts but it is hard to draw conclusions. It could be that the difference in the SDF values is still quite small. Although, on the other hand, the gradient should be largest near the edge of the splats so if anything I would expect that precision issues are less likely there… Hmmm…

[aleksandrk]

How do you declare the textures and what texture format do you use?

[trzy]

@aleksandrk The code is the exact same C# code in both cases:

splatTex = new RenderTexture (sizeX, sizeY, 0, RenderTextureFormat.ARGB32, RenderTextureReadWrite.Linear);
splatTex.Create ();
splatTexAlt = new RenderTexture (sizeX, sizeY, 0, RenderTextureFormat.ARGB32, RenderTextureReadWrite.Linear);
splatTexAlt.Create ();

sizeX and sizeY are both set to 8192.

At run-time, I can see that bilinear sampling is being used (changing it to point from Inspector, by the way, makes the problem even worse).

[aleksandrk]

I suggest to try and compare preprocessed shader source between the two versions.

[trzy]

You mean by viewing the generated code for each? Any ideas on what to look for? The old code is written as a surface shader and I’ve converted the new code to follow a URP lit shader template.

The problematic lines are identical between shaders (except for the change from tex2d() to SAMPLE_TEXTURE2D()) but I can have a look later today. Might be beyond my skill level to pick up on any subtleties.

[aleksandrk]

No, since you’re on 2020.2 there’s a checkbox “Preprocess only” right above the “Show compiled code” button. If you toggle it, it will show preprocessed source, which you can then compare. I’d look at the places where the texture you’re sampling is declared and where it gets sampled.

[trzy]

Thanks for helping guide me here. Gave that a try. Here is pretty much the complete preprocessed code (lines 39-41, 52-53 are the relevant ones):

Texture2D _SplatTileNormalTex;
SamplerState sampler_SplatTileNormalTex;
Texture2D _SplatTex;
SamplerState sampler_SplatTex;
Texture2D _WorldTangentTex;
SamplerState sampler_WorldTangentTex;
Texture2D _WorldBinormalTex;
SamplerState sampler_WorldBinormalTex;
cbuffer SplatShaderLitBuffer {
    float4 _SplatTileNormalTex_ST;
    float _SplatTileBump;
    float _SplatEdgeBump;
    float _SplatEdgeBumpWidth;
    float4 _SplatTex_ST;
    float4 _SplatTex_TexelSize;
};
struct Attributes
{
    float4 positionOS : POSITION;
    float3 normalOS : NORMAL;
    float4 tangentOS : TANGENT;
    float2 uv : TEXCOORD0;
    float2 uvLM : TEXCOORD1;
};
struct Varyings
{
    float2 uv : TEXCOORD0;
    float2 uvLM : TEXCOORD1;
    float4 positionWSAndFogFactor : TEXCOORD2;
    half3 normalWS : TEXCOORD3;
    half3 tangentWS : TEXCOORD4;
    half3 bitangentWS : TEXCOORD5;
    float2 uv_SplatTex : TEXCOORD7;
    float4 positionCS : SV_POSITION;
};
void ComputeSplat (out float4 splatMask, out half3 splatNormal, float2 uv_SplatTex, half3 tangentWS, half3 binormalWS)
{
    float4 splatSDF = _SplatTex . Sample (sampler_SplatTex, uv_SplatTex);
    float4 splatSDFx = _SplatTex . Sample (sampler_SplatTex, uv_SplatTex + float2 (_SplatTex_TexelSize . x, 0));
    float4 splatSDFy = _SplatTex . Sample (sampler_SplatTex, uv_SplatTex + float2 (0, _SplatTex_TexelSize . y));
    half splatDDX = length (ddx (uv_SplatTex * _SplatTex_TexelSize . zw));
    half splatDDY = length (ddy (uv_SplatTex * _SplatTex_TexelSize . zw));
    half clipDist = sqrt (splatDDX * splatDDX + splatDDY * splatDDY);
    half clipDistHard = max (clipDist * 0.01, 0.01);
    half clipDistSoft = 0.01 * _SplatEdgeBumpWidth;
    static const float _Clip = 0.5;
    splatMask = smoothstep ((_Clip - 0.01) - clipDistHard, (_Clip - 0.01) + clipDistHard, splatSDF);
    float splatMaskTotal = max (max (splatMask . x, splatMask . y), max (splatMask . z, splatMask . w));
    float4 splatMaskInside = smoothstep (_Clip - clipDistSoft, _Clip + clipDistSoft, splatSDF);
    splatMaskInside = max (max (splatMaskInside . x, splatMaskInside . y), max (splatMaskInside . z, splatMaskInside . w));
    float4 offsetSplatX = splatSDF - splatSDFx;
    float4 offsetSplatY = splatSDF - splatSDFy;
    float2 offsetSplat = lerp (float2 (offsetSplatX . x, offsetSplatY . x), float2 (offsetSplatX . y, offsetSplatY . y), splatMask . y);
    offsetSplat = lerp (offsetSplat, float2 (offsetSplatX . z, offsetSplatY . z), splatMask . z);
    offsetSplat = lerp (offsetSplat, float2 (offsetSplatX . w, offsetSplatY . w), splatMask . w);
    offsetSplat = normalize (float3 (offsetSplat, 0.0001)) . xy;
    offsetSplat = offsetSplat * (1.0 - splatMaskInside) * _SplatEdgeBump;
    float2 splatTileNormalTex = _SplatTileNormalTex . Sample (sampler_SplatTileNormalTex, uv_SplatTex * 10.0) . xy;
    offsetSplat += (splatTileNormalTex . xy - 0.5) * _SplatTileBump * 0.2;
    float3 worldTangentTex = _WorldTangentTex . Sample (sampler_WorldTangentTex, uv_SplatTex) . xyz * 2.0 - 1.0;
    float3 worldBinormalTex = _WorldBinormalTex . Sample (sampler_WorldBinormalTex, uv_SplatTex) . xyz * 2.0 - 1.0;
    float3 offsetSplatWorld = offsetSplat . x * worldTangentTex + offsetSplat . y * worldBinormalTex;
    float2 offsetSplatLocal = 0;
    offsetSplatLocal . x = dot (tangentWS, offsetSplatWorld);
    offsetSplatLocal . y = dot (binormalWS, offsetSplatWorld);
    splatNormal = offsetSplatWorld * splatMaskTotal;
}
Varyings LitPassVertex (Attributes input)
{
    Varyings output;
    VertexPositionInputs vertexInput = GetVertexPositionInputs (input . positionOS . xyz);
    VertexNormalInputs vertexNormalInput = GetVertexNormalInputs (input . normalOS, input . tangentOS);
    float fogFactor = ComputeFogFactor (vertexInput . positionCS . z);
    output . uv = ((input . uv . xy) * _BaseMap_ST . xy + _BaseMap_ST . zw);
    output . uvLM = input . uvLM . xy * unity_LightmapST . xy + unity_LightmapST . zw;
    output . uv_SplatTex = ((input . uvLM . xy) * _SplatTex_ST . xy + _SplatTex_ST . zw);
    output . positionWSAndFogFactor = float4 (vertexInput . positionWS, fogFactor);
    output . normalWS = vertexNormalInput . normalWS;
    output . tangentWS = vertexNormalInput . tangentWS;
    output . bitangentWS = vertexNormalInput . bitangentWS;
    output . positionCS = vertexInput . positionCS;
    return output;
}
half4 LitPassFragment (Varyings input) : SV_Target
{
    SurfaceData surfaceData;
    InitializeStandardLitSurfaceData (input . uv, surfaceData);
    float4 splatMask;
    half3 splatNormal;
    ComputeSplat (splatMask, splatNormal, input . uv_SplatTex, input . tangentWS, input . bitangentWS);
    float3 splatAlbedo = surfaceData . albedo;
    splatAlbedo = lerp (splatAlbedo, float3 (1.0, 0.5, 0.0), splatMask . x);
    splatAlbedo = lerp (splatAlbedo, float3 (1.0, 0.0, 0.0), splatMask . y);
    splatAlbedo = lerp (splatAlbedo, float3 (0.0, 1.0, 0.0), splatMask . z);
    splatAlbedo = lerp (splatAlbedo, float3 (0.0, 0.0, 1.0), splatMask . w);
    half3 normalWS = TransformTangentToWorld (surfaceData . normalTS,
    half3x3 (input . tangentWS, input . bitangentWS, input . normalWS));
    normalWS = normalize (normalWS + splatNormal);
    half3 bakedGI = SampleSH (normalWS);
    float3 positionWS = input . positionWSAndFogFactor . xyz;
    half3 viewDirectionWS = SafeNormalize (GetCameraPositionWS () - positionWS);
    BRDFData brdfData;
    InitializeBRDFData (splatAlbedo, surfaceData . metallic, surfaceData . specular, surfaceData . smoothness, surfaceData . alpha, brdfData);
    Light mainLight = GetMainLight ();
    half3 color = GlobalIllumination (brdfData, bakedGI, surfaceData . occlusion, normalWS, viewDirectionWS);
    color += LightingPhysicallyBased (brdfData, mainLight, normalWS, viewDirectionWS);
    color += surfaceData . emission;
    float fogFactor = input . positionWSAndFogFactor . w;
    color = MixFog (color, fogFactor);
    return half4 (color, surfaceData . alpha);
}

ComputeSplat() gets called from the fragment shader and right at the beginning the texture is sampled. Looks pretty straightforward to me.

The original shader once preprocessed looks like this:

void surf (Input IN, inout SurfaceOutputStandard o) {
    float4 splatSDF = tex2D (_SplatTex, IN . uv2_SplatTex);
    float4 splatSDFx = tex2D (_SplatTex, IN . uv2_SplatTex + float2 (_SplatTex_TexelSize . x, 0));
    float4 splatSDFy = tex2D (_SplatTex, IN . uv2_SplatTex + float2 (0, _SplatTex_TexelSize . y));
    half splatDDX = length (ddx (IN . uv2_SplatTex * _SplatTex_TexelSize . zw));
    half splatDDY = length (ddy (IN . uv2_SplatTex * _SplatTex_TexelSize . zw));
    half clipDist = sqrt (splatDDX * splatDDX + splatDDY * splatDDY);
    half clipDistHard = max (clipDist * 0.01, 0.01);
    half clipDistSoft = 0.01 * _SplatEdgeBumpWidth;
    float4 splatMask = smoothstep ((_Clip - 0.01) - clipDistHard, (_Clip - 0.01) + clipDistHard, splatSDF);
    float splatMaskTotal = max (max (splatMask . x, splatMask . y), max (splatMask . z, splatMask . w));
    float4 splatMaskInside = smoothstep (_Clip - clipDistSoft, _Clip + clipDistSoft, splatSDF);
    splatMaskInside = max (max (splatMaskInside . x, splatMaskInside . y), max (splatMaskInside . z, splatMaskInside . w));
    float4 offsetSplatX = splatSDF - splatSDFx;
    float4 offsetSplatY = splatSDF - splatSDFy;
    float2 offsetSplat = lerp (float2 (offsetSplatX . x, offsetSplatY . x), float2 (offsetSplatX . y, offsetSplatY . y), splatMask . y);
    offsetSplat = lerp (offsetSplat, float2 (offsetSplatX . z, offsetSplatY . z), splatMask . z);
    offsetSplat = lerp (offsetSplat, float2 (offsetSplatX . w, offsetSplatY . w), splatMask . w);
    offsetSplat = normalize (float3 (offsetSplat, 0.0001)) . xy;
    offsetSplat = offsetSplat * (1.0 - splatMaskInside) * _SplatEdgeBump;
    float3 worldTangentTex = tex2D (_WorldTangentTex, IN . uv2_SplatTex) . xyz * 2.0 - 1.0;
    float3 worldBinormalTex = tex2D (_WorldBinormalTex, IN . uv2_SplatTex) . xyz * 2.0 - 1.0;
    float3 offsetSplatWorld = offsetSplat . x * worldTangentTex + offsetSplat . y * worldBinormalTex;
    float3 worldTangent = float3 (1, 0, 0);
    float3 worldBinormal = float3 (0, 1, 0);
    float2 offsetSplatLocal = 0;
    offsetSplatLocal . x = dot (worldTangent, offsetSplatWorld);
    offsetSplatLocal . y = dot (worldBinormal, offsetSplatWorld);
    float4 normalMap = tex2D (_BumpTex, IN . uv_MainTex);
    normalMap . xyz = UnpackNormal (normalMap);
    float3 tanNormal = normalMap . xyz;
    tanNormal . xy += offsetSplatLocal * splatMaskTotal;
    tanNormal = normalize (tanNormal);
    float4 MainTex = tex2D (_MainTex, IN . uv_MainTex);
    half4 c = MainTex * _Color;
    c . xyz = lerp (c . xyz, float3 (1.0, 0.5, 0.0), splatMask . x);
    c . xyz = lerp (c . xyz, float3 (1.0, 0.0, 0.0), splatMask . y);
    c . xyz = lerp (c . xyz, float3 (0.0, 1.0, 0.0), splatMask . z);
    c . xyz = lerp (c . xyz, float3 (0.0, 0.0, 1.0), splatMask . w);
    o . Albedo = c . rgb;
    o . Albedo = offsetSplatX . xyz * 20;
    o . Metallic = _Metallic;
    o . Smoothness = lerp (_Glossiness, 0.7, splatMaskTotal);
    o . Alpha = c . a;
}

Seems effectively identical?

[trzy]

I discovered that changing the texture type from RenderTextureFormat.ARGB32 to RenderTextureFormat.ARGBFloat improves things a little (see below) but it is still far from perfect and the original shader does not need this modification.

The splats are blitted into SplatTex using a different shader, which I’ve also translated to URP. As far as I can tell (I’ve examined the RenderTexture by dumping it to disk), there’s nothing wrong there and the aliasing is caused by the shader I posted earlier.

Edit: ARGBFloat should be completely unnecessary because the splat source texture atlas is a tga file in RGB8 format. There should not be any substantial change in precision.

[trzy]

I think I know what’s going on. My render texture is way too large for the tiny scene I have. I only have a cube and a few spheres in my scene whereas the original codebase has a substantial amount of geometry. As a result, the splats are rendered in very high fidelity to the 8192x8192 render texture and the delta between any two neighboring texels is 0 or very near to it. I think that is the primary source of the noise.

I tried creating a project with using the built-in render pipeline rather than URP and I do still see some difference in the characteristic of the noise for similar sized scenes but it’s difficult to say conclusively. By reducing the render texture from 8192x8192 to 2048x2048 in my scene, I have:

Now to figure out how to add gloss, squash some warnings about keywords being declared both globally and locally, and move on