Visual artifacts in baked lighting can occur for a number of reasons. These may commonly appear as unexpected dark or light patches, streaks or smearing. Often these artifacts are the result of “texel invalidity”.
Here are some examples:
These visual artifacts appear when the lightmapper marks texels (pixels in texture or lightmap space) as ‘invalid’. GI team developer @uy3d explains texel validity it like so:
When the lightmapper shoots a ray from a surface point and hits another surface, it checks its surface normal. In doing so, the lightmapper makes sure that it hit the surface from the outside. If the normal points in the wrong direction, it checks whether the object is double-sided.
If that’s also not the case, it determines that it has hit an invalid surface. For example, it might be inside a mesh or otherwise see only back-faces. This usually happens if you have set up your scene with intersecting meshes, (e.g. you have a cube halfway through the ground). In such a case, the surface texels of the ground that are covered by the cube will now hit the cube from the inside. This will in turn mark the texels as invalid. The ground outside the cube will hit the outer side of the cube, and so it will be marked as valid.
In short, valid means the lightmapper sees a front-face or a double-sided material, and invalid means it sees a back-face and doesn’t know what to do. This is because there is no definition of a surface that does not face “outwards” in path tracing.
You can inspect texel validity in the Scene View, selecting the Texel Validity view mode. Valid texels are marked as green, invalid as red, and undetermined as orange.
Fixing the issue
Solution 1: Ensure mesh normals face outwards
In the scene above, there are visible artifacts on the walls around the red cube. A quick glance at the texel validity view mode can reveal which faces are causing the artifacts.
It’s evident from this view that the red cube has some of its normals pointing inwards. This means that, while baking, rays fired from neighbouring texels on the walls see the back-faces of those sides of the cube. They are then marked as invalid (depending on the backface tolerance), and cause artifacts to appear on the neighbouring walls. To fix this, the user may ensure all normals face outwards, in the direction they will be seen by neighbouring geometry. Here’s how to do this in 3Ds Max, Maya LT and Blender.
Solution 2: Enable Double-Sided GI
Double-sided materials should be used for surfaces that receive light from both sides. This includes foliage, grass, cutouts or any surface which has its back-faces visible to neighbouring texels. You can find the option to enable double-sided GI in the Material Inspector.
This option is off by default to ensure that texels inside intersecting geometry are - in this case correctly - marked as invalid. Here, texel invalidity prevents the leaking of either shadow or light around such objects. This occurs in situations where texels occupy both sides of a geometry intersection. Many of the rays spawned from these texels would see no light inside the objects, and would converge to a dark result and vice versa. For this reason, the default behavior is to extrapolate lighting into invalid texels. This fixes the problem of intersecting geometries.
In the Built-In rendering pipeline, “Double Sided Global Illumination” must be turned on even if the shader is authored as double-sided. This is because in Built-In, the Backface Culling shader properties are not used for Global Illumination. This is different in other SRPs. In HDRP and URP for example, making your material Double-Sided means it also appears double-sided to the lightmapper.
Solution 3: Increase Mesh Extrusion
In some cases, the better approach is to modify the mesh of objects which present back-faces to neighbouring geometry. This will need to be done in a DCC program instead of using fixes in Unity such as double-sided GI. Increasing the mesh extrusion (thickness) will create true manifold geometry with outward-facing normals. The lightmapper can then use these normals to create optimal indirect paths.
Solution 4: Tweak Backface Tolerance
Backface Tolerance determines the percentage of ray hits which are permitted to hit neighbouring backface geometry before its lightmap texels are marked as invalid.
Backface Tolerance is found in the Lightmap Parameters asset. It can be assigned to geometry using the Mesh Renderer component. Adjust this parameter if you intentionally or unavoidably have objects revealing back-faces to neighbouring texels. Increasing Backface Tolerance on objects seeing this backface geometry decreases the likelihood of the lightmapper marking texels invalid.
Please note that tweaking this value could be detrimental to baked occlusion and could lead to incorrect lighting values. Sometimes invalidity is the intended outcome for certain texels, and it will not always result in visible artifacts.
If you are still having issues with artifacts, please feel free to post your issue on the forum and we will take a look.
Thanks to @kristijonas_unity and @DavidLlewelyn