Baked Normal Map Discrepancies

Questions & Answers
1

I’ve baked out normal maps for various objects, but I’m confronted with this problem;
[816-Portrait+of+a+brick;+an+abstract+masterpiece.png|816]

In this quick example I’ve taken a high-poly model (shown left, in Sculptris) and an equivalent low-poly made in Blender (shown right, in Unity). Both models were exported as .obj files to xNormal where everything was baked into a normal map. Annoyingly all the small bevelled faces now appear concave and the large faces appear convex. I can fix this of course by re-baking it all for a hard-edged model and setting the smoothing angle to 0 in the Unity .fbx importer for the low-poly. This leads to very noticeable seams, as would be expected, so it’s not really an acceptable workaround.

So first off, is this something I’m doing wrong, or is there a problem with Unity as it stands? I’ve been treating it as my fault for a while now, but this thread made me think this might not be the situation (this is also why I’m posting it here as a Unity specific question, as opposed to asking in forums for other programs);

http://forum.unity3d.com/threads/38781-Calculating-Unity-s-tangent-basis-for-xNormal

To my amature eyes it certainly does look as though the smoothing calculations aren’t all matching up as they should.

Secondly if it’s not my fault does anyone know of a good workaround, or better yet a fix for this? Does anyone currently not have this problem with these kinds of models, and if so what are you doing to make normal maps?

2

How does your lowpoly look like? Smoothing Groups and UVs are most important. You should ALWAYS have different Smoothing Groups for different UV shells and vice versa. Try splitting the UVs/Smoothing Groups accordingly and rebake.

I don’t have experience with this with Blender, though. But I had a normal mapped asset from Max in Unity and it looked fine (Tangent base used: R: X+, G: Y+, B: Z+), not as good as in Marmoset, but no normal map errors. I used my own shader, though, so this might have been a factor, too.