Normal map flat instead of smooth in shadow

Hi. Im having a problem i cant quite figure out.

Ive got an .fbx file using a legacy/bumped diffuse shader, with a normal map. But when things are not directly in the directional light in the map, things get flat instead of smooth. Notice left arm of the shirt.

The normal map looks like this, smooth.

Sometimes this works perfectly fine, but if i overwrite the .fbx file with a copy of the same .fbx file, it will often become messed up.

Anyone got some ideas for what could cause this?

Images are not showing up.