Normal maps showing up white.

I have a scene with multiple cubes with bumped diffuse shaders applied to them. The cubes sometimes appear white with the normal map applied and change (show up normal/appear white) depending on the view angle.

Are your normal maps set as normal maps in the inspector?

Yes they are. When I first applied them to the shader it said I needed to fix them which I did. I’m not generating from greyscale. It works fine if I only have one cube but as soon as I have multiple with the same shader it messes up.

Without normal:

With normal:

I only have one directional light in my scene so I tested it with a spot light and it works fine then. Do normal maps work with only directional lights?