When using a physically based sky and rotating the sun, you incur a big performance cost on the GPU.
consistently around 1.8ms - 2.5ms, on a GTX 1070. That’s a good chunk of a 60FPS budget.
when I have volumetric clouds in the scene, the cost is increased by quite a bit and becomes 2.5ms-2.6ms. 
Any workaround/hacks to improve this?
Actually, there’s an onDemand update mode, I can just use that and reduce the sky update rate.
This makes things better, and usable, but i’m still interested to see if anyone knows how to improve this further because the performance cost is still the same, it just doesn’t update each frame.
OnDemand could also lead to some visual issues when I need to update the sky more than once in a blue moon due to the 2ms cost, depending on how fast the directional light is moving. And the atmosphere change wouldn’t look as smooth, so I’m still interested in anything that would improve the performance cost.