Skybox ground color is orange and makes everything that faces down orange too

I think you can either bake global illumination (ex. lightmapping ) or use a real-time GI solution (Enlighten, HDRP SSGI, …).

Have you tried adjusting the environment lighting intensity multiplier? This is a simple way to approximate indoor GI as long as the results are acceptable.