[Help Wanted] Understanding DynamicGI vs Reflection Probe Lighting Behavior

Unity Engine
1

I am encountering a lighting behavior that I can’t wrap my head around. I was wondering if anyone might have insight they could share?

I’m using the built-in render pipeline. I have a custom shader that blends two skyboxes for smooth linear transition between highly varied skyboxes.

When this lighting transition occurs I call DynamicGI.UpdateEnvironment to update the global reflection probe. Updating the dynamic GI lighting is computationally expensive for each or every nth frame. For this reason I introduced a custom reflection probe with mask:nothing that I can update every frame while the skyboxes are transitioning.

The custom reflection probe is functional but I would best describe the output as "wrong’ because as intensity increases details get blown out and the diffuse texture is not brightened. I’m using the built-in Unity diffuse shader, and so it isn’t altogether clear why this alternative approach yields suboptimal results.

I have all other lighting (real-time, baked, etc) turned off to help show the difference in output of built-in global render probe vs a custom render probe.

Here are screenshots that really highlight the difference. The panel on the left shows the custom render probe and on the right it shows skybox environment lighting.

Unlit (Expected Behavior)

2651×1958 348 KB

Built-In Render Pipeline, Skybox, Intensity 3 w/ DynamicGI.UpdateEnvironment (Expected Behavior)

2648×1994 420 KB

Built-In Render Pipeline, Custom Render Probe, Intensity 3 (Unexpected Behavior)

2645×1947 433 KB

Material Properties

471×1002 58.8 KB

Could someone please explain what I am failing to understand, or how I might modify the custom render probe to behave more like the Unity built-in global render probe? Alternatively, what does one do when they need environment lighting to update in real-time, if the above custom render probe approach is not correct?

beats head against wall

Thank you!

2

You’re setting the intensity of different things. In the DynamicGI example you’re adjusting the brightness of ambient lighting (which controls the default ambient lighting probe), but leaving the reflection intensity in both the lighting settings (which controls the default reflection probe) and on the reflection probe at 0. Because you still have the reflection probe in the scene, the lighting settings for the default reflection probe’s intensity probably won’t do anything since you’re just seeing the reflection from the reflection probe you’ve placed.

In the second example you’re setting just the reflection probe’s intensity to 3, which means the reflection probe will be bright. But reflection probes don’t affect ambient lighting probes. They only update the reflection probe.

The answer to the question of “what do I need to do to update environment lighting in real time” is … it depends on what kind of environment you’re trying to do real time lighting adjustments to. And that the answer may be you might need both a real time reflection probe and to call DynamicGI.UpdateEnvironment().

3

Thank you, I greatly appreciate your insight. Your description of terminology, which I am confusing, helps me to understand the problem better. What I am struggling to understand is, given a dynamic or changing skybox, how does one apply ambient light to meshes which represents the current state of the skybox?

Do I switch Environment Lighting Source to single color, and then sample the texture to determine a single ambient color from the texture (and give up on trying to have ambient light spatially (spherical harmonics?) match what is in the skybox)?

If DynamicGI.UpdateEnvironment is not intended to be used each frame to update ambient lighting from a skybox, what would the alternative be?

Would I need a custom shader for all mesh game objects which applies a dynamic skybox cubemap texture?

I am just really confused.

Here is a sample scene which has a central point light source (emits orange). I am hoping to add ambient light to the scene so that the back-side of stellar objects (i.e. asteroids) are not black. The skybox blends as the player travels through space, and so this is why I am looking for a real-time mechanism to update ambient light contributed by the dynamic skybox. If I call DynamicGI.UpdateEnvironment every nth frame, it slows framerate and causes visual lighting glitches as async updates occur.

Shows scene with point light source and no ambient lighting (e.g. back side of asteroids). How to apply ambient lighting based based on the state of dynamic skybox?

2667×1350 183 KB

4

That function updates the default reflection probe and the default ambient lighting probe. The problem is updating the reflection probe is quite expensive as it has to render a cube map, which is 6 views, and then blur the mip maps which can take several milliseconds. And even worse updating the ambient lighting from said cubemap can take an additional tens to hundreds(!) of milliseconds to do.

Basically unless you’re on a recent console (that isn’t the Switch) or a high end PC, calling that every frame isn’t really plausible. Calculating the ambient lighting probe from a dynamic skybox is unfortunately an inherently costly process that there’s no easy workaround for. Especially since Unity calculate the ambient lighting on the CPU and has to copy the cube map back from the GPU.

There are a couple of options.

  1. Fake it. Use a single color ambient that you control manually. Can work if the color of your dynamic skybox is something you can easily set and there isn’t a lot of randomized color or dynamic animation systems. You can also use directional lights set to non-important to augment the ambient lighting, as those get baked into the spherical harmonic used for ambient lighting.

  2. Pre-bake it. For Falcon Age we needed dynamically changing lighting between a set of fixed skyboxes / lighting setups. So I serialized the SH for each lighting setup and dynamically blend between them manually. This is super cheap, but obviously relies on having a limited number of sky setups rather than purely dynamic.

  3. Skip it. Don’t use ambient lighting at all and only use the reflection probe. This one sounds a little insane, but you can use the smallest mipmap of a reflection probe as an ambient light of sorts. This requires using all custom shaders though, and doesn’t work great on lower end hardware that doesn’t support cube map edge filtering.

5

It is good to know what the practical limit is for workable solutions. Regarding #2, are you reading/writing UnityEngine.Rendering.SphericalHarmonicsL2 for each light probe?

6

Sort of. More specifically just overriding the global probe with an SH I construct on the fly.

RenderSettings.ambientProbe

A nice feature of Unity's SphericalHarmonicsL2 is you can lerp between two of them pretty easily.
RenderSettings.ambientProbe = probeB * (1f - blendFactor) + probeA * blendFactor;

I know other people have written stuff where they actually place ambient probes in the world and update them in real time, but that level of fidelity wasn’t needed for Falcon Age. And it’s certainly not needed for something like a space game where the “ambient” lighting is all coming from the skybox, and maybe a single nearby planet. The trick of using a non-important directional light is a great way to add lighting coming from planets btw. You can add it to the SH in c# too, but that would mean the planet itself would be affected. Instead you can use layers and culling masks so it only affects the objects you want it to.

7

Thank you for your help. I really appreciate it. I went with your idea of storing spherical harmonic values and interpolating. You can see the result, below.

tautgrandearwig

One thing I do that is likely incorrect is that I create the SH values without skybox rotation. In-game these skyboxes are randomly rotated around the y-axis. I’m not seeing any position or rotation information in the spherical harmonics or global light probe configuration. Does this mean I can’t rotate the global light probe?

Re: non-important lights. I am not sure I fully understand. If I create a directional light for a planet’s surface, wouldn’t it affect other objects in the scene? Are you thinking a spot light?

8

You’ll have to ask yourself how important that is. If the skyboxes you’re using are mostly uniform in color and brightness, like there’s none that have very different brightnesses or colors from one hemisphere to the other, then it might not be worthwhile fixing this.

Otherwise rotating a spherical harmonic is possible, but isn’t something Unity has built in support for. It’s also a destructive process; rotating an SH by anything other that 90 degree rotational steps along an major axis is a lossy operation. That is in itself not really a problem, an SH is already a very vague approximation, so it not being perfectly the same after a rotation doesn’t really matter. It wasn’t accurate before either, and the rotated version isn’t really any less accurate, just different. Basically it means each time you need to rotate the data, you’ll want to start with the originally cached SH data and rotate that rather than re-rotating already rotated data.

A google search for rotating SH data brought me to this page:
http://filmicworlds.com/blog/simple-and-fast-spherical-harmonic-rotation/
Which is … a lot of math with a lot of expectation that you already understand how spherical harmonics work.

An alternative that doesn’t require you know how spherical harmonics work would be to deconstruct your SH into a number of faux directional lights. Use the sh.Evaluate() function to get the color from the SH from the 6 major axis directions, and 8 corner directions. Then rotate those vectors and reapply them to a new SH using the sh.AddDirectionalLight() function to reconstruct a new rotated light probe. You might need to fudge the intensity of the directional lights to something other than 1.0 to get an SH of similar equivalent brightness as the original, I’ve honestly never tried to do this myself. This also isn’t going to be cheaper than the above link, just easier to implement. Realistically you’re only doing this once every so often when blending to a new skybox, so it’s shouldn’t really be a problem regardless.

No, I mean directional light. And yes it’ll affect other objects in the scene, and yes it’ll be wrong. But it doesn’t matter because no one will notice it’s wrong. This is explicitly to fake light bouncing off of a planet’s surface and affecting objects that are near it, like your ship, in the cheapest way possible. I’d give a planet a single “light color”, and then calculate the cosine for the ship’s position over the planet relative to the light direction, and use that as the ambient light coming from the planet. Maybe fudge the cosine value a little to give some wrap lighting.

As an example, in Zelda: Breath of the Wild, the entire world’s ambient lighting is determined by where Link is standing. If Link is standing on green grass, the entire world is underlit by green. No one notices that it’s wrong. The same is true here, as long as the planet isn’t the single most important source of lighting the fact it’s only “correct” for your ship and wrong for everything else likely won’t even register for most people.

Presumably you’re also lighting your scene with a directional light from the sun? That’s wrong too, it should be a point light, but no one would notice unless you have multiple objects around the sun large enough to be visible when the sun isn’t filling the entire screen. (And even then I might have two sets of objects, one set of “distant” objects lit by a sun point light w/o shadows, and the “near” objects like your ship and stuff nearby that are lit by a directional light.)

The planet light is similar in that it’s just there to add a little extra. Though you can skip it entirely, especially when you’re using skyboxes as bright as you are.

9

I just wanted to say thank you for your insights and invaluable assistance. The interpolation approach works well, and I am grateful to have your assistance!