Confusion about Gamma vs. Linear

Ewanuk · September 18, 2017, 4:30pm

I’m confused about what effect the Gamma/Linear Color Space option in the Player Settings is actually doing in the graphics pipeline.

If it’s set to Gamma, everything is done in gamma-space in the shaders, nothing gets done in linear space. Textures are read in “as is”, lighting is done, and it’s pushed out to the display device, still in it’s gamma color space, which is what the display device expects. Cool.

If it’s set to Linear, all textures are read in (for example, all tex2D() sample calls) with gamma correction applied so we can work w/ the texture data in linear space. Lighting is done in linear space, and when the Renderer is finished, it maps the final image back to gamma space for the display device.

So, in photoshop, I create this 0-255 gradient, saving it as a PNG:

Due to how images work, this is actually stored in gamma color space. So a color value at the u = 0.25 column should be closer to 50% than 25%, do I have that right?

Now, I import it into unity, which is set to use the linear color space. I have the sRGB flag set to “true”:

If this flag is true, I expect the data to be coming in gamma-corrected. If it’s false, I expect the data to be coming in “as is”.

To verify this, I’ve written a shader that renders the red-channel of a texture as a heatmap:

fixed4 frag (v2f i) : SV_Target
{
    fixed4 col = tex2D(_MainTex, i.uv);

    if (col.r < 0.25)
    {
        return fixed4(1, 0, 0, 1);
    }
    else if (col.r < 0.5)
    {
        return fixed4(1, 0.5, 0, 1);
    }
    else if (col.r < 0.75)
    {
        return fixed4(1, 1, 0, 1);
    }
    else
    {
        return fixed4(0, 1, 0, 1);
    }
}

Now, I’m currently in Linear Space. Ideally, if the flag is true (unity should decode the image data back into linear space), I expected to see something like this:

However, I’m actually seeing this:

When I turn the sRGB (Color Texture) flag to false, it looks “correct” like the first heatmap. It works opposite to what I expected.

Furthermore, when I have the color-space set to gamma, it looks like the first image, Shouldn’t it look like the second image because it’s reading the texture data “as is”? Not applying gamma correction when it reads the texture data?

What am I misunderstanding about how the linear vs. gamma color space works with respect to the Unity Pipeline?

Thanks

bgolus · September 18, 2017, 8:07pm

PNG saved out of Photoshop will indeed be stored in gamma space, but really that’s more about telling tools how to read & display the data. A value of 127/255 in the image file could be gamma, or could be linear.

That gradient you have is 0/255 on the left, 127/255 in the middle, and 255/255 on the right. Sampled in gamma (sRGB) space that 127/255 is indeed going to be a linear value of ~0.2, but if sampled in linear space it’s ~0.5. It’s a quirk of the human visual system and computer displays that a value of “~0.2” appears half as bright as “1.0” But you’re not using this gradient to display it as a gradient, you’re using this gradient to be texture UVs.

The short version is you’re using that gradient as data, not color. Data should almost always be linear because you want 127/255 to be 0.5. Color should be sRGB because you want 127/255 to be perceived as half as bright as 255/255.

The difference between linear and gamma space render is in gamma space all textures are sampled as if they’re linear because the frame buffer itself is in gamma space. A value of 127/255 in the frame buffer will be displayed as ~0.2, so the textures don’t need to be sampled in gamma space. The sRGB flag on textures doesn’t actually do anything when using gamma space rendering as all textures are sampled linearly. The conversion between gamma and linear space only needs to happen when they’re different.

Ewanuk · September 18, 2017, 9:43pm

So if I’m understanding correctly, in this case, the image data is actually going in a straight line from 0 - 255 (Red Line). I had expected to be the Blue Curve. So, the idea that “All images are stored as gamma” isn’t quite correct. Like you said, it’s about how to interpret the data in the image file.

3224919--247347--Reference Curves.PNG

Now, working with this texture in the Linear color space, since I had the checkbox ticked, it’s applying the gamma correction to it (not reading it “as-is”), resulting the Green Curve, which explains the heatmap results.

You mean this statement is true when you’re working in Gamma Space correct? What does 127/255 in the frame buffer get displayed as when working in Linear Space? I noticed some subtle color differences in the heatmap displayed when I switched between the two.

Thanks for the help!

bgolus · September 18, 2017, 10:29pm

Here’s the basic conversion path for different image and rendering color spaces.

sRGB texture, gamma space rendering

sRGB texture value = 127/255 (sampled with no conversion)
shader value = 0.498 (shader outputs directly to frame buffer)
sRGB frame buffer value = 127/255 (render to screen with no conversion)
on screen value = 127/255 (monitor display curve)
“0.2” measured brightness relative to full brightness, appears half as bright to human eye as “1.0”

linear texture, gamma space rendering

linear texture value = 127/255 (sampled with no conversion)
shader value = 0.498 (shader outputs directly to frame buffer)
sRGB frame buffer value = 127/255 (render to screen with no conversion)
on screen value = 127/255 (monitor display curve)
“0.2” measured brightness relative to full brightness, appears half as bright to human eye as “1.0”

sRGB texture, linear space rendering

sRGB texture = 127/255 (sampled with sRGB to linear conversion)
shader value = 0.2 (shader outputs directly to frame buffer)
linear frame buffer = 0.2 (render to screen with linear to sRGB conversion)
on screen = 127/255 (monitor display curve)
“0.2” measured brightness relative to full brightness, appears half as bright to human eye as “1.0”

linear texture, linear space rendering

linear texture = 127/255 (sampled with no conversion)
shader value = 0.5 (shader outputs directly to frame buffer)
linear frame buffer = 0.5 (render to screen with linear to sRGB conversion)
on screen = 188/255 (monitor display curve)
“0.5” measured brightness relative to full brightness, appears ~3/4th as bright to human eye as “1.0”

Ewanuk · September 19, 2017, 3:30pm

Okay, that makes a lot more sense. Thanks again.

A big confusion point was that I thought working in linear-space meant colors would be intuitive, {1.0, 0.5, 0.0} in linear space should be “orange”, but it’s actually perceived as something like {1.0, ~0.8, 0.0}. Whereas {1.0, 0.5, 0.0} in gamma space actually looks “correct”.

In other words, smooth gradient data in gamma-space would plot as a straight line (the Red Line). But in linear-space would plot as a steep curve (Green Line). In linear color space mode, we’re working with this “less-intuitive” but accurate color space. Then when we’re done rendering, it gets converted back to gamma-space data, which is what our display device expects.

TL;DR, Linear-Space {0.5, 0.5, 0.5} should look ~75% white, Gamma-Space {0.5, 0.5, 0.5} should look middle-grey

Voxel-Busters · March 17, 2018, 10:16am

Can you please explain more about this line? I might not have got what exactly you mean.

bgolus · March 18, 2018, 9:16pm

The human eye does not perceive light linearly.

If you have two lights where one outputs 50% as much light energy as the other, humans will perceive it as ~72% as bright as the brighter light.

To have it be perceived as 50% as bright it needs to output ~20% as much light energy.