DLSS + Custom Pass with shader using HDSceneColor - is this scenario supported?

[mgeorgedeveloper1]

Hello all,

We use a few different custom passes in our game.

One is a custom pass to draw all the glass windows. It reads HDSceneColor to create a blurred effect on the glass surface (it is in a custom pass so that it can also blur the transparent water previously drawn - long story that I don’t want to get into here).

Another is our UI, which is in camera space, so that we can use a custom shader that again draws a blurred background using HDSceneColor.

So far all is good and well…

Now let’s enable DLSS in the asset and on the camera. Just using some basic settings, like a forced 50% render size:

Now this is where things start to go badly wrong.

1. Any shader in a custom pass that samples from HDSceneColor, now reads bogus information for 75% of the screen. It’s like the HDSceneColor buffer being read from is at the 50% scale, but the current shader is seemingly operating at full resolution.

I can’t upload more images - but basically… same issue in the UI.

2. Furthermore - the UI “Tiles” shown before, now also jitter like crazy:

DLSS jitter and bad sampling from HDSceneColor node.

So after a long and tedious wall of text, I guess my first question is… can DLSS play well alongside custom passes using HDSceneColor (shader graph node)?

Secondly, can screen space UI be used at all without causing this crazy jitter?

[PutridEx]

I believe I’ve seen this somewhere in the issue tracker, as a bug.
But I’m not sure, it’s been a while and it might’ve been a different issue so I can only hope so as it sounds like a critical issue.

Hopefully someone with more info can help out.

[m0nsky]

Which version of Unity are you using?
If you’re using _ScreenSize anywhere in your shaders, try swapping them for _PostProcessScreenSize

[mgeorgedeveloper1]

@m0nsky

Which version of Unity are you using?
If you’re using _ScreenSize anywhere in your shaders, try swapping them for _PostProcessScreenSize

Unity 2022.2

It’s nothing as fancy as hand-written shaders or access to parameters such a “_ScreenSize”.

Here’s what we have… a pretty simple custom pass with Before Post Process injection point:

And then a Shader Graph that simply reads from HD Scene Color using screen co-ords. The screen co-ordinates are the issue, since they seem to be in full res, but the HD scene color buffer seems to be at 50% res?

[mgeorgedeveloper1]

And I have some kind of breakthrough, which is a hack/work-around that does not make a lot of logical sense.

If I multiply the UV to match the DLSS percentage, then the HD Scene Color node reads correctly:

But the reason this is a little baffling, is because:

• DLSS injection is set to AFTER POST PROCESS - this is when the final upscale should be happening in my understanding.
• Custom pass injection point is set to BEFORE POST PROCESS, so all buffers should be in the 50% dimensions.

So I guess “Screen Pos” is just always seen as full res… so how would one read from HD Scene Color in a manner that is agnostic to the DLSS percentage?

NOTE - even reading HD Scene Color using default UV (no input), the issue is the same. So it seems the HD Scene Color node does not have a correct understanding of DLSS scaling.

---

Anyway, this still leaves the insane Jittering shown in my original post in the video, which I will dig into separately to try to find a solution.

[larsbertram1]

if you look into the lit code for refraction you will find something like this:

float2 samplingUV = samplingPositionNDC * _RTHandleScaleHistory.xy;

when using a blurred sample you have to be even more carefully as here you will sample from lower mip levels of the color pyramid which will contain white spill at the upper and left edge:

float ceilRefractionBlur = ceil( abs(MipLevel) );
float2 maxCoord = _RTHandleScaleHistory.xy - pow(2.0, 2.0 + ceilRefractionBlur) * _ScreenSize.zw;
samplingUV.xy = min(samplingUV.xy, maxCoord);

[reddotgames]

@mgeorgedeveloper_1 How did you managed to fix this?

[mgeorgedeveloper1]

@reddotgames

@mgeorgedeveloper1 How did you managed to fix this?

I think in v2022.3.8, the use of HD Scene Color seems to work OK in a custom pass. I will have to check again to confirm this, because I only recently switched back to a custom pass for our glass, and I didn’t notice an issue.

But I also use FSR2, and this still has the issue for sure.

So the only way I know to solve it, is to manually adjust the HD Scene Color input UV by the scaling factor that is currently active. This is on my to-do list, fixing the FSR2 incorrect sampling of scene color, so I will report back here as soon as I can.

[mgeorgedeveloper1]

In the latest LTS 2022.3.18, no hacky adjustments are required to correctly read the HDSceneColor node for either DLSS or FSR3 (store package).