Ray-tracing effects better denoise solution

Unity Engine
, , , , , , , , , ,
1

I’ve tried every DXR effect, and they’re all (maybe except RTSSR) suffer from noise or/and ghosting problems:

-Ambient occlusion requires ridiculous amount of samples to look somewhat decent in motion, like 16 or even 32 to eliminate the noise. Performance hit is massive then, which restricts real-time usage.
-Ray-traced shadow’s denoiser creates ‘lag’ between caster and the shadow, leaving their unmatched. Additionally, if the radius is higher than 1, it generates an ugly outline between them. Disabling denoiser and cranking up sample count to the max leads to horrible performance and grainy shadowing, especially with foliage.
-RTGI generates ghosting and noise on moving objects, which are too noticeable to ignore. Changing level to ‘quality’ just swaps the noise pattern and dumps down performance even further.

I’ve tested every possible denoiser setting, radius, motion rejection on and off, different TAA settings, DLAA, resolutions. Nothing really removes all these issues, or the performance cost is too high (Framegen would’ve been a help here, but Unity’s still lacking it for some reason). So the question is: what steps are taken if any to improve denoising for DXR in HDRP?

2

GI example:

3

Check this asset: HTrace: Ambient Occlusion HDRP | Fullscreen & Camera Effects | Unity Asset Store
It has the same ray-tracing under the hood but offers much better denoising and performance scalability

4

Thank you, I’m aware of this asset. As of now it covers only GI and maybe AO, and has its own set of limitations, so Unity’s DXR implementation is still relevant.

5

Maybe you confused it with HTrace GI?
What I linked is HTrace AO it’s a different asset and it doesn’t provide any GI but instead builds on top of Unity’s RTAO. I don’t know about any major limitations, it does everything Unity’s AO does I think and maybe more…

6

Pretty much spot on

GI - creates super ghosting and sometimes feels like it is SSGI. Try to change the missed ray to probe volume, and mix it with APV. Not ideal for RT, but seems like that is what they sort of had in mind - looking at the latest implementation

AO - don’t use RT AO, for some reason, it is super heavy, and if not set to high, any denoising, pretty much just messes up everything. If you go low and set radius low enough, then there is just super noise everywhere.

Shadows - And this is really stupid, but there is a obvious bug

Reflection - It is just not usable…breaking and ghosting all the time. It never really resolves, constantly waving all the time, even if it is set to high

Lastly, the lack of RR makes all Raytraced effects to be so minimal, and if you pump up the indirect value, it goes crazy with noise. So if all optimizations that Unity put in is employed, it barely effects the screen.

I think they want you to use APV and just slightly use Raytracing near camera. The stuff really doesn’t work well without it. Although the two doesn’t really blend well. Especially for interior or dynamically lit areas, like doorways.

So my latest workflow, is to bake APV, and gently let Raytracing touch up the areas near camera. Theoretically good, but in practice, it is horrible. A lot of elbow grease, it lacks dynamic adaptation, the same high performance hit. But most importantly, it looks very bad. Even with everything set to high. The frame inconsistancy gets really bad, etc.

And ofc, physcially incorrect. There is just too little GI indirect contributing, AO denoising is messy, shadows don’t resolve in time, and at low settings, the edge filters look horrible.

To be fair, they are all usable, as long as you have a RTX4090 or above at 1440p. But, only 1 effect at a time. GI, is mostly my choice, but with APV.

But then, H-trace often does a better job with APV so…I don’t know. I’d give up on reflections, that one is a mess. Both SSR and RTR. Probably the worst implementation of RT reflections. Just use Reflection probes…XD

7

Just popping here to add a little bit of context.

We have to stop thinking about raytracing solutions as a silver bullets. As you have probably seen, as any other features, they have limitations: ghosting, noise, cost…Everything is a trade off between those 3.

We don’t WANT you to use APV, we want users to understand that raytracing NEEDS fallbacks most of the time. (especially for reflections or GI). Once you exhausted your bounces and/or ray length, you need something to fallback on, otherwise, you will get black or worse (wrong information like fallbacking on sky inside). This is why you need to have reflections probes or APV even when you are using raytracing. (here’s some informations in the documentation)

As an example, it’s the same with SSR, without reflections probes, there WILL be missing information on screen. So one does not really go without the other in most cases.

Now that we have context, the denoiser usually needs some history to be able to denoise properly, in the case of your video, the global illumination override probably have Receiver Motion Rejection checked, meaning that the movement of your characters triggers the rejection of the history, making the denoiser job near impossible, thus the noise you see. If you uncheck it, you probably have less noise but you will be more prone to ghosting especially to faster you go. It’s all a trade off.

I’ll leave you with the DXR triangle of pain

image
image600×492 43.6 KB

8

Sample scene is packed up with all sorts of fallbacks: APV, dozens of reflection probes, even lightmaps. Doesn’t make much of a difference in terms of reducing noise artifacts from moving objects as you can see. Ghosting maybe, but that’s a lesser RTGI problem imo.

It doesn’t, otherwise there would be shimmering on the character while standing still at the first seconds. Don’t take my word for it, just load HDRP sample project, pick the highest quality preset (5HDRPRaytracingRTGIQuality), take a couple of back and fourth runs and see for yourself.

Now, I have a feeling after reading your message that you’re quite happy of the DXR overall performance here at Unity. Like if something is not looking good it’s rather a skill issue of dumb users can’t handle their fallbacks properly, or if not, see the image from 2021 which explains everything. Truth is, we’re having Ray Reconstruction for a couple years since then, and it recently was upgraded along with DLSS 4. Needless to say Unity has neither. Hell, even third party publishers like HTrace have better denoising options (I mean, look at this), while not even being able to access the engine proper way. Meantime, denoising of HDRP’s RT effects is still somewhere in that 2021, and there’s no signs it is a subject of change as I understand.

9

I’m not saying it’s perfect, I’m not even saying I’m satisfied personally, I’m just trying to give users our tips and tricks to get the best of what we currently have and what we are stuck with given that there’s little to no update planned on it.