Custom post processing using depth texture + TAA

As it turns out, when creating a post processing effect using the CustomPostProcessVolumeComponent classes and setup. Effects can be executed at the AfterOpaqueAndSky, BeforePostProcess and AfterPostProcess injection points.

The issue however, is that at these points the depth texture is being jittered by TAA, meaning any effect using the depth texture isn’t viable. Screen capture - 6c307935bde39daa35cbad0ea0e8c617 - Gyazo

The only workaround I found, was to implement post processing as a Custom Pass instead. At which point, the depth texture is unjittered. Though custom passes do not support volume blending, as they do with regular custom postprocessing volumes.

My conclusion is either create post processing that doesn’t support TAA, or doesn’t support volume blending, which isn’t ideal.

I have tried to create a CustomPass, which simply copies the depth texture to another RT, then use that in post processing. However it is also jittered.

Anyone had any luck with circumventing this?

Hello,

The depth buffer is indeed jittered from the beginning of the frame, here’s what it looks like in the graph of the frame:

This basically means that you can’t render anything that needs depth test after the post processes.

Depending on your use case you might be able to find a way around using a mix of custom pass and custom post-process. What I did once to overcome this issue was to render my objects with a custom pass in before post-process into the custom color buffer but with the camera depth bound so I have the correct depth test. Then I made a composite path that takes the custom color buffer and just blends it on top of the camera color.

It worked for my use case but it’s not a universal solution, would you mind sharing more details about what you want to achieve?

Hey, that image from docs? :slight_smile:

Thank you for your reply, and clarification. I somehow missed that graph in the documentation, but illustrating where the jittering occurs is very helpful!

Some content: I’m in the process of porting the the effects from this package. Some of the effects use depth, as a means of lerping values over a distance. The Edge Detection effect derives geometry normals from the depth texture, where as another reconstructs the world-space positions from depth.

So far I’ve had success creating the effects that use depth as a Custom Pass, executing Before Post Process. Where I’m fading some values like opacity (if applicable) using CustomPass.fadeValue, to still support volume blending of a kind. Though, I don’t quite understand why executing a Custom Pass Before Post Process is unjittered, but a Custom Postprocess at Before Post Process is jittered.

Either way, it seems like I’ll have to mix both systems like you suggested, based on if an effect uses the depth texture or not. The only issue this poses for me is having to find a clear way of communicating to the end users which effect requires what kind of setup to render.

Would there be a downside to creating all effects as Custom Passes? This would make set up more consistent

As you can see in the code here: https://github.com/Unity-Technologies/Graphics/blob/master/com.unity.render-pipelines.high-definition/Runtime/PostProcessing/PostProcessSystem.cs#L496 we do TAA just before executing the custom post-processes Before Post Process list but the custom pass is done before the post-process step (so before TAA).

If you’re fine with the workflow they offer then there is not really any downside to custom passes over post processes.
The volume workflow is a bit different, there is no profile asset on the disk, there is no “stack” support, which means that if you have two custom pass volumes overlapping, the effect list in both volumes will be executed. But on the other hand, you have a lot more control over the rendering of objects in custom passes.

Do you think adding a custom post-process injection point before TAA will help solve your issues? If yes we can probably add it (need to check few things before but should be good).

If making this change doesn’t conflict with the grand scheme of things, it would be helpful!

For a personal project, I wouldn’t necessarily mind a mix between custom post processing and custom passes for effects, but for an asset store package, setup should be more straightforward and less fragmented. The URP also uses the volume system for post processing, so this is also more familiar to users (more so since it’s similar to the PPSv2). So I’d say it’s more of a UX point.

Hey, a quick update about before TAA post processes :slight_smile:
The PR adding the injection point has been merged: Add custom post process before TAA injection point by alelievr · Pull Request #817 · Unity-Technologies/Graphics · GitHub
It should be available in package 10.x.x

1 Like