The Future of Graphics Customization: Current Issues and Expectations

Graphics Render Pipelines
, , , ,
1

This post is written from the perspective of both a long-time user and an asset developer. When I use “we” or “us,” I am speaking on behalf of our team. The post serves as both feedback on our current experience and a question about the future of pipeline customization.

Over the past two years, we have developed a number of visual effects for Unity, including AO (for all pipelines), SSGI, and WSGI (for HDRP). Throughout this process, we have faced several challenges, particularly in integrating our effects into Unity’s rendering pipelines. Below are the most significant issues:

1) Injection Points for Custom Passes / Effects in HDRP

Unfortunately, there isn’t a single injection point that conveniently works for any custom screen-space / lighting effect (AO, GI, SSR, SSGI, RTGI, Shadows, etc.). The information computed by these effects must be available before the deferred lighting pass runs, which leaves us with only two injection points: BeforeRendering and AfterDepthAndNormal. The first one is not viable, as most required data is not ready at this stage. AfterDepthAndNormal, however, has the following issues:

  • Depth isn’t fully prepared. We must ask users to force the Depth Prepass as a workaround.
  • Camera Motion Vectors are unavailable, requiring us to render them manually and merge them with Per-Object Motion Vectors.
  • The Depth Pyramid has not yet been updated, so we must generate it ourselves.
  • Light Tile / List information has not yet been updated, preventing proper use of reflection probes, contact shadows, etc.

However, all of the above is available just a few passes later when HDRP itself computes screen-space effects. Why not add another injection point at that stage and make this data accessible to developers as well? We need a reliable injection point where all fundamental data (Depth, Normals, Motion Vectors, Albedo if possible) is ready for any complex (e.g. requiring a denoiser) effect.

2) Replacing Native Screen-Space / Lighting Effects

There is no official or straightforward way to override native effects such as AO, SSR, SSGI, or others. Each effect has a dedicated output buffer where its final result is stored before being used in deferred/forward shaders and the rest of the pipeline. However:

  • These buffers are not public, preventing direct access.
  • They are not created if the corresponding effect is disabled.
  • Unity aggressively strips related shader code and data (e.g., AO in URP) when the native effect is disabled. Even if we manage to inject a custom AO, it may be discarded at compile time unless the native AO is enabled too.

We had to implement complex workarounds to make this work. As developers, we need a clean and efficient way to replace Unity’s effects. Given that each effect (e.g., SSGI) is an isolated module that simply passes its final result to the pipeline, it should be feasible to make these final output buffers publicly accessible for writing and prevent them from being stripped when needed. We’re not suggesting they always remain active - just that there be a mechanism to signal that we require them for our custom effects.

3) Incompatibility Between Graphics Features

Certain graphics features that we need do not work together. A prime example is Override Shaders:

  • SRP Batcher was introduced.
  • Override Shaders were introduced but are incompatible with SRP Batcher.
  • BRG / GPU Resident Drawer was introduced but is incompatible with Override Shaders.

This has resulted in a chain of features that cannot be used together, making it difficult to fully utilize Override Shaders.

Will these issues be addressed in the future, especially in light of the Unified Rendering coming?

If it helps, here’s how this could work in an ideal scenario (from our point of view):

  • Additional injection points are introduced, specifically considering screen-space and lighting effects.
  • More data is available at these points, or we have the ability to request it to be prepared by the pipeline.
  • Final output buffers for screen-space / lighting effects are exposed for writing and can be requested (and created) even if the corresponding native effects are disabled.
  • A mechanism allows us to indicate that a given feature should not be stripped from shaders, even if the native effect responsible for it is disabled, enabling us to pass custom results.
  • Major features are made compatible with each other (e.g., Override Shaders with SRP Batcher / BRG).

Thank you!

2

Really hoping they see this one!

3

Hopefully we get some progress on these issues. I am following your development of H-Trace.

4

This post highlights a lot of struggles I’ve also had with implementing more elaborate post processing effects.

I’ve had to majorly pull apart URP in many cases, force the engine to have early / late opaque rendering passes etc to handle, for example, excluding certain objects from Ambient Occlusion. The render graph feels like a step in the right direction, but a lot more is needed. Please listen to this post and consider the presented ideas.

5

Hi :slight_smile:

Thank you for your detailed feedback and for your contributions to the Unity ecosystem! We really appreciate the high-quality assets you’ve developed, and the insights you’ve shared here will help us refine our rendering pipelines.

As part of our long-term graphics strategy, we are committed to improving both URP and HDRP iteratively, ensuring they remain reliable solutions for studios and developers while moving toward a more unified rendering approach in the long term.

1) Injection Points for Custom Passes / Effects in HDRP

We are internally looking into the issues you mentioned with AfterDepthAndNormal. Some of the missing data may be a bug rather than a design limitation, and we will follow up in this thread once we have more information.

Expanding extension points in HDRP and refining API accessibility are areas we are actively evaluating as part of our broader effort to bring URP and HDRP closer together in terms of extensibility. However, we also need to balance flexibility with long-term architectural adaptability.

We know that adding more injection points is important, and we are exploring ways to introduce them without restricting future pipeline upgrades.

HDRP follows a coherent lighting philosophy, where effects are designed to interact predictably (e.g., fog applies to all surfaces, SSAO affects only diffuse lighting, and SSGI integrates with all emissive surfaces). Because of this deep integration, replacing individual effects can be challenging, as they often rely on non-public interfaces that are not currently designed for external modifications. However, we are investigating ways to improve modularity and extensibility without breaking HDRP’s internal coherence.

2) Replacing Native Screen-Space / Lighting Effects

In the short term, we are focusing on improving extensibility, including modularizing URP post-processing to rely only on public APIs and unifying key components such as the render graph compiler, lighting, and post-processing data.

We are also specifically investigating the URP AO stripping issue, as this concern has been raised multiple times.

This modularization should provide exactly what you need, as our internal effects will rely only on public APIs, making it significantly easier to override and exchange built-in effects. Our initial focus is on URP, and we will later explore how to unify extension points in HDRP in a way that aligns with our long-term rendering strategy.

3) Incompatibility Between Graphics Features

We recognize that certain features currently have compatibility limitations, particularly with BRG and GPU Resident Drawer. While some of these limitations are inherent to how these systems work, we are actively exploring ways to improve compatibility where feasible.

Could you clarify what you mean by Override Shaders? Understanding your specific use case would help us better assess where compatibility improvements are most needed and how we can better support your requirements moving forward.

4) Future of Unified Rendering and Extensibility

As we move toward a Unified Rendering Pipeline, ensuring flexibility for customization remains a key priority. While we can’t share specific details yet, we are designing this system with extensibility in mind and actively incorporating feedback like yours.

Our goal is to iteratively bring URP and HDRP closer together by improving their setup, authoring, extensibility, and execution frameworks while continuing to refine them individually. This approach ensures that developers can rely on both render pipelines for years to come, while also benefiting from a more cohesive and extensible rendering ecosystem over time.

Thanks again for your input! We encourage you to continue sharing your insights, as they help shape the future of Unity’s rendering technology.

6

Hi!
Thank you for this very informative response!

And thank you for your kind words! :blush:

In that case, I’ll provide a bit more detail about these issues - hopefully, you’ll find it useful.

Depth Buffers

There are two depth buffers available at the AfterDepthAndNormal point. The first one is already bound under the _CameraDepthTexture name. As far as we understand, all mips of this buffer, except for Mip0, are from the previous frame. Mip0 may also be incomplete, which can be fixed using the “Depth Prepass within Deferred” option or by enabling any feature (e.g., Decals) that forces this Prepass.

The second available depth buffer is CustomPassContext.cameraDepthBuffer, which we can pass to shaders ourselves. However, it always seems to contain data from the previous frame, making it unusable for our purposes. Our current solution is to use _CameraDepthTexture + Depth Prepass.

Camera Motion Vectors

This one is weird because the AfterDepthAndNormal point sits right between the rendering of per-object motion vectors and camera motion vectors. We fix this by rendering camera motion vectors ourselves, which wouldn’t be necessary in an ideal scenario.

Depth Pyramid

This is the least problematic issue. Since we may need to generate a pyramid with a different filter (e.g., Min instead of Max) or lay it out differently (e.g., using Mip levels instead of an atlas), its absence isn’t a major concern. We can manage without it.

Light Tile / List Data

We haven’t run into major problems yet because we haven’t needed it so far (luckily), but we expect to in the near future. A couple of examples where this might be necessary: reflection probe fallback for SSGI / SSR (especially with a scalarized loop similar to the one used in the native SSGI) and Contact Shadows. Unfortunately, there’s not much we can do about this. Updating everything manually on our end doesn’t seem feasible, so this one may become a real issue.

From our perspective, most of these issues could be resolved by moving the AfterDepthAndNormal point slightly further in the pipeline - after Camera Motion Vectors but before Screen-Space Effects.

That’s why we strive to inject our effects directly into the pipeline (by overwriting native buffers) rather than simply overlaying them on top of the output.

Yes, this is understandable. I just want to clarify that we are already successfully replacing effects - it just involves suboptimal workarounds. So, we’re not necessarily asking for a new way to do this, but rather to see if some of the existing obstacles we’ve encountered could be lifted to make the process cleaner and more straightforward.

Here’s an example:

HDRP’s SSGI fills a buffer called CameraSSSDiffuseLighting, which is bound to SSGI shaders as _IndirectDiffuseTextureRW. This buffer is then used throughout the pipeline (e.g., it is passed as _IndirectDiffuseTexture to the deferred shader). This setup is very convenient because by simply overwriting the data in this one buffer, we can propagate our custom GI to the rest of the pipeline.

However, there are a few issues:

  1. This buffer isn’t created if native SSGI is disabled.
  2. We can’t bind our own buffer as _IndirectDiffuseTexture because the native SSGI either overwrites it or, if SSGI is disabled, it gets replaced later in the pipeline by a 1×1 dummy texture.
  3. Even if we could bind our buffer, we can’t be sure HDRP will actually use its content when native SSGI is disabled.

To overcome these limitations, we use a rather complex workaround involving reflection, swapping pipeline resources, keeping native SSGI active but modified to stay in a “dormant” state, and managing keywords on our side. But in the end, all of this effort boils down to just overwriting the CameraSSSDiffuseLighting buffer - which we successfully do. From there, everything else works perfectly, we just wish it was easier and less cumbersome.

One more concern is that so many steps involved in the workaround make the whole system rather fragile. If one day a major HDRP update occurs in one of the components we rely on, it may break this system.

Sounds awesome!

I meant this feature, which was introduced in HDRP 14. I believe I’ve seen it referred to as both Replacement Shaders and Override Shaders in some places.

We use it for voxelization. Since we need surface properties (albedo, emissive) for GI, we don’t see any viable alternatives. For example, rendering with Material Override, which is commonly suggested, doesn’t work for this case. Shader Replacement / Override is the only way to maintain the link between meshes and their material properties, making it crucial for us.

As I mentioned earlier, Override Shaders don’t support the SRP Batcher, which is unfortunate and slows down our voxelization to a degree. They also aren’t compatible with BRG / Resident Drawer which requires waaaay less than ideal workarounds which have actually led some of our users to abandon BRG / Resident Drawer in favor of continuing with WSGI.

What’s more, the warning “BatchRendererGroups currently don’t support Override Shaders” is constantly triggered - even when there’s no overlap between objects rendered with Override Shaders and objects rendered with BRG. This isn’t a dealbreaker, but it’s quite inconvenient. So it would be really awesome if Override Shaders received some upgrades and could leverage SRP Batcher / Resident Drawer.

7

At the AfterOpaqueDepthAndNormal injection point, this _CameraDepthTexture is not bound so I’d say that what you’re seeing is the value from the last SetGlobalTexture which is the last camera rendering (but it could be any camera, not the same one). It’s not safe to rely on this buffer to read the last frame depth information.

CustomPassContext.cameraDepthBuffer should contain the full depth buffer of the current frame because this injection point is after both the depth pre-pass and the G-Buffer. This injection point is used to modify the depth buffer before the depth pyramid generation, so you only have access to the mip 0.

This injection was designed to write and modify object motion vector data (rendering custom objects in the scene) and not consume MV data so I see what kind of issue you’re facing. Unfortunately, right now we don’t have an injection point that can allow you to access this information as well as replace built-in SSAO, SSR, SSGI, SSSSS, etc.

I’m in the process of adding a new injection point for After Opaque rendering allowing you to write into the SSS buffer if you’re willing to use reflection to access the buffer. This is an option we could take to make it less painful to overwrite HDRP effects without exposing a new API.

We can’t move it but we could add a new one (without public API related to lighting buffers of course).

I think this could also come from the RenderGraph API being exposed publicly for custom passes. It’s something we plan to have to be able to explicitly read/write textures and keep them alive automatically if you use them (even if they stay internal and you access them by reflection).

8

Thank you!

I just double-checked and found that it may indeed contain previous frame data, but only under certain forced conditions. The only way I discovered to completely corrupt this buffer is by disabling decals (which are enabled by default) in the project or camera settings. I suspect there are other ways to trigger this issue, but with default settings and Depth Prepass active, this buffer seems to always contain valid data.

Yes, I checked it again and found that it works, which contradicts my previous test where this buffer, on the contrary, did not contain current frame data. Moreover, one of our users independently ran the same test and encountered the same issues with this buffer. This leads me to suspect that there is either a bug or a specific set of conditions under which this buffer can become corrupted too.

Yep, I think this could work for our case.

Yes, that would be great :slightly_smiling_face: