XR Foveated Rendering does not work with HDRP

Unity Engine
, , ,
1

Tested with Pico 4 through Virtual Desktop, using CharlesWard’s instructions + script from here: OpenXR foveated rendering in Unity 6 - #2 by CharlesWard.

My OpenXR runtime is VDXR.

I had to make a small modification to execute IsVrRunning() when triggering the action, else the xrDisplay would initially be null.

I’ve seen conflicting information about the state of foveated rendering in HDRP. For example:

This page suggests it’s not yet implemented:

This page suggests it is implemented:

However, it doesn’t work for me, under DX11 or DX12. On this note, even if DX11 would be Nvidia only, I would still really like that feature as DX12 and Vulkan have some rendering issues in my project.

Foveated rendering is a hugely important performance feature so I hope this can be fixed.

2

We are also having issues working with Unity 6 HDRP & PSVR 2.
Unity said to us via support that foveated rendering in HDRP for PSVR2 is still being worked on, and there is no estimate when it’s ready

This is a major setback to us, I hope it will be worked on soon

3

Hi, I’ve tried again with PCVR foveated rendering to no success.

I’ve tried a fresh URP project (Note: I need HDRP foveation), and still can’t get it to work. My HDRP project is on 6000.0.54 DX12. The URP test was 6000.3.2 and is DX12 by default.

When enabling SRP Foveation with the “FoveationStarter” script I took from the Unity docs (and following the other brief setup instructions, URP): Foveated rendering in OpenXR | OpenXR Plugin | 1.17.0-pre.1.

The following log is spammed:

Use of GfxDeviceD3D12::DispatchComputeProgram is forbidden during a render pass!

The Editor log also contains this:

Features requested to be enabled: (1)
[XR] [24964] [05:06:31.951][Info   ]   Foveated Rendering: Version=1, Company="Unity", Extensions="XR_UNITY_foveation XR_FB_foveation XR_FB_foveation_configuration XR_FB_swapchain_update_state XR_FB_foveation_vulkan XR_META_foveation_eye_tracked XR_META_vulkan_swapchain_create_info"
[XR] [24964] [05:06:31.951][Info   ] 
[XR] [24964] [05:06:31.951][Info   ] Requested feature extensions not supported by runtime: (6)
[XR] [24964] [05:06:31.951][Info   ]   XR_FB_foveation: Feature="Foveated Rendering": Version=1, Company="Unity"
[XR] [24964] [05:06:31.951][Info   ]   XR_FB_foveation_configuration: Feature="Foveated Rendering": Version=1, Company="Unity"
[XR] [24964] [05:06:31.951][Info   ]   XR_FB_swapchain_update_state: Feature="Foveated Rendering": Version=1, Company="Unity"
[XR] [24964] [05:06:31.951][Info   ]   XR_FB_foveation_vulkan: Feature="Foveated Rendering": Version=1, Company="Unity"
[XR] [24964] [05:06:31.951][Info   ]   XR_META_foveation_eye_tracked: Feature="Foveated Rendering": Version=1, Company="Unity"
[XR] [24964] [05:06:31.951][Info   ]   XR_META_vulkan_swapchain_create_info: Feature="Foveated Rendering": Version=1, Company="Unity"
[XR] [24964] [05:06:31.951][Info   ] 
[XR] [24964] [05:06:31.951][Info   ] System Properties: Name="Oculus Quest2", VendorId=-11714
[XR] [24964] [05:06:31.951][Info   ] System Graphics Properties: MaxWidth=16384, MaxHeight=16384, MaxLayers=16
[XR] [24964] [05:06:31.951][Info   ] System Tracking Properties: OrientationTracking=True, PositionTracking=True
[XR] [24964] [05:06:31.951][Info   ] 
[XR] [24964] [05:06:31.951][Info   ] ==== OpenXR API Spec Version ====
[XR] [24964] [05:06:31.951][Info   ] 
[XR] [24964] [05:06:31.951][Info   ] XrInstance Created with OpenXR API Version: 1.0.34

With strength 1 or 0, there is no difference in VR.

There is mixed messaging from Unity as to whether foveated rendering is supported in HDRP; and following the documentation, I can’t get it working in either URP or HDRP (though HDRP support is what I need).

CharlesWard stated in June 2024 that foveated rendering works in PCVR: OpenXR foveated rendering in Unity 6 - #4 by CharlesWard.

Attempting to tag @CharlesWard as he may be the only person who knows how this works. It would be super helpful if you could provide instructions or a working sample project (for HDRP please). Thanks!

EDIT: Just noticed that Charles Ward’s profile says he hasn’t posted or been seen since June 2024… in that case, I really hope someone else at Unity can explain this.

EDIT 2: Opened bug report IN-130668.

4

I think this is why it doesn’t work: Unity Issue Tracker - [DX12][XR] Foveated rendering does not work if Shadows and Post Processing are enabled on DX12

5

A 6 month old bug report that is still “under consideration”, for a pretty important VR feature. Thanks for the link!

It’s making me a bit worried about some other issues - like SSR being broken for VR, which I also reported.

I’ll see if I can repro this with URP and then see about HDRP.

However, I was fortunate enough to get a pretty much immediate response from mbucchia on the Virtual Desktop discord (who develops the VDXR runtime) and was chatting with him about this. He said the Unity extensions used for SRP Foveation are only for mobile support - regarding the log file snippet I showed him (thanks mbucchia!).

So there could be further issues or, if I understood him correctly, other extensions that are used for PCVR runtimes that Unity may or may not support.

His impression of Unity engine regarding support for PCVR foveation seemed fairly negative, so I hope we can get some much-needed Unity official guidance here to improve that.

6

In my URP test project, I tried disabling shadows and post-processing under the camera, and this makes no difference for me.

7

I agree with you. All of Unity’s efforts around XR seem to be focused on mobile XR.

8

Yeah - the general lack of consistency in their operational direction is a bit difficult to deal with. If I had anticipated them putting HDRP on the backburner, I probably would have used a different engine. That said I’m really happy with the visual quality I’ve got with the pipeline - Pierre D’s guides are super helpful and there’s great potential! (Thanks Pierre D! Interestingly you both share the same name!)

Anyway, I wasn’t going to post this until later, but my best theory for a solution right now is direct use of Variable Rate Shading (Unity 6.1). In my mind, this should be able to replace SRP Foveation - but again the guidance with HDRP seems to be lacking.

I will experiment with this after I upgrade to 6.3 in the future and will report back. However I still hope for an official Unity solution regarding SRP Foveation for HDRP, and for the other HDRP issues affecting VR ¯\(ツ)/¯.

9

I remembered that Valve released a Unity repo related to Steam Frame, it’s here: Unity/com.valvesoftware.openxr.utils at main · ValveSoftware/Unity · GitHub.

They mention utilities for foveated rendering - and that it requires Vulkan and URP. I have an HDRP project myself so it’s not a solution for me. Could even be related to running games directly on Frame.

10

Unity QA confirmed there are no plans to support foveation in HDRP.

I thanked them for the clarification, and asked about VRS as an alternative, so I will see if they can confirm anything regarding VRS in HDRP for PCVR.

Personally, I think the lack of foveation support is about a 7/10 severity issue, however I’ve also been in contact regarding my DLSS issue which I think is 10/10 severity, and we are still discussing that. I really hope they will agree to fix DLSS in the very short term and to fix foveation some time this year.

It would be such a relief to have one of these working properly.

11

The bug report is now closed with “Won’t Fix” Unity Issue Tracker - [DX12][XR] Foveated rendering does not work if Shadows and Post Processing are enabled on DX12 this is very frustrating.

12

Regarding the possibility of VRS replacing SRP Foveation; Unity QA told me that VRS support in HDRP was paused, but that it is technically usable through a low-level API in custom passes. They say it is theoretically possible to use VRS as a foveation solution in HDRP.

I don’t know what this entails, but they said I could write back if I want to ask more - so when I upgrade to 6.3, I’ll see if this is at all feasible for me, and try reaching out again for any guidance on this if I can’t figure it out.

I thanked them because they clarified a lot and it was super helpful!

Regarding Unity’s relatively-newfound favouritism for URP - I’ve been on all three pipelines, and HDRP was always obviously superior. The only reason you would ever switch from it is if you can’t manage performance - so that’s really all Unity needed to focus on. I don’t know why they spent time adding e.g. a compute-shader-less Forward+ to URP when the path already existed in HDRP. Just add that to HDRP and encourage migration, if it’s even really needed, because doesn’t pretty much everything now support compute shaders?

STP - did they really need to put time into that when FSR existed?

I think the backpedaling to support URP is counter-productive and isn’t helpful for Unity’s image. My viewpoint for a while has been that Unity should just kill BRP, URP, and move on with HDRP, making it easier to scale. I felt this way even when I was using URP.

Overall, HDRP is in a good place aside from this critical issue with foveation (and the DLSS sharpening issue). Some of this has been said before but I felt like adding my feedback. Overall I very much like HDRP, it’s just… not being kept up as I feel it should be. HDRP seems to receive very little attention from Unity these days - but thanks for what we have!

I’m not going to complain too much because it looks like Unreal 5 doesn’t support eye-tracked foveation, and their fixed foveation isn’t compatible with DLSS, for some reason - as far as I’ve been able to research.

It would still be nice if Unity could sort this. Thanks for reading!