Trying to get an HDRP portal to work on VR

[Sheynes]

NEWS : This subject is under an unity issue. Please vote here if you also want portals systems in VR Unity Issue Tracker - [Shadergraph] ScreenPosition node does not work when XR is active

Hello there !

I’ve made the project free to access here : Sheyne / HDRPPortalVR · GitLab
So the main goal is to have a working HDRP VR portal (And potentially URP later), since I havn’t found any on the asset store nor on the internet.
I managed to get a working HDRP portal system.

I’ve been using this youtube video :

Coding Adventure: Portals

Sebastian Lague do share it’s portal project with us on github : GitHub - SebLague/Portals: Portals in Unity

So basically I used shadergraph with the Unlit node and I converted the shader he’s using to have his code working with HDRP. I also had to put his render function for the portals in a LateUpdate loop, since the OnPreCull event was removed from HDRP as seen here : [https://forum.unity.com/threads/fee...e-render-pipelines.470095/page-8#post-3408481]( Feedback Wanted: Scriptable Render Pipelines page-8#post-3408481)

The Shader in shadergraph is very simple. I just connected the screen position as UV and declared a texture2D as an entry for the texture sampled.

A camera for each portal follow the player camera location and the render will be captured on renders textures to apply them on the portals materials, using the shader above.
As it can be seen, in this video : Screen capture - 338782343c0992fc2fcbbf2cb9c1975c - Gyazo the cameras follow the player location.
I also had to remove the post processing from this cameras because it stacked with the player post processing camera and ended up showing different results on the portal.

Then, our portal.cs script apply at runtime a texture2D from the camera attached to the portal to the portal screen material. It’s done here : https://hastebin.com/qewevarati.csharp in the void render() function. This function is called as I said above in late update, as the last script in the script execution order.
The script also do other things like taking care of cutting the mesh when traveling the portal but it doesn’t matter for now.

Here is the result : a portal properly working in HDRP Screen capture - 8ad09a3aec9d4762d678017c508954e4 - Gyazo

Now the problem is : Whenever I try to plug my VR headset on (Oculus CV1), everything turn wrong. The result can be seen here : Screen capture - 79ce55f67240dd13056e33b66b95a25d - Gyazo
It seem to put both player eyes on the screen as a render texture at the same time.

I’d like to be able to apply the left eye of the render camera on the left eye render of the portal and the right eye of the render camera on the right eye render of the portal.

I do not really know where to start, and if it require to change the shader in shader graph if there is even a way to do it. Since we are on HDRP we cannot really handwrite the shaders.

Thanks if you have any start of answers !

[Sheynes]

It looks like someone over here ran into the same problem as me : creating portal effect using a render texture and Shadergraph with screen position node in VR - Questions & Answers - Unity Discussions

[Sheynes]

Getting Screen Coordinates for Single Pass Stereo VR?

I'm trying to make a basic depth shader for VR using LWRP, which means I'm required to use single pass stereo. I'm aware that this means my depth texture is twice the width of each eye. I can't seem to find a way to correctly sample this depth texture so that it would appear the same in stereo as it would in mono. Each eye has the same screen coordinates, and so the double-wide depth texture looks like this.[image] I would like to ultimately have the left eye go from 0 to 0.5 and the right …

In this link, the author suggest to do a custom node in shader graph with this piece of code

OUT = IN;
float4 scaleOffset = unity_StereoScaleOffset[unity_StereoEyeIndex];
//check that stereo is enabled
if (scaleOffset.x > 0){
    OUT.x /= 2;
    OUT.x += scaleOffset.z;
}

Which end up looking like this :

Unfortunately, the shader is pink :

With this message : Shader error in ‘Shader Graphs/PortalSheyneVR’: undeclared identifier ‘unity_StereoScaleOffset’ at line 2254 (on d3d11)

The unity documentation suggest me to enable the single pass in stereo rendering method : Unity - Manual: Single Pass Stereo rendering (Double-Wide rendering)
But it is obsolete, since I am using the XR plugin management :

And using Multi Pass or Single Pass Instantied doesn’t change anything :

It do seem like I have to find a way to replace the “unity_StereoScaleOffset”. Does someone have an idea ?

[Sheynes]

Ok, after futher investigation it looks like
unity_StereoEyeIndex
and
unity_CameraProjection
cannot be used neither in the custom shadergraph node for some reason. I start to think that something is missing in my shadergraph files.

Edit: I had undeclared identifier errors in shadergraph with HDRP 7.3.1, I still have this problem in HDRP 7.5.2 and rollbacked in 7.3.1

[Sheynes]

I uploaded a version of this project here : Sheyne / HDRPPortalVR · GitLab if someone want to give it a try.

The scene is under the scene folder. The vr device HMDMock should be on by default so you can try the scene even without any headset, and the shader is actually in portal/Scripts/Core/SheyneVRPortal it doesn’t work on vr tought.

[Sheynes]

Ok it seem that the VR camera flipped bug came from the HDRP 7.3.1 as said here : [VR] Additional cameras render to texture issue
I’m updating to 7.4.1

EDIT : It ended up on an Issue, the same as here : SteamVR doesn't work with HDRP 7.4.1

I’m updating to 7.4.2

Doesn’t working.

EDIT 2 : It is finally working on HDRP 7.4.3 and Unity 2019.4.16f

But the portal ScreenPosition is all fucked up in the shader thought : Screen capture - 1bc7fb14259ed924fde49d8a28c7dc4c - Gyazo

So to resume :
Now I need

1. to force render my portals cameras in VR mode,
2. to capture the left render on a render texture, and the right render on another render texture
3. to change my shader so it take the left texture on left eye and the right texture on right eyes (I’m not even sure I can do that in HDRP)
4. to hope using screenPosition will be enough in the shader to put the textures on the plane without artefact, or to find a way to compute the right position of the portal screen on the player view.

[Sheynes]

So it’s said here : How to create stereo RenderTextures and cameras?
That unity doesn’t actually support multiple vr camera for performance reason, which mean i’ll have to uses 1 vr camera and 4 cameras, two per portal. It start to seem crazy.

[Sheynes]

So far I havn’t been able to make it work. I am still looking for it. I feel like i need to use a written shader instead of shadergraph. I’m trying to hire people to make one. So if you feel interested and feel like you can do it drop a PM.

To resume : It works in HDRP non VR.
It doesn’t work on HDRP VR

[sunsi12138_1]

Hello,I think it is because the normal portal shader is using screen position to remap the texture.But in vr the camera would render twice in different position and combine them to one.And I’m working on these effect too,maybe I will post my further progress soon.

[ArianSadafi]

Hello,
I’m trying the same on URP and have the same problem with VR. Could anyone find a solution?

[Sheynes]

I am going to give this a new try this evening. Hopefully since 6 months elapsed Unity may have debuged shadergraph to allow the use of VR on the screen position node.
@sunsi12138_1 any news from your side ?

[Sheynes]

So I updated the project to unity 2021.2, added two camera per portals, changed the script, made good use of the Eye Node from shadergraph 12.0.0. So far so good. Only one limitation : Shadergraph screenposition node still not take account the fact you are using a VR headset. So the render textures from the cameras doesn’t fit the portal screen properly.

I don’t seem to be able to do any call to unity keywords in the custom node from shadergraph. So i’m blocked. It seem just impossible to do a proper portal without hand writing a shader. And unity provide no documentation about a way to do that outside of shadergraph. The shader is Unlit, so it should not be too much to handle for a skilled shader person.

I am willing to pay someone to solve this issue. So feel free to contact me in DM if you are interested in making this portal system to work in VR.

[Sheynes]

Out of despair I opened a report issue, asking if it was possible to fix the ScreenPosition node for use in VR, or at least to give me a workaround

[Sheynes]

So here is the link with the issue, if anyone want vr portals, please upvote this Unity Issue Tracker - [Shadergraph] ScreenPosition node does not work when XR is active

[Cyril_m]

I’ve been following the same tutorial (best portal tutorial I’ve seen so far tbh), sticking to non-VR (for now) but still using HDRP. I’d solved the OnPreCull issue (differently, yours is simpler and best I think) but couldn’t get past the SetTexture one. Sadly the hastebin link seems to be dead, would mind telling me how you got past this, pretty please? Or fixing the link? That’d be much, MUCH appreciated

Edit: there also seems to be an issue with recursive rendering, I had to comment out the portalCam.Render() bit from Render() (my script is still in an early state though since I couldn’t get past the part I mentionned above)

Edit2: solved my problems, nevermind!

[Sheynes]

It’s been two more months now and so far we are still waiting for Unity to fix this issue to be able to create a VR portal. with shadergraph Unity Issue Tracker - [Shadergraph] ScreenPosition node does not work when XR is active
If you are also looking for VR shaders, feel free to upvote it so it gets more priority

[Shizola]

I voted. Did they reply? They normally reply pretty quick to my bug reports.

[Sheynes]

@Shizola

I voted. Did they reply? They normally reply pretty quick to my bug reports.

They actually replied two day after I summited the bug report, so they created the ticket properly. But now it is in a stage where it’s on their todo. And since two months it’s been locked in this state with no news.
Does unity usually reply after the bug is in the issuetracker ? Cause they havn’t to me

[AdamBebko]

Same problem here, I’m also willing to pay a bit if someone can figure it out.

For me, it seems the right eye’s view is in the spot of the left eye on the render texture and vice versa, causing you to see double. I’m not too savvy with shaders, and this problems is over my head. I’m on URP.

My scene is like this:
A secondary camera capturing the view through the portal, then using its render texture in a shader graph to paint in the pixels on the other end of the portal in screen space.

[AdamBebko]

Here’s an example. In this scene, the portal camera is a child of the main camera, so if everything works, the portal should basically be invisible besides its little border. You can see that in the portal, the left eye is seeing what appears to be the right eye’s view, and the right eye is seeing the left eye’s view.

For example, imagine cutting out the portal from the left eye’s screen and pasting it in the right eye’s box. It would then be correct I think.

Know any way to swap the eyes in the shader?