Duplicating VR stereo rig set up for render textures

Unity Engine
1

Hi,

I’m developing for Oculus Quest and apparently, single view stereo is not supported, so I’m using multi-view stereo where the scene is rendered twice. I’m trying to create a portal effect by drawing to two render textures (one for each eye). However, I can’t get this to look quite right in VR and I’m stumped as to why. It’s just a little off.

To create the effect, I take the position of Camera.main relative to the portal and then position cameras on the other side of the portal, one for each eye in stereo mode, otherwise just a single camera.

I obtain the projection matrices from Camera.main. Spacing the cameras apart by the stereo separation and rotating for convergence seems to have no discernible effect.

Here is how it should look (a working screenshot when running on the PC without stereo; notice how the transition from the blue map to the orange map lines up perfectly):

6055820--655358--Perfect.png
6055820--655358--Perfect.png889×407 39.5 KB

And in VR:

6055820--655361--Imperfect.jpg
6055820--655361--Imperfect.jpg1440×1440 125 KB

I’m not sure what accounts for this discrepancy. Here is the code (attached to the portal object). As you can see, I’m trying to match the camera parameters to the left and right eyes. I assume that the projection matrices contain the stereo offset and convergence.

using System;
using UnityEngine;

public class Portal : MonoBehaviour
{
  [Tooltip("Camera observing the other side of the portal.")]
  [SerializeField]
  private Camera m_otherCamera;

  [Tooltip("The other portal transform, which must be the equivalent transform to this portal's.")]
  [SerializeField]
  private Transform m_otherPortal;

  private MeshRenderer m_ourPortalRenderer;

  private void LateUpdate()
  {
    Transform cameraTransform = Camera.main.transform;
    Vector3 userOffsetFromPortal = cameraTransform.position - transform.position;
    m_otherCamera.transform.position = m_otherPortal.transform.position + userOffsetFromPortal;
    m_otherCamera.transform.rotation = m_otherPortal.rotation * Quaternion.Inverse(transform.rotation) * cameraTransform.rotation;
  }

  private void Start()
  {
    if (m_otherCamera.targetTexture != null)
    {
      m_otherCamera.targetTexture.Release();
    }

    Debug.LogFormat("Stereo={0}", Camera.main.stereoEnabled);
    Debug.LogFormat("Separation={0}", Camera.main.stereoSeparation);
    Debug.LogFormat("Convergence={0}", Camera.main.stereoConvergence);

    if (!Camera.main.stereoEnabled)
    {
      m_otherCamera.targetTexture = new RenderTexture(Camera.main.pixelWidth, Camera.main.pixelHeight, 24);
      m_ourPortalRenderer.material.SetTexture("_LeftEyeTexture", m_otherCamera.targetTexture);
    }
    else
    {
      // Disable the camera and attach stereo cameras
      m_otherCamera.enabled = false;

      //float separation = 0.5f * Camera.main.stereoSeparation;
      //float convergenceAngle = 90f - Mathf.Atan2(Camera.main.stereoConvergence, separation) * Mathf.Rad2Deg;

      GameObject left = new GameObject("LeftEye");
      left.tag = m_otherCamera.gameObject.tag;
      left.transform.parent = m_otherCamera.transform;
      GameObject right = new GameObject("RightEye");
      right.tag = m_otherCamera.gameObject.tag;
      right.transform.parent = m_otherCamera.transform;

      Camera leftCamera = left.AddComponent<Camera>();
      Camera rightCamera = right.AddComponent<Camera>();
      leftCamera.CopyFrom(m_otherCamera);
      rightCamera.CopyFrom(m_otherCamera);

      leftCamera.projectionMatrix = Camera.main.GetStereoProjectionMatrix(Camera.StereoscopicEye.Left);
      rightCamera.projectionMatrix = Camera.main.GetStereoProjectionMatrix(Camera.StereoscopicEye.Right);

      leftCamera.targetTexture = new RenderTexture(Camera.main.pixelWidth, Camera.main.pixelHeight, 24);
      rightCamera.targetTexture = new RenderTexture(Camera.main.pixelWidth, Camera.main.pixelHeight, 24);

      leftCamera.enabled = true;
      rightCamera.enabled = true;

      // Messing with the separation rotation doesn't help
      left.transform.localPosition = Vector3.zero;
      right.transform.localPosition = Vector3.zero;
      //left.transform.localPosition = -Vector3.right * separation;
      //left.transform.localRotation = Quaternion.AngleAxis(convergenceAngle, Vector3.up);
      //right.transform.localPosition = Vector3.right * separation;
      //right.transform.localRotation = Quaternion.AngleAxis(-convergenceAngle, Vector3.up);

      m_ourPortalRenderer.material.SetTexture("_LeftEyeTexture", leftCamera.targetTexture);
      m_ourPortalRenderer.material.SetTexture("_RightEyeTexture", rightCamera.targetTexture);
    }
  }

  private void Awake()
  {
    m_ourPortalRenderer = GetComponentInChildren<MeshRenderer>();
    Debug.Assert(m_otherCamera != null);
    Debug.Assert(m_otherPortal != null);
    Debug.Assert(m_ourPortalRenderer != null);
  }
}

My shader samples from either _LeftEyeTexture or _RightEyeTexture depending on which eye is being drawn.

2

I have the exact same problem, did you ever figure this out?

3

Sadly, I never did. There is another thread in one of the Unity sub-forums that I started with some more discussion. But I don’t think a working solution was ever found. Some people are apparently able to do it, though.