UIToolkit RenderTexture and UGUI World Space UI

Unity Engine
,
1

Hi,

We are using UIToolkit to generate the main UI of our app. This includes a RenderTexture visual element that displays a 3D scene. We raycast through the RenderTexture to interact with 3D objects and that is working fine using this code to determine the viewport pos:

        public static bool IsRaycastingViewport(out Vector2 localPanelPos, out Vector2 viewportPos)
        {
            localPanelPos = Vector2.zero;
            viewportPos = Vector2.zero;
           
            var mousePos =  new Vector2(Input.mousePosition.x, Screen.height - Input.mousePosition.y);
            var panelMousePos = RuntimePanelUtils.ScreenToPanel(PanelRaycaster.panel, mousePos);
            //Debug.Log($"Mouse pos: {panelMousePos}");
            var target = _panelRaycaster.panel.Pick(panelMousePos);
            if (target != null)
            {
                localPanelPos = target.WorldToLocal(panelMousePos);
                if (target.name == "RenderTexture")
                {
                    var x = localPanelPos.x / target.contentRect.width;
                    var y = 1 - localPanelPos.y / target.contentRect.height;
                    viewportPos = new Vector2(x, y);
                    return true;
                }
            }

            return false;

This info is then passed onto our ‘PhysicsRaycaster’ that handles the 3d raycast:

            var raycastResult = new RaycastResult();

            Ray ray = (isViewportPos)
                ? camera.ViewportPointToRay(pos)
                : camera.ScreenPointToRay(pos);
           
            bool physResult = Physics.Raycast(ray, out RaycastHit hit, float.MaxValue, layerMask);
            if (physResult)
            {
                raycastResult.HitGameObject = hit.collider.gameObject;

This all works fine. However, we are trying to place a world-space UGUI Canvas inside the 3d scene and naturally the Physics raycast won’t work with that. When we try to use GraphicsRaycaster it seems it requires a ScreenPosition which ends up always hitting our UIToolkit RenderTexture visual element.

It seems like we’re in a pickle as we want to ignore the RenderTexture for the ‘UGUI Raycast’ to work but also require it to determine where the Raycast through the RenderTexture should be…

What would be the recommended (if any) approach be here?

Thanks! Jake

1 Like
2

Couldn’t this be solved by using layers / layermasks?

The difference between Layers vs LayerMasks:

“There are 10 types of people in this world: those who understand binary, and those who don’t.”

3

Hey @Kurt-Dekker , thanks for the input.

I believe the EventSystem Raycast doesn’t accept a LayerMask?