Screen Space from Render Texture World Space Position

Questions & Answers
1

Hello,

I have a scene with a two camera setup. The main camera is perspective, and displays a UI that includes a panel with a render texture (render texture does not take up the entire UI). The Render Texture is created by another camera, which is orthographic, looking down -y, and provides a god’s eye view of the world. I have objects in the world and need to take their world position and translate them into screen space positions of the first camera to use for UI buttons, parented to the render texture panel (so that their positions match the world positions of the objects within the render texture).

Thanks.

5

I hope this can help someone:

using UnityEngine;
using UnityEngine.UI;

public class PositionObjectOnWorldImage : MonoBehaviour 
{
	public Camera worldCamera; //world camera with RenderTexture
	public RectTransform worldImage; //RawImage which shows RenderTexture from camera

	RectTransform helper; //object to help with transformations
	void Start()
	{
		helper = (new GameObject ("helper", typeof(RectTransform))).GetComponent<RectTransform>();
		helper.SetParent (worldImage, false);
		helper.anchorMin = Vector2.zero;
		helper.anchorMax = Vector2.zero;
	}

	public Vector3 GetPositionOnUI(Vector3 worldPosition)
	{
		//first we get screnPoint in camera viewport space
		Vector2 screenPoint = RectTransformUtility.WorldToScreenPoint (worldCamera, worldPosition);
		//then transform it to position in worldImage using its rect
		Vector2 positionInImage = new Vector2 (screenPoint.x * worldImage.rect.width / worldCamera.pixelWidth, 
			                          screenPoint.y * worldImage.rect.height / worldCamera.pixelHeight);

		//after positioning helper to that spot
		helper.anchoredPosition = positionInImage;
		//... return 3D position of that helper for any other RectTransform.position to use
		return helper.position;
	}
}

For improving speed: worldImage.rect.width / worldCamera.pixelWidth and worldImage.rect.height / worldCamera.pixelHeight should be calculated once, outside the method, and only recalculated if any of those dimensions change.

Incidently, if worldImage.rect is used to set size of world camera render target to always be 1:1, it can immediately be: helper.anchoredPosition = screenPoint;

2

So far I have:

Vector3 ConvertWorldToScreen( Vector3 positionIn ) {

Texture tex = this.GetComponentInParent<RawImage>().mainTexture;
RectTransform rectTrans = this.GetComponentInParent<RectTransform>();
Rect rect = rectTrans.rect;

Vector3 screenCoord = orthoCam.WorldToScreenPoint( positionIn )
Vector2 localPoint;
RectTransformUtility.ScreenToLocalPointInRectangle( rectTrans, screenCoord, null, out localPoint );
int px = Mathf.Clamp( 0, (int)( ( ( localPoint.x - rect.x ) * tex.width ) / rect.width ), tex.width );
int py = Mathf.Clamp( 0, (int)( ( ( localPoint.y - rect.y ) * tex.height ) / rect.height ), tex.height );

Vector3 worldCoord = rectTrans.TransformPoint( localPoint );
Vector2 offset = rectTrans.offsetMin;
return new Vector3( offset.x + worldCoord.x, offset.y + worldCoord.y, 1f );
}

The icons are slightly off, moreso in the x direction than the y (they appear above and to the right of where they should be). I have the anchor on the panel set to the lower left corner.

3

Actually this works too, although coordinates are still a bit off (looks like a scaling issue).

Vector3 ConvertWorldToScreen( Vector3 positionIn ) {
     
    RectTransform rectTrans = this.GetComponentInParent<RectTransform>(); 
    
    Vector3 screenCoord = orthoCam.WorldToScreenPoint( positionIn );
    Vector3 worldCoord = RectTransformUtility.PixelAdjustPoint( screenCoord, rectTransform.transform, rectTrans.GetComponentInParent<Canvas>() );
    
    Vector2 offset = rectTrans.offsetMin;
    return new Vector3( offset.x + worldCoord.x, offset.y + worldCoord.y, 1f );
}
4

I have the same problem, anyone know a solution to this?