Overlay canvas and world space coordinates

Unity Engine
1

I’m looking for good and reliable ways of translating coordinates between screen space (and overlay canvas) position and vice versa, that also take canvas scaling into consideration.

My common use cases are:

  1. Translate mouse position to world position.
  2. Translate game object position from the world to overlay position in order to place UI element near that object.
  3. Translate anchored overlay UI element to world position.

For case 1 I use Camera.main.ScreenToWorldPoint(Input.mousePosition) and it seems to work okay for my purposes.

For case 2 I use RectTransformUtility.WorldToScreenPoint(Camera.main, obj.transform.position);

For case 3 I use Camera.main.ScreenToWorldPoint(rectTransform.transform.position) and it seems to give good results.

Are there any officially advised ways of translating between overlay canvas and world space coordinates?

31 Likes
2

this is the best way to translate between these two coordinate systems.

You could do a Matrix 4x4 transform, but it’s alot more complicated than the provided Unity Methods

1 Like
3

Hallelujah! I’ve been fighting with case 3 for a little while now. Appears to work for mapping an object in world space to a point on the overlay UI.

The only issue I had is that it maps my sprite to the depth (z) of the camera, and the sprite won’t render that way initially. Easy to work around. Does anyone know of a built-in fix?

4

I just want to say I keep coming back to this post. So helpful.

5 Likes
5

I found this after a couple years because i was having a problem with the conversion between world coordinates back to screen coordinate position…

WorldToScreenPoint is slightly off!

When canvas scaler mode is set to: stretch to screen size

The problem is that the canvas screen point is not actually the same screen size as the current resolution… the canvas screen size is more like, the best optimized view screen resolution… under the stretch parameters you can see the variables for Reference Resolution… this is the resolution of the canvas.

A better explanation of CANVAS SCALING is by using my most recent problem…

Quick Description of my problem:
I was using the Canvas to display a sprite ( a healthbar) which floats over all the characters. The problem however, was when i tried to obtain the WorldToScreenPoint it kept giving a result that was slightly off… for example: the healthbar looked a little okay when the character was immediately in front of the camera… but as the character walks to the edge of the camera’s fulstrum, the screens x,y placement becomes more and more incorrect.

Days and days of research and trying different combinations finally showed me that maybe there is a scaling issue which pointed me to look at Canvas / Canvas Scaler / scaling mode: scale with screen

Originally, this worked wonderfully when i had only 1 character and his healthbar stayed stuck to the top of the screen like old classic double dragon games. BUT when i made the decision to have many characters and they all need “floating healthbars”, i didnt come back to re-evaluate whether this option needed to change.

Setting the canvas Scaler to: keep constant pixel size
, fixes the problem and i now have the correct WORLDtoSCREENpoint that i needed! And now the healthbar floats beautifully above the characters…

BUT WAIT, ANOTHER PROBLEM! Now, if the screen resolution is small… the Ui sprite is obsurdly large… and if the screen resolution is high definition then Ui sprite is way too small!

QUESTION: So how do i use the “scale with screen size” mode, but yet also still get back a correct WorldToScreenPoint?

ANSWER: you must take into consideration the overal scaling of the canvas when it is stretched to fit (whatever current resolution that you are using)

INSTEAD OF:

   RectTransform myRect = GetComponent<RectTransform>();
    Vector2 myPositionOnScreen = Camera.main.WorldToScreenPoint (myOwner);
    myRect.anchoredPosition = myPositionOnScreen

YOU CALCULATE THE OVERALL SCALE FACTOR LIKE THIS:

RectTransform myRect = GetComponent<RectTransform>();
        Vector2 myPositionOnScreen = Camera.main.WorldToScreenPoint (myOwner);
 
    Canvas copyOfMainCanvas = GameObject.Find ("Canvas").GetComponent <Canvas>();
    float scaleFactor = copyOfMainCanvas.scaleFactor
 
    Vector2 finalPosition = new Vector2 (myPositionOnScreen.x / scaleFactor , myPositionOnScreen.y / scaleFactor);
    myRect.anchoredPosition = finalPosition;

If this helped anyone please log in to give me a thumbs up…

29 Likes
6

Great answer @karma0413 . Thanks a lot!

1 Like
7

Thanks for this thread! and this reply!

@karma0413 , One question though. This doesn’t work when I assign this to “RectTransform.AnchoredPosition”.
Don’t I need to use below?
Vector2 myPositionOnScreen = Camera.main.WorldToViewportPoint(myOwner);

 public static Vector2 GetAnchoredPositionFromWorldPosition(Vector3 _worldPostion, Camera _camera,  Canvas _canvas)
        {
            //Vector2 myPositionOnScreen = _camera.WorldToScreenPoint(_worldPostion); // for transform.position?
            Vector2 myPositionOnScreen = _camera.WorldToViewportPoint(_worldPostion); //for RectTransform.AnchoredPosition?
            float scaleFactor = _canvas.scaleFactor;
             return new Vector2(myPositionOnScreen.x / scaleFactor, myPositionOnScreen.y / scaleFactor);
        }
1 Like
8

work with me

1 Like
9

Hello everyone, sorry for replying to this old topic but Im stuck and I cant find a proper solution. :frowning:

Apparently case 3 is not working for me, It looks like the issue is with the canvas render mode ( Im using Screen Space-Camera ).
Do you guys know how to solve it?

Thank you very much :slight_smile:

2 Likes
10

Hope the Thread is not to old for answering. Just pops up when you search UI and WorldSpace issues.
None of the solutions worked for me.
I have some UI Icons and I want a worldSpace gameObject flys to the position.

My best solution so far is:

Vector2 myPositionOnScreen = Camera.main.ViewportToWorldPoint(gameObject.transform.position);
Vector3 finalPosition = new Vector3(myPositionOnScreen.x, myPositionOnScreen.y * 3, 0f);

The x coordinate is kind of accurate but the y position is around 3 times to small. Was also checking the _canvas.scaleFactor, which is around 1.5f in my case so it’s not covering my case here.

If someone knows the “right way” I would appricate it!

11

** @allpublic **

“I have some UI Icons and I want a worldSpace gameObject flys to the position.”

So which canvas type are you using?

Screen Space - overlay canvas, or perhaps some other canvas type?

Remember, when you are working with UGUI, you are working Canvas space, and most likely is different for different canvas types.

It probably would be better to ask this in your own thread, with detailed information about your setup.

12

I’m trying to do this exact thing. (“I have some UI Icons and I want a worldSpace gameObject flys to the position.”) I’m using a Canvas with Screen Space - Camera setting (not using Canvas Overlay).

I can move the world space game object but it’s not going to the correct position for the UI icon on the canvas.

Can you please post your solution? I’d pay for help with this. :slight_smile:

This did not work for me:

theWorldSpacePositionOfUIElement = Camera.main.ScreenToWorldPoint(myUIElement.rectTransform.transform.position);

Thanks very much!

  • Jeff
13

@FiveFans

It ain’t going to work simply like that. Your UI plane in front of camera could be of any size based on its distance from camera, and when using screen space camera your object could be anywhere in 3d space…

In addition to that, a personal observation. IMO this works best if you turn your object into an ‘icon’ on click, then make the icon object fly towards the defined Canvas UI element location in Canvas space. Here both Screen Space - Overlay and Screen Space - Camera worked IIRC but it has been a while. Both chest and flying icons are uGUI UI elements, and coins on the ground are GameObjects.

7511747--926186--20210921_fly_to.gif

1 Like
14

@eses

Thank you for your help. I started down the path of making the item that the player collects also be instantiated on the UI Canvas so that I could move it easily to the target location on the canvas. That was a really good suggestion.

I happened upon a solution on StackExchange that gave the position of a World Space object (collected by the player) on Canvas space. I reversed it and now it works as I needed.

UICamera is my camera used for the Screen Space - Camera setting on the Canvas (I don’t use the Overlay or World canvas)
thisCanvas is the canvas.

var position = UICamera.WorldToScreenPoint(MyUIObject.transform.position);
position.z = (thisCanvas.transform.position - UICamera.transform.position).magnitude;
theLocationOfUIObjectInWorldSpace = Camera.main.ScreenToWorldPoint(position);

My game is 2D and it works exactly as I needed now. I’m a happy person now. I hope this can help someone in the future.

  • Jeff
4 Likes
15

@FiveFans @eses

Thanks you so much for this!

I was having a nightmare trying to get textmesh score game objects to traverse towards my gui score counter but this last post has sorted it for me.

The only adjustment I made was to the ‘plane distance’ on my canvas, it was set to 100 and as a result the initial text was reducing in size as it traversed in the z plane, to a point where it was becoming unreadable. Changing it to 20 resolved that.

I’ve posted my camera and canvas setups if it helps anyone else.

Thanks again!
LS

7773870--980247--Canvas.jpg 7773870--980250--Cameras.jpg

16

The value of these 3 lines cannot be overestimated. I thought it would be easier. Thank you, you saved my day!

17

This thread has helped me tremendously to figure out this problem. Honestly, if feels like I’m playing Wordle, trying to find the combination of techniques that work based on Camera modes and Canvases, i.e. Overlay, Camera Space, etc. What works in one mode, breaks in another.

My situation is using Cinemachine, Timeline, URP and stacked cameras. My main camera is transitioning from one position to another, so I can’t use Camera.main for positioning until the frame where I need this to happen occurs, so I am enabling this script at that point.

Scene is set up like this:

  • MainCamera is Base
  • UICamera is Overlay, Stacked and a child of the MainCamera
  • UICanvas is Screen Space - Camera and is also a child of MainCamera (not sure if this is necessary

The goal is to underline an array of UI text with a linerenderer that then points to a real world object. This is to label parts of a 3D model. I found this script worked for my setup:

using UnityEngine;

public class UILineLabels : MonoBehaviour
{
    public RectTransform[] m_labels;
    public Transform[] m_targets;
    public LineRenderer[] m_lines;
    public Camera m_UICamera;
    public Canvas m_canvas;
    private void OnEnable()
    {
        for (int i = 0; i < m_lines.Length; i++)
        {
            Vector3[] corners = new Vector3[5];
            m_labels[i].GetWorldCorners(corners);

            var position = m_UICamera.WorldToScreenPoint(corners[0]);
            var position1 = m_UICamera.WorldToScreenPoint(corners[3]);

            position.z = (m_canvas.transform.position - m_UICamera.transform.position).magnitude + .1f;
            position1.z = (m_canvas.transform.position - m_UICamera.transform.position).magnitude + .1f;

            Vector3 firstLinePosition = Camera.main.ScreenToWorldPoint(position);
            Vector3 secondLinePosition = Camera.main.ScreenToWorldPoint(position1);

            Vector3[] positions = new[] {firstLinePosition , secondLinePosition,  m_targets[i].position };
            m_lines[i].SetPositions(positions);
        }
    }
}

Note that this assumes the same number of Labels, Targets and Lines. Resizing the width of labels allows for making the underline match the text.

18

you are my life save thx XD

19

I’ve tried lots of things from this thread and others and have had the most luck with:

uiObject.transform.position = RectTransformUtility.WorldToScreenPoint(Camera.main, worldObject.transform.TransformPoint(Vector3.zero));
2 Likes
20

Thanks man :sunglasses: