[Solved] [C#] Manually Convert World to Screen Position

Unity Engine
1

I need to take this Function Camera.WorldToScreenPoint(Vector3 Position) and write it my self so I can use it in a separate thread. I’m calculating hundreds of these at once so it’s important that I keep it off the main thread so it doesn’t use tons of CPU power.

I could do it in open GL converting it via Matrix but I’m not sure what to do in Unity.

Anyone have any ideas?

Solved.

 /// <summary>
    /// Updates The Target Icons Position (Threaded Version).
    /// </summary>
    void UpdateTarget(TargetUIInfo info)
    {     
        bool behindCamera = false;
        // If target is pinned to a side of the screen.
        bool horizontalPin = false;
        bool verticalPin = false;
        // Calculate Screen Point of Object.
        Vector3 pointOnScreen = Vector3.zero;
        Vector4 v = _ViewPort * new Vector4(info.targetPosition.x, info.targetPosition.y, info.targetPosition.z, 1);
        Vector3 _ViewPortPoint = v / -v.w;
        // Will keep _ViewPort Under 1 for multiplication.
        Vector3 normalized_ViewPort = new Vector3(_ViewPortPoint.x + 1, _ViewPortPoint.y + 1, 0);
        normalized_ViewPort /= 2;
        // Get Vector (Distance essentially to use in Dot Product).
        Vector3 heading = info.targetPosition - _CameraPosition;
        // Check is object is behind camera.
        behindCamera = (Vector3.Dot(_CameraForward, heading) < 0) ? true : false;
        if (behindCamera)
        {
            // Calculate Inverse Transform Point to determine where target is located behind camera.
            Vector3 inverseTransPoint = Quaternion.Inverse(_CameraRotation) * (info.targetPosition - _CameraPosition).normalized;
            // Positive Right / Negative Left
            if (inverseTransPoint.x > 0)
            {
                pointOnScreen.x = Screen.width;
                horizontalPin = true;
            }
            // Positive Top / Negative Bottom
            if (inverseTransPoint.y < 0)
            {
                pointOnScreen.y = Screen.height;
                verticalPin = true;
            }
            // Position target positions to side of screen if inverseTransPoint indicated pinning.
            if (!verticalPin)
                pointOnScreen.y = normalized_ViewPort.y * Screen.height;
            if (!horizontalPin)
                pointOnScreen.x = normalized_ViewPort.x * Screen.width;
            // Special Case if both are 0/0 and target is behind.
            if (!verticalPin && !horizontalPin)
            {
                pointOnScreen = info.targetPosition;
            }
        }
        else
        {
            // Regular Positioning of Target Icon
            pointOnScreen.x = normalized_ViewPort.x * Screen.width;
            pointOnScreen.y = normalized_ViewPort.y * Screen.height;
        }
        // NGUI Required (Will not be neccesary when Moved over to Unity UI).
        pointOnScreen *= ratio.y;
        // Set position of UI Overlay (via Cache as transform cannot be accessed in Thread).
        info.updatedPosition = pointOnScreen;       
    }
2

What thread you do it in will not affect how much “CPU power” it uses.

The trick with performant game engineering is to find out what you DON’T have to calculate every frame.

And in any case, don’t optimize until you see an actual performance problem. And if you see a performance problem, the first thing you must do is MEASURE where it is coming from (see the Profiler) before making random changes to your codebase.

1 Like
3

Nice Sentiment but I’ve actually optimized it already but I still want it off thread because it’s not super important but it’s important to keep up to date.

If you’re wondering what it is, it’s target reticles for potentially hundreds of (filterable) targets. And depending on how fast you are moving / turning they can move pretty quickly so I need them updated every frame.

Also they may come in and out of existence and are not sorted so trying to frame split the calculations can cause targets to not be updated properly.

Keeping them off thread allows a continuous update without actually using cpu cycles from the main thread.

And yes it definitely affects how much CPU power it uses, especially on systems with multiple cores.

I have multi-threaded tons of things and noticed massive performance improvements, need examples?

1 Like
4

How about making them follow world position of their objects, drawn last (on top) with scale and rotation controlled (or possibly in vertex shader)

On topic: this should get you viewport coordinates (tho my matrix math is a bit rusty)

        Matrix4x4 MVP = cam.projectionMatrix * cam.transform.localToWorldMatrix.inverse * objectTransform.localToWorldMatrix;
        Vector4 v = MVP * new Vector4( 0, 0, 0, 1 );
        Vector3 ViewportPoint = v / -v.w;
5

It occurs to me this morning that you probably just want to give a world position, instead of an object transform, inwhich case you can simplify a little to

Matrix4x4 VP = cam.projectionMatrix * cam.transform.worldToLocalMatrix;
Vector4 v = VP * new Vector4( worldPos.x, worldPos.y, worldPos.z, 1 );
Vector3 ViewportPoint = v / -v.w;
6

I was actually looking into those matrices earlier but got side tracked. Thank you for your input I will implement later.

The problem is they act as buttons and having them off In the distance may not work. So the ui scaling in the distance would probably not work as ray cast from mouse point is finicky.

7

Let me make sure I understand your problem.

  • you have a large block of Vector3 data, probably originating in transform.position fields on your GameObjects
  • every frame you want to run this data through a world-to-screen transform so you have an up-to-date screen coordinate for each worldspace Vector3

Option 1: (which I would choose)

  • just pump it all through the function on your main thread and use the results

Option 2: (which you propose)

  • create a worker thread to do the work, and give it a shared area of memory to use

Now every frame you will:

  • reach a point when you have updated all of your object positions
  • synchronize with your worker thread (i.e., stop the main thread and wait) to make sure it is “Ready” to receive data
  • marshall (copy) all the Vector3 data from main thread memory (GameObjects, etc.) over to the shared memory area
  • signal the worker thread that it may begin processing

Now you will go about the rest of your frame, until you need the converted data back from your worker thread.

  • now you will block to verify that the worker thread has indeed finished processing everything for this frame
  • when it is finished, you will marshall (copy) the data back from shared memory and into every place that it was originally.

Now you can go ahead and do things with the resultant computations.

Am I missing something?

1 Like
8

I actually dont care whether its ready.

i start the worker thread,
quickly copy the entire array
compute all positions
send a vector2 back because 3 is pointless in 2d space

(main thread updates based on position change)

so it wont be every frame and it will be on a spread out core computation wise.

I need the main thread for AI as there are hundreds of them if not thousands of pieces.
I have it option 1 atm, it claims way too much main thread time.

Edit: Matrix Calculations are extremely expensive.

9

My information may be wildly out of date, but I don’t think Unity utilises more than one core.

EDIT: You say you can’t frame split the points, but have you thought about some sort of space partitioning? That way, you would be able to prune a bunch of points before calculating anything.

DOUBLE EDIT: Whatever you choose to do, I’d probably start out by implementing a working, naive solution first, so you can compare. Do the matrix transforms first, IMO.

10

Came up with a way that doesnt use matrices

Projects the world position onto the cameras near clip plane, then converts to vieport space

The ‘intersection’ veriaable is the world position on the projected point, if you want to use a world space UI, then you could stop at that point

    public Transform objectTransform;
    public Camera cam;
    public RectTransform image;
    public RectTransform canvas;


        Vector3 camPos = cam.transform.position;
        Vector3 camDir = cam.transform.forward;
        Vector3 camNearClipPoint = camPos + camDir * cam.nearClipPlane;

        float nearPlaneHeight = ( Mathf.Tan( cam.fieldOfView * 0.5f * Mathf.Deg2Rad ) * cam.nearClipPlane );
        float nearPlaneWidth = nearPlaneHeight * cam.aspect;

        Vector3 worldPos = objectTransform.position;
        Vector3 dir = ( camPos - worldPos ).normalized;


        float dotNumerator = Vector3.Dot( ( camNearClipPoint - worldPos ), camDir );
        float dotDenominator = Vector3.Dot( dir, camDir );
        float length = dotNumerator / dotDenominator;
        Vector3 intersection = worldPos + dir * length;


        Vector3 intersectionDir = Quaternion.Inverse( cam.transform.rotation ) * (intersection - camNearClipPoint);
        Vector3 viewPortPos = new Vector3( intersectionDir.x / nearPlaneWidth, intersectionDir.y / nearPlaneHeight, 0 );

        image.localPosition = new Vector3( viewPortPos.x * canvas.rect.width * 0.5f, viewPortPos.y * canvas.rect.height * 0.5f, 0 );



    }
11

I don’t know, man. I think you can do all that in a single matrix multiplication.

EDIT: If the problem is not being able to use Unity’s API in your other thread, I would write my own matrix multiplication algorithm, since you can get the projection matrix from the camera(Camera.worldToCameraMatrix). Write your own basic Matrix class, if you can’t use Unity’s.

12

Thanks for all the input guys, Already implemented. Saved about 9 FPS when calculating > 100 targets at once.

Unity Does utilize more than one core, they even say it in the patch notes. It just doesn’t have native threading through scripts so you have to manually use thread pools or threads.

So you cannot access things like GameObject / Transform / Camera Matricies.
So you cache them and then call them in the threads obviously checking for their existence first as threads don’t throw error messages in the console.

13

Cool. Mind showing us how you did it?

14

So it turns out I need to figure out the formula for InverseTransformPoint also. Unfortunately I am not well versed in Matrices.

15

Sure once I solve the issue of getting the direction when the object is behind. Which I need to solve InverseTransformPoint for. The code is currently very dirty and I’d like to show off the clean version. It’s also using NGUI from back when I was experimenting with this project so it’s a bit odd in some parts.

Edit: Solved it. Simple

Quaternion.Inverse(Camera Rotation) * (Target Position - Camera Position));
16

I assume you know this, but in case someone reads this thread and doesn’t.

The transformation/projection matrix is just a single matrix you can multiply a vector by, that both translates(moves), rotates, scales and shears. So, if you have a point in 3D space, and you multiply it by the cameras projection matrix, you get the points position on the screen. The neat thing here, is that you multiply all the points by the same matrix.

All transforms have localToWorldMatrix and worldToLocalMatrix, and I bet if you decompile the transform functions, they just multiply be these. If you don’t have one of them, you can always get the inverse from Matrix(it has a variable .inverse).

EDIT: I just realized hpjohn mentioned most of this. In any case, I’d advise anyone looking to manipulating points and vectors in 3D to get a basic understanding of transformation matrices.

This is probably a good article.

17

Yeah I’m beginning to understand these things. Unity usually does the work for you but being on another thread doesn’t allow me to access.

18

You could make your own matrix class, just 4x4 array, and write your own matrix multiplaction method. It’s pretty simple. Finding the inverse is much more complicated, so if possible, I’d send both matrices to your thread.

If you don’t want to bother with it, you could also just use a library.

19

I’m all set actually. I figured it all out, it works just like it did in the main thread (conceptually) I just have to test it out full scale again tomorrow. Then I’ll post the code.

20

Looking forward to it :slight_smile: