Using CAH from SOHCAHTOA to translate e.MousePosition to a Vector3 where the Y axis is zero.

I’ve put a lot of time into scouring and reading forum posts, etc… so forgive me if I didn’t find the answer somewhere in the forum, because I certainly tried.

I have a custom editor program that I’m working on which needs to compare world space Vector3 positions to the Event.current.mouseposition in order to get their distance so that my program can identify which world position of an array of points is closest to the current mouse cursor position. My original implementation, which worked fine initially, was to convert each of the world space Vector3s to the mouseposition spade, which I’m pretty sure is considered GUISpace. This was accomplished using the following method:

    float GetDistanceFromMouse(Event e, Vector3 worldPosition)
    {
        Vector2 guiPosition = HandleUtility.WorldToGUIPoint(worldPosition);
        return Vector2.Distance(e.mousePosition, guiPosition);
    }

This worked great, until my array of world points filled up to a large number of points to be analyzed every pass within the scope of OnSceneGUI. That large number at the moment is 60,000 points, but I may need it to expand further than that, but currently the latency produced by this many calls to HandleUtility.WorldToGUIPoint is not going to fly.

So my new approach, was to think that instead of converting a variable number of points to match the GUISpace of Event.current.mouseposition, I would instead convert Event.current.mouseposition to match the world space of the Points and compare them.

It’s worth noting that in my implementation, the Y axis of the world space coordinates is ignored, I really just want to compare the distance between the (x,z) of the world points to the (x,z) of the Event.current.mouseposition as if they were both placed at an equal Zero Y coordinate on the Y plane.

So my new idea, was to take the source world coordinate where the mouse position enters the screen, project it down the direction of the screen based on the editor camera’s (Camera.current) forward direction and determine at what (x,z) point the ray would intersect with the Y=0 coordinate. In order to accomplish this, I assumed the following code would work:

using System.Collections;
using System.Collections.Generic;
using UnityEngine;


public class RayTest : MonoBehaviour {

    // Use this for initialization
    void Start () {
       
    }
   
    // Update is called once per frame
    void Update () {
       
    }

}



using System.Linq;
using System.Collections;
using System.Collections.Generic;
using System.IO;

using UnityEngine;
using UnityEditor;


[CustomEditor(typeof(RayTest))]
public class RayTestEditor : Editor
{
    private RayTest m_SelectedRayTest;

    private GUIStyle guiStyleLegend;

    GUIStyle Legend
    {
        get
        {
            if (guiStyleLegend == null)
            {
                guiStyleLegend = new GUIStyle();
                guiStyleLegend.alignment = TextAnchor.UpperLeft;
                guiStyleLegend.fontStyle = FontStyle.Bold;
                guiStyleLegend.normal.textColor = Color.green;
                guiStyleLegend.padding = new RectOffset(20, 20, 0, 0);
            }
            return guiStyleLegend;
        }
    }


    void OnEnable()
    {
        m_SelectedRayTest = (RayTest)target;
    }


    public void OnSceneGUI()
    {
        Event e = Event.current;
        //this prevents selecting other objects in the scene
        int controlId = GUIUtility.GetControlID(FocusType.Passive);
        HandleUtility.AddDefaultControl(controlId);
        Tools.current = Tool.None; // prevents de-selection of the gameobject when clicking outside of it

        Repaint();


        Vector3 mousePos = HandleUtility.GUIPointToWorldRay(e.mousePosition).origin;
        float originY = mousePos.y;
        float angleX = (90 - Camera.current.transform.rotation.eulerAngles.x);
        float theta = angleX * Mathf.Deg2Rad;
        float cos = Mathf.Cos(theta);
        float sine = Mathf.Sin(theta);
        float hypotenuse = originY / cos; // using CAH from SOHCAHTOA, the formula to get the hypotenuse from the know adjacent side (the height of the GUIPointToWorldRay.origin) and the theta angle ((90 - Camera.current.transform.rotation.eulerAngles.x) * Mathf.Deg2Rad)
        mousePos = HandleUtility.GUIPointToWorldRay(e.mousePosition).GetPoint(hypotenuse);

        m_SelectedRayTest.transform.position = mousePos;
        m_SelectedRayTest.transform.rotation = Quaternion.LookRotation(Vector3.down);
       
        string caption = string.Format("Origin Y: {0:N0}\r\nEuler X: {1}\r\nAdjusted X: {2}\r\nHypotenuse: {3:N0}\r\nCos: {4:N4}\r\nSine: {5:N4}\r\nResult: {6}", originY, Camera.current.transform.rotation.eulerAngles.x, angleX, hypotenuse, cos, sine, mousePos);

        Handles.BeginGUI();
        GUI.Box(new Rect(30, 20, 300, 150), caption, Legend);
        Handles.EndGUI();

    }

}

As a test, the above code should be placed in two files:
Assets\Scripts\RayTest.cs
Assets\Editor\RayTestEditor.cs

The RayTest script would then be dropped on a Quad created via the Game Object > 3D Object menu.

Upon running this, you’ll notice that you get a Y coordinate of 0 in the “Result” only when you have the mouse centered in exactly the center middle of the screen. But as you drag the mouse around the Y coordinate will move up or down.

I know that the problem is that my angle of theta currently relies solely on the angle of the Camera.current.transform.rotation, but I need to also take the angular offset that occurs when the mouse is not at the middle center of the screen into consideration, but so far I’m not sure how to find that value.

Any help would be greatly appreciated. Thanks in advance!!

Well nevermind, writing this out actually helped me to solve it…

The solution is that I shouldn’t be using the Camera.current.transform.rotation.eulerAngles.x, but rather the Quaternion.LookRotation(HandleUtility.GUIPointToWorldRay(e.mousePosition).direction).eulerAngles.x.

My new code is as follows:

    public void OnSceneGUI()
    {
        Event e = Event.current;
        //this prevents selecting other objects in the scene
        int controlId = GUIUtility.GetControlID(FocusType.Passive);
        HandleUtility.AddDefaultControl(controlId);
        Tools.current = Tool.None; // prevents de-selection of the gameobject when clicking outside of it

        Repaint();


        Ray ray = HandleUtility.GUIPointToWorldRay(e.mousePosition);
        Vector3 mousePos = ray.origin;
        float originY = mousePos.y;
        float angleX = (90 - Quaternion.LookRotation(ray.direction).eulerAngles.x); //(90 - Camera.current.transform.rotation.eulerAngles.x);
        float theta = angleX * Mathf.Deg2Rad;
        float cos = Mathf.Cos(theta);
        float sine = Mathf.Sin(theta);
        float hypotenuse = originY / cos; // using CAH from SOHCAHTOA, the formula to get the hypotenuse from the know adjacent side (the height of the GUIPointToWorldRay.origin) and the theta angle ((90 - Camera.current.transform.rotation.eulerAngles.x) * Mathf.Deg2Rad)
        mousePos = ray.GetPoint(hypotenuse);

        m_SelectedRayTest.transform.position = mousePos;
        m_SelectedRayTest.transform.rotation = Quaternion.LookRotation(Vector3.down);
       
        string caption = string.Format("Origin Y: {0:N0}\r\nEuler X: {1}\r\nAdjusted X: {2}\r\nHypotenuse: {3:N0}\r\nCos: {4:N4}\r\nSine: {5:N4}\r\nResult: {6}", originY, Camera.current.transform.rotation.eulerAngles.x, angleX, hypotenuse, cos, sine, mousePos);

        Handles.BeginGUI();
        GUI.Box(new Rect(30, 20, 300, 150), caption, Legend);
        Handles.EndGUI();

    }

Thanks for helping me to help myself Unity peeps!!

2 Likes