Strange Editor Raycast bug

Unity Engine
1

I have a project that uses raycasting in the editor to determine the 3d world position of the mouse cursor. The code is pretty straight forward and has always worked in the past. Recently I noticed that my selected points are off by quite a bit. I spent several hours trying to figure out what’s wrong with my code only to realize that the issue only occurs in upgraded unity editors.

I pulled a commit from my git that was prior to my project upgrade to test my theory and found out that the code works perfectly fine in 2017.2.0f3. So I copied the project folder and did nothing except open it in 2018.2.16f1 and the problem occurs. It seems like the editor canvas window does not start at 0,0 in the bottom left corner. so all of the math to calc where the mouse position is is off by something like 120 pixels on both the x and z.

I wish I could get a screenshot of the effect in both versions, but when I print screen, it doesn’t capture the mouse so you can’t really see how far off it is.

The code below seems to be the standard way to do this. hit.point should be the 3d position after raycasting.

Anybody notice anything similar, or have any ideas?

            SceneView sv = SceneView.lastActiveSceneView;
            Vector3 mousePos = new Vector3(Event.current.mousePosition.x, sv.camera.pixelHeight - Event.current.mousePosition.y);

            RaycastHit hit;

            var ray = sv.camera.ScreenPointToRay(mousePos);

            if (Physics.Raycast(ray, out hit))
            {
2

I created two scripts to demonstrate the issue.

Reproduce:

Start a brand new project in the 2017 version
Add a plane with a mesh collider
Add the following 2 scripts to the new project

using System.Collections;
using System.Collections.Generic;
using UnityEngine;

public class Test : MonoBehaviour {

    // Use this for initialization
    void Start () {
       
    }
   
    // Update is called once per frame
    void Update () {
       
    }
}

And to the editor folder:

using System.Collections;
using System.Collections.Generic;
using UnityEditor;
using UnityEngine;

[CustomEditor(typeof(Test))]
public class TestEditor : Editor
{

    // Use this for initialization
    void Start()
    {

    }

    // Update is called once per frame
    void Update()
    {

    }

    Vector3 pos;
    private void OnSceneGUI()
    {
        SceneView sv = SceneView.lastActiveSceneView;
        Vector3 mousePos = new Vector3(Event.current.mousePosition.x, sv.camera.pixelHeight - Event.current.mousePosition.y);

        RaycastHit hit;

        var ray = sv.camera.ScreenPointToRay(mousePos);

        if (Physics.Raycast(ray, out hit))
        {
            int controlId = GUIUtility.GetControlID(FocusType.Passive);
            HandleUtility.AddDefaultControl(controlId);
            Handles.color = Color.blue;
            Handles.SphereCap(controlId, hit.point, Quaternion.identity, 1);

        }
    }
}

Observe it working properly in 2017 my moving the mouse over the plane. The ball should follow the mouse.

Copy the project and open it in 2018. Observe the misalignment.

3

Posting the solution in case anyone else encounters this error and this thread:

var ray = HandleUtility.GUIPointToWorldRay(Event.current.mousePosition);

1 Like