Screen resolution leads to strange raycast effects

Hi!

I ran into a problem that’s driving me nuts.

Here’s the test case:
Create a new 3D project in Unity and create a plane with…
Position.x = 0.5, Position.y = 0.0, Position.z = 0.5
Rotation.x = 0.0, Rotation.y = 0.0, Rotation.z = 0.0
Scale.x = 0.1, Scale.y = 1.0, Scale.z = 0.1
which leads to a square lying on the xz plane with scale 1x1.

For live debugging information I create a UI panel (500 x 300 pixels) and inside this panel a text field (480x280 pixels, centered inside the panel)

Furthermore I insert a script (code follows) that shall write some raycast information into the textbox.

Set predefined main camera transform data to “something” i.e. to…
position.x = 1.08, position.y = 0.94, position.z=-0.6
Rotation.x = 31.9912, Rotation.y = 328.213, Rotation.z = 10.5431
Scale.x = 1, Scale.y = 1, Scale.z = 1
so you can see the 1x1-field next to our coordinate sytem origin.

Now, when I build and run a windows exe file, I can choose the screen resolution.

My physical screen resolution is 1366 * 768 pixels. If I choose this option and move mouse cursor, the script tells me the data I expect, which means: I get the expected world coordinates.

But if I choose another screen resolution, i.e. 1024 * 768 pixels, I get data that is not corresponding to the data I expect.
Worse: if I move the mouse cursor over the UI elements, the script seems not to find the UI element at the right place (and in the right size).

If you have any suggestions what I am doing wrong, please let me know. Maybe I have to do something with camera settings, but I don’t know. It looks like Unity is using my physical screen resolution for raycast, but my chosen screen resolution for showing UI elements and doing raycasts.

If it’s important to know:
I have Windows 7, 64 bit version
and use Unity Personal Edition x 64, Version 5.1.2f1

Here’s the code of the testing script.

using UnityEngine;
using UnityEngine.UI;
using System.Collections;

public class RaycastAtPlane : MonoBehaviour
{
   // Set textbox in editor to get debugging information.
   public Text textbox;
  
   static Plane ground = new Plane (Vector3.up, Vector3.zero);

   void Start ()
   {
  
   }
  
   void Update ()
   {
     /*
      * First get the point, but only if not inside a UI element.
      */
     if (UnityEngine.EventSystems.EventSystem.current.IsPointerOverGameObject ()) {
       /*
        * mouse is pointing onto an UI element.
        */
       textbox.text = "Pointing on UI element";
       return;
     }

     float enter;
     /*
      * alternate solution: GetComponent<Camera>...
      */
     Camera cam = Camera.main;
     Ray ray = cam.ScreenPointToRay (Input.mousePosition);

     if (ground.Raycast (ray, out enter)) {
       int screenWidth = Screen.width;
       int screenHeight = Screen.height;
       Vector3 p = ray.GetPoint (enter);
       textbox.text =
          "Screen width =  " + screenWidth
         + "\nScreen height = " + screenHeight
         + "\nPlane x = " + p.x
         + "\nPlane y = " + p.y
         + "\nPlane z = " + p.z
         + "\nScreen x = " + Input.mousePosition.x
         + "\nScreen y = " + Input.mousePosition.y
         + "\nScreen z = " + Input.mousePosition.z;
     } else {
       textbox.text = "Not pointing to plane.";
     }
   }
}

Hello! I have the same issue and I was wondering if you were able to solve it

Hi!

Unfortunately I wasn’t. But I did some more tests and now I have a better understanding of what’s wrong.

First I extended the test script so I can see, what pixel width and pixel height the camera “knows”: as expected they are the same as Screen width and Screen height.

So I wrote a method (for tests only!) to get the highest possible of all resolutions (which I suppose might be the physical resolution). For my laptop monitor I got 1366 * 768 (the right values).

To see if screen resolution and ray position differ by highWidth / camWidth respectively highHeight / camHeight I divided the mouse position coordinates for my ray calculations.

The result is that I get the approximately right world coordinates when the ray hits the corners of my test plane. That means I get the mouse position using physical screen resolution, not visual screen resolution without recalculating them according to visual and physical screen resolution.

But this calculation is only for showing the problem. This is NOT the solution. First it does not work inside the editor, because the editor tells me I’d only have a resolution of 640 * 480 (but doing the correct calculations and is using my full screen resolution at all, when not using the factors from above.)

Second (and even worse), it does not solve the “UI clicked” problem: As a further test I created a button at the right lower corner of my test panel and printed some data into a text field when the button is clicked. The result of the test was, that the button coordinates are also in the “wrong” coordinate system, which means I was able to click the button at the physical screen coordinates not at the screen coordinates used for visualization.

The solution must be to tell the “mouse position calculator” to calculate the mouse position based on chosen resolution used for visualization. Whatever component that is, it must be an engine component. So as long as I don’t find a possibility to configure Unity, this looks like an engine bug.

However, if you don’t use any UI elements, the calculation might be an interim solution for you. For me it’s not, because I use the UI system. Forcing the player to use the physical screen resolution could be another (interim) solution for the problem.

I am not deep inside Unity, maybe there is a configuration possibility to correct this behaviour. If anyone knows something about it, please let me know.

Here’s the updated script.

using UnityEngine;
using UnityEngine.UI;
using System.Collections;

public class RaycastAtPlane : MonoBehaviour
{
   // Set textboxes in editor to get debugging information.
   public Text textbox;
   public Text textboxButtonClick;
   public Text textboxResolutionInfo;


   static Plane ground = new Plane (Vector3.up, Vector3.zero);

   void Start ()
   {
   
   }
   
   void Update ()
   {
     /*
      * First get the point, but only if not inside a UI element.
      */
     if (UnityEngine.EventSystems.EventSystem.current.IsPointerOverGameObject ()) {
       /*
        * mouse is pointing onto an UI element.
        */
       textbox.text = "Pointing on UI element";
       return;
     }

     float enter;
     /*
      * alternate solution: GetComponent<Camera>...
      */
     Camera cam = Camera.main;

     Vector3 mousePosition = Input.mousePosition;
     float x = mousePosition.x;
     float y = mousePosition.y;
     float z = mousePosition.z;

     int camWidth = cam.pixelWidth;
     int camHeight = cam.pixelHeight;

     Resolution highestRes = getHighestResolution ();
     int highWidth = highestRes.width;
     int highHeight = highestRes.height;
     
     float widthFactor = (float)highWidth / camWidth;
     float heightFactor = (float)highHeight / camHeight;
     x = x / widthFactor;
     y = y / heightFactor;

     Ray ray = cam.ScreenPointToRay (new Vector3 (x, y, z));

     if (ground.Raycast (ray, out enter)) {
       int screenWidth = Screen.width;
       int screenHeight = Screen.height;

       Vector3 p = ray.GetPoint (enter);
       textbox.text =
          "Screen width =  " + screenWidth
         + "\nScreen height = " + screenHeight
         + "\nCamera width =  " + camWidth
         + "\nCamera height = " + camHeight
         + "\nHighest width =  " + highWidth
         + "\nHighest height = " + highHeight
         + "\nPlane x = " + p.x
         + "\nPlane y = " + p.y
         + "\nPlane z = " + p.z
         + "\nScreen x = " + Input.mousePosition.x + "; adapted = " + x + "; (Factor = " + widthFactor + ")"
         + "\nScreen y = " + Input.mousePosition.y + "; adapted = " + y + "; (Factor = " + heightFactor + ")"
         + "\nScreen z = " + Input.mousePosition.z + "; adapted = " + z;
     } else {
       textbox.text = "Not pointing to plane.";
     }
   }

   public void buttonClicked ()
   {
     if (textboxButtonClick != null) {
       textboxButtonClick.text = "Button clicked.";
     }
   }

   Resolution getHighestResolution ()
   {
     Resolution[] all = Screen.resolutions;
     Resolution ret = all [0];
     string resolutions = "Resolutions";
     foreach (Resolution res in all) {
       resolutions += "\nResolution found: " + res.width + " * " + res.height;
       /*
        * Beware!
        *
        * For productive use it would be far better to compare x and y descrete!
        *
        * Even better would be to get physical resolution from Unity itself.
        */
       if (res.width >= ret.width && res.height >= ret.height) {
         ret = res;
       }
     }
     textboxResolutionInfo.text = resolutions;
     return ret;
   }
}

Has anyone filed a bug report on this one? I have the same problem. I use raycasting to handle mouse events myself so I can use the same GUI for VR and non-VR by doing everything in world space. This works correctly as long as the resolution is set to native, but at other resolutions it doesn’t work anymore.

This is kind of a big deal because anybody that runs my game in a non-native resolution is going to find their UI doesn’t work.

Well, I guess I could just replace the Windows mouse cursor with my VR cursor when in non-VR mode. That’ll probably work, but it still seems like something isn’t really right there with the ScreenPointToRay() function.

For example, I run triple 1920x1200 monitors. If I set my build to run full screen at 1920x1200 (instead of triple that in the x direction), putting the Windows mouse cursor in the center of the left screen puts my virtual cursor in the center of the middle screen. Putting the Windows cursor at the right edge of the left screen puts my virtual cursor at the right edge of the right screen.

It’s all just scaled up three times in the x direction while the y is where it should be. Not sure, but it would seem maybe the ScreenPointToRay() function isn’t taking this into account correctly?

Hi Todd.

I upgraded my test project to 5.2.1 some minutes ago and ran some tests. Using one monitor to show the “game”, the UI seems to work correctly now. The ray calculations don’t. Actually 5.2.3f1 is the sota version, I’m downloading it at the moment to test it later.

You use three monitors next to each other? I only have two monitors, but I cannot tell the “GUITest Configuration” screen to use more than one monitor at a time. I’d like to test your problem, but I don’t know how to use multiple monitors. Is it a pro feature? I only use the personal edition of Unity.

Update:
At Version 5.2.3f1 UI works as expected (1 monitor), raycast doesn’t.

Three monitors, yes. I’m using NVidia’s software that comes with the card which has an extended type of mode that just treats everything as one giant monitor. So if I pick full screen in my build, the game spans across all three in whatever resolution I pick. If it’s in windowed mode you can just drag it wherever you want like any other window, it’s just one giant desktop. The display option when you start the game just shows one monitor, so I guess the NVidia software is just tricking it into thinking it’s one giant monitor. So no, I don’t think this is a Unity or Unity Pro thing, it’s just your video card setup probably.

After looking more at this I’m not so sure that the ScreenPointToRay() function or the raycast itself is really the problem. My monitors’ native resolution would be 5870x1200 total across all three screens, so when I set the build to run at 1920x1200 it stretches that across all three. The mouse input then sees only within that 1920x1200 range because that’s the game resolution. I.e., the left side of the first monitor would be x = 0 and the right side of the “game screen” (all the way over on the third monitor) would be 1920 instead of 5870.

I’m comparing this with the system mouse cursor, so when the system mouse is sitting at at 1920 (the right side of my left/first monitor) I’m then rendering my VR mouse cursor at 1920 also which in this case is on the far right side (third monitor). So I’m not actually sure now why it should be any different.

The real problem here is that I can’t match up the Windows mouse cursor with my own without somehow checking the maximum resolution. Then I could probably just divide the numbers and get my VR mouse to the same spot as the Windows cursor. For now I’ve just hidden the Windows cursor and use my VR cursor instead even if it’s not in VR mode. That seems to work, and it has the sort of neat side effect that if you’re running the game in a window, the system mouse cursor when you’re off the window blends right into the VR cursor when you move it over the game window. Kind of cool, really, although the VR cursor sensitivity then scales in the x direction too.

So for me the problem isn’t really the raycast, that actually makes sense that it works this way so I wouldn’t call it a bug, my case is because the Input class is telling me the system mouse position is at 1920 when I’m really at 5870 as far as Windows is concerned. I’m not really sure what should be done about that though. What I’d like to do is poll maybe a second coordinate to get the actual Windows mouse position instead of the “Unity position at the resolution you have specified” like you’d do in a regular Windows app. Not sure how workable that really is though.

For now I’m just losing the Windows mouse cursor by hiding it and using the VR cursor all the time.