Is Mouse.current.WarpCursorPosition(...); working in Linux?
Not working here within editor, Ubuntu 22.04 and Unity 2023.3.b2
Doc says, it should work on all platforms.
edit: Turns out it does work but only for X11 and not Wayland.
Please fix 
So this is something Unity probably isn’t able to fix on their own.
As this has brought up severe concerns for backwards compatibility the only place to fix this is for Wayland itself.
Relevant bug reports:
Bonus question: Why is Cursor.lockState able to lock the mouse cursor to the center screen with X11 or Wayland but WarpPosition can’t?
Cursor locking doesn’t have to be implemented using WarpPosition. Having a cursor locked or confined to a window are common operations so they have a dedicated API.
1 Like
You say common but that’s exactly the question. Whatever it is using, it still has to make a call to the window manager that is running. X11 or Wayland.
With Cursor.lockState this call is successful and with WarpPosition it isn’t. Both are setting a position, so why do they differ?
Setting mouse position and confining or freezing it is not the same.
Cursor.lockState only needs to confine the cursor so that you can’t click outside the window. As sideeffect of having the cursor being locked after it’s unlocked it might reappear somewhere else. But it doesn’t mean the API is suitable for freely setting the position.
If you take a look at wayland documentation for cursor freezing/confining protocol https://wayland.app/protocols/pointer-constraints-unstable-v1 it provides optional hint where to restore the cursor after unfreezing. But that’s only a hint and the desktop environment is in control whether to use it and when to use it. Could you emulate cursor warping by abusing the restore position hint - probably. Would it work robustly and reliably - most likely not.
Unity is probably not communicating with wayland directly. There were some talks on them working on native wayland support, but I don’t think that’s available unless you are using latest experimental unity version. LTS like 2021 is still using X11 api and xwayland. Which exact API Unity is using to implement the cursor lock mode I don’t know. You will need to pay Unity undisclosed amount for the source code access, or reverse engineer Unity to be sure. But since cursor lock mode works, but the warpCursor doesn’t I am quite confident that X11 provides some kind of API which allows constraining cursors without arbitrary setting it’s position and thus supported by xwayland.
Here is one test project I did in Unity.
using System.Collections;
using System.Collections.Generic;
using UnityEngine;
using UnityEngine.InputSystem;
public class CursorTest : MonoBehaviour
{
// Start is called before the first frame update
bool lockCursor = false;
bool warpCursor = false;
public GameObject lockedIndicator;
public RectTransform t1;
public RectTransform t2;
public UnityEngine.UI.Text text;
void Start()
{
}
// Update is called once per frame
void Update()
{
if (Input.GetKeyDown(KeyCode.A))
{
lockCursor = !lockCursor;
Cursor.lockState = lockCursor ? CursorLockMode.Locked : CursorLockMode.None;
lockedIndicator.SetActive(lockCursor);
}
if (Input.GetKeyDown(KeyCode.S))
{
warpCursor = !warpCursor;
}
if (warpCursor)
{
Mouse.current.WarpCursorPosition(new Vector2(Screen.width / 2, Screen.height / 2));
}
t1.position = Mouse.current.position.ReadValue();
var scr = new Vector3(Screen.width * 0.5f, Screen.height * 0.5f, 0);
var mouseRel = (Vector3)Mouse.current.delta.ReadValue();
t2.position = scr + mouseRel;
string tx = $"pos: {t1.position.x} {t1.position.y} | delta: x:{mouseRel.x} y:{mouseRel.y}";
if (lockCursor)
{
tx += " | lock";
}
if (warpCursor)
{
tx += " | warp";
}
text.text = tx;
}
}
Which allows you to play with the cursor lock and cursor warp functionality. What I observed was that:
- on Wayland while the cursor lock was active the reported absolute position was fixed roughly in the center of window and after disabling it cursor was in the center. That doesn’t necesarily mean unity can freely change the position. As expected warping doesn’t work.
- On x11 locked mode behavior was similar. Enabling warping demonstrated that simulating cursor lock by each frame warping it to center of window isn’t a good solution, if you move the mouse slowly it’s mostly fine. But if you move it quickly it’s posible to move and click outside the window, which is undesirable for a game.
- On windows results were mostly simular. One interesting difference was that on Windows in the cursor locked mode, the marker t1 slightly moved in the direction I was moving the mouse by a few pixels. The movement was significantly smaller than when simulating lock by warping, but it was there.
One thing you can easily look at is how open source libraries handle this situation. SDL https://www.libsdl.org/ is a commonly used crossplatform library used for stuff like window creation and input. From the Unity changelogs it seems like under the hood Unity is also using SDL for some stuff on Linux although it’s unclear how much and in which situations. Since the observed behavior is different I guess this is one of the situations where they have Unity has their own implementation. Anyway SDL is a good example how these functions could be implemented using the APIs provided by system.
#include <SDL.h>
#include <SDL_events.h>
#include <SDL_keycode.h>
#include <SDL_mouse.h>
#include <SDL_stdinc.h>
#include <format>
#include <iostream>
int main() {
//SDL_HINT_MOUSE_RELATIVE_MODE_WARP
SDL_Window *SDLWindow{nullptr};
SDL_Surface *SDLWindowSurface{nullptr};
SDL_Init(SDL_INIT_VIDEO);
SDLWindow = SDL_CreateWindow("Hello Window", 0, 0, 500, 300, 0);
SDLWindowSurface = SDL_GetWindowSurface(SDLWindow);
SDL_FillRect(SDLWindowSurface, nullptr,
SDL_MapRGB(SDLWindowSurface->format, 40, 40, 40));
SDL_Event event;
bool exit = false;
bool lock = false;
bool warp = false;
SDL_MouseMotionEvent mouseEvent;
while (!exit) {
bool hadMouse = false;
while (SDL_PollEvent(&event)) {
switch (event.type) {
case SDL_QUIT:
exit = true;
break;
case SDL_KEYDOWN:
switch (event.key.keysym.sym) {
case SDLK_a:
lock = !lock;
SDL_SetRelativeMouseMode(lock ? SDL_TRUE : SDL_FALSE);
break;
}
break;
case SDL_MOUSEMOTION:
hadMouse = true;
mouseEvent = event.motion;
break;
}
}
std::cout << std::format("mouse: {} {} | rel: {} {} | lock:{} warp:{}",
mouseEvent.x, mouseEvent.y, mouseEvent.xrel,
mouseEvent.yrel, lock, warp)
<< std::endl;
SDL_UpdateWindowSurface(SDLWindow);
}
SDL_Quit();
}
With SDL behavior is a bit different. When relative cursor mode is activated both on X11 and Wayland SDL hides the cursor and confines it to window. Reported mouse position changed within the dimensions of window, but the relative movement was correct even if you pushed it against corner. But you aren’t really supposed to look at the absolute mouse position while the relative mode is active and you can’t interact with anthing anyway, so it doesn’t matter too much whether cursor is fully frozen or whether it can move within bounds of windows as long as relative movement is correct.
If you look at the SDL source code for Wayland support you will find that the code matches with practical observation. WarpMouse is for the most part unsupported https://github.com/libsdl-org/SDL/blob/f7f9478caf2fb61fabfc9b31bd6a0b3abc56a791/src/video/wayland/SDL_waylandmouse.c#L545 .
And SetRelative locks the cursor https://github.com/libsdl-org/SDL/blob/f7f9478caf2fb61fabfc9b31bd6a0b3abc56a791/src/video/wayland/SDL_waylandmouse.c#L577 and if you dig deeper you will find that it uses the methods from the paged I linked before https://wayland.app/protocols/pointer-constraints-unstable-v1#zwp_pointer_constraints_v1:request:lock_pointer:arg:region
Looking at X11 implementation I am not quite sure how it works. X11_SetRelativeMouseMode either returns an error or does nothing claiming to have done the thing. The observed behavior is similar to as on Wayland, but the code for doing it isn’t there https://github.com/libsdl-org/SDL/blob/75dd7e1658f8e0b2fbf90e85949b559ee84c172b/src/video/x11/SDL_x11mouse.c#L388 Maybe it’s happening somewhere else based on a flag.
Seems like SDL does have a fallback mode https://github.com/libsdl-org/SDL/blob/f7f9478caf2fb61fabfc9b31bd6a0b3abc56a791/src/events/SDL_mouse.c#L1139 which implements cursor locking by warping it to center of screen, but it should only activate on platforms that don’t have native cursor locking support. In theory there is flag to force it, but I failed to activate it and test how well it works compared to the naive version I made in Unity. Maybe it works slightly better if lower level engine code does it at optimal moment, compared to doing it somewhere in Update.
3 Likes
Excellent post! Thanks!
FWIW we use gdk_display_warp_pointer in the Linux Editor and SDL_WarpMouseInWindow in the Linux player (which ends up in XWarpPointer).
1 Like
Been some time. I gave this another go. WarpPosition is still broken in the editor. When you read Mouse.current.Position and set the same position via Warp, it ends up somewhere else. Some editor offset isn’t applied it seems.
But what’s worse. WarpPosition screws with the input system. When you read cameraActions.CameraAxis.ReadValue<Vector2>() and then use WarpPosition the values are wrong.
See, these are the values when there’s a constant left motion with the mouse. When the cursor position is set in between with
WarpPosition there’s an unexpected value in the different direction resetting the whole delta to 0 again.
Now to see if there’s something fundamentally wrong I’ve implemented XWarpPointer and XQueryPointer. Works perfectly.
None of this happens in Windows and TBH I’ve not the slightest clue what’s going on here. There seems to be an order problem and mismatch of the internal mouse position that’s used for the delta and the actual value. But I can’t explain why that is not an issue in Windows.
So, pretty unhappy about the state. Broken on 2 fronts. 
I have a pretty similar issue. In the editor view, if I right click and try to do a 360° with the view, it works perfectly fine with X11, but with Wayland, the cursor gets stuck at the screen borders, if I try to do it vertically, it becomes a complete mess. Not sure if this is related, so I am just giving this feedback as well.