mousePosition is off after unlocking the cursor from locked mode

After unlocking my cursor from lockMode.locked it kinda shifts and the position don’t match with the actual cursor position.

If I want to click on a button, I need to move the cursor up to another button and click, and in the end the button below is clicked. The event manager also shows exactly that game object that the “real” cursor is not hovering over.

Any ideas how to fix this?

Here is the code that in Update:

void Update()
{
    if (Input.GetKeyDown(KeyCode.U))
    {
        isOpen = !isOpen;
        debugMenu.SetActive(isOpen);
    }

    if(!isOpen)
    {
        Cursor.lockState = CursorLockMode.Locked;
        Cursor.visible = false;
    }
    else
    {
        Cursor.lockState = CursorLockMode.Confined;
        Cursor.visible = true;
    }
}

Just an idea CursorLockMode.Confined depend of the game window, maybe your code is working but the Game window size have an impact on the mouse position.
Try to change the aspect of the Game window or the set his resolution.

Also use CursorLockMode.None could solve this issue if it fit your needs.