Have a gameObject follow the mouse in sync. So that the mouse pointer is always over the object. Even as the camera is rotated.


So a couple weeks ago I asked a similar question here : http://answers.unity3d.com/questions/10268/have-a-gameobject-follow-the-mouse-in-sync-so-that-the-mouse-pointer-is-always-o

Today, I realized the solution to the problem wasn't so straightforward.

What I want is to be able to drag a cube around with the mouse, in a way that syncs perfectly with the mouse. The mouse cursor should always remain over the object, and visually, the object should move as much as the mouse did. This should work no matter what the rotation angle of the camera is. The cube should also be exactly under the point where we clicked, it shouldn't snap to its center.

Here is what I have that works, when the camera is looking perfectly perpendicular to the Z axis, but breaks when the camera is rotated or moved around at different angle:

    private Vector2 snapOffset;

void OnMouseDown()
    Vector3 temp = Camera.mainCamera.WorldToScreenPoint(transform.position);

    snapOffset = new Vector2(Input.mousePosition.x - temp.x, Input.mousePosition.y - temp.y);
    Debug.Log("snap offset: " + snapOffset);

void OnMouseDrag()
    Vector3 mousePosition = new Vector3(Input.mousePosition.x - snapOffset.x, Input.mousePosition.y - snapOffset.y, 0);
    mousePosition.z = transform.position.z - Camera.mainCamera.transform.position.z;

    Debug.Log("mousePosition: " + mousePosition);

    transform.position = Camera.mainCamera.ScreenToWorldPoint(mousePosition);

This code works, if the camera is pointing straight at the cube from in front. If you rotate the camera, say 45 degrees, it will stop working.

I want a solution that does not use Ray-casting.

Thank you.

I think your question could use come clarification as to how you are moving your camera, but I think I get your meaning.

Have you considered something like:

Vector3 screenPosition;

//Store the screen position of the object when the mouse clicks
void OnMouseDown()
    screenPosition = Camera.mainCamera.WorldToScreenPoint(transform.position);

//When the mouse drags, change the object's screen position accordingly.
void OnMouseDrag()
    //Read the mouse input axes
    screenPosition.x += Input.GetAxis("Mouse X");
    screenPosition.y += Input.GetAxis("Mouse Y");

    //move the object to the world position to not change screen position
    transform.position = Camera.mainCamera.ScreenToWorldPoint(screenPosition);

This way the object is not centred to the mouse, but will keep its relative position on screen to where the mouse was clicked.

If the camera rotates in this script, then you can easily check if the mouse is held down and call transform.position = Camera.mainCamera.ScreenToWorldPoint(screenPosition); when the camera rotates

If the camera rotates in a separate script (which it likely does), I suggest making the relevant data public to the camera rotation so that it can update the object's transform if applicable (the mouse is still down). The exposed data could be the selected object and its stored screen position or a public function like updatePosition which will make the call to set the transform and then your rotate function and mousedrag can both call this.

Try using something like `Camera.ScreenPointToRay`.

So i.e.

Ray ray = Camera.main.ScreenPointToRay( Input.mousePosition );
transform.position = ray.GetPoint( distanceFromScreen );

Where distanceFromScreen is a fixed distance from the screen.

You'll have to use raycasting if you want to make sure your object doesn't go "behind" something, though. So you'd cast a ray, do a trace, and put your object's position at either what I had before, or the hit.point, whichever is closer.