how do i attach an object to the mouse on click

I wish to attach an object to the mouse when the mouse button is down and the object is released when mouse button is up.
How can i do that ?
(I have a version where the object moves after the mouse but i drop the object when i move the mouse too fast)

Transform dragObject;

void Update() {

  if (dragObject) {
    // dragObject not null, move it according to mouse input
  }

  if (Input.GetMouseButtonDown(0)) {
    RaycastHit hitInfo = new RaycastHit();
    if ( Physics.Raycast( Camera.main.ScreenPointToRay( Input.mousePosition ), out hitInfo )) {
      dragObject = hitInfo.collider.transform;
    }
  }

  if (Input.GetMouseButtonUp(0)) {
    dragObject = null;
  }

}