Move an object by changing it's velocity

I’m trying to get the object from one side of the screen to the other. I use 2 vector3 points and subtract them. That gives me the velocity to the object that should make the object move. However tho object doesn’t move enough. If let’s say I move the mouse 100 units the object only moves 80 units. I’m going something wrong but I don’t know what it is.

Here is my code to move the object:

    Rigidbody rigidbody;
    // Use this for initialization
    void Start()
    {
        rigidbody = GetComponent<Rigidbody>();
    }

    public Vector3 gameObjectSreenPoint;
    public Vector3 mousePreviousLocation;
    public Vector3 mouseCurLocation;
    void OnMouseDown()
    {
        //This grabs the position of the object in the world and turns it into the postion on the screen
        gameObjectSreenPoint = Camera.main.WorldToScreenPoint(gameObject.transform.position);
        //Sets the mouse pointers vector3
        mousePreviousLocation = new Vector3(Input.mousePosition.x, Input.mousePosition.y, gameObjectSreenPoint.z);
    }

    public Vector3 force;
    public Vector3 objectCurrentPosition;
    public Vector3 objectTargetPosition;
    public float topSpeed = 20;
    void OnMouseDrag()
    {
        mouseCurLocation = new Vector3(Input.mousePosition.x, Input.mousePosition.y, gameObjectSreenPoint.z);
        force = mouseCurLocation - mousePreviousLocation;//Changes the force to be applied
        mousePreviousLocation = mouseCurLocation;
        multiplier++;
    }

    public void OnMouseUp()
    {
        //Makes sure there isn't a ludicrous speed
        if (rigidbody.velocity.magnitude > topSpeed)
            force = rigidbody.velocity.normalized * topSpeed;
    }

    public void FixedUpdate()
    {
        rigidbody.velocity = force;
    }

I’m working on an Astroids clone for mobile and I’m trying to get the spaceship to move with your mouse drag precisely. I’m also trying to get it to glide through space when you let go of the ship. Which right now work pretty well.

If you want the user to have -precise- control, I think you’re probably better off not using physics to calculate movement, and instead writing your own, precise, non-physics-based movement code.