How to change Force direction immediately?

Hi, I am trying to add a force on rigidBody by using a vector I am getting from the user’s mouse position: up will change the vector to be:

vector+=Vector3.up  * MOVEMENT_SPEED;

And the same thing with left,right,down,forward and backward.

The problem is that there is a small delay every time I change the position. For example, if I go up and then want to go down,after moving the mouse down it will first go up for a bit and only then down.

I am using:

void FixedUpdate()
{
	rb.AddForce(K * vector);//K=Const
}    

Is there a way to change the force direction immediately when I change the vector’s position?

Try setting the velocity directly Unity - Scripting API: Rigidbody.velocity