How to make Rigidbody follow touch.position?

I need to make a rigidbody follow a touch position on screen. I have tried many ways but none of them would work as I want to. Since I want to move a rigidbody, I cannot translate its position by just converting screen to world point. Moreover, I cannot MovePosition of the rigidbody because when a ball rests on the rigidbody, the ball goes through it. I managed to move it by using a change in velocity but still, not as expected; the rigidbody moves in the right direction but does not follow the finger but accelerates to the desired direction. This is the current code:

if (Input.touchCount > 0 && Input.touchCount < 3) {
		
		for (var touch:Touch in Input.touches){
			
			if (touch.phase == TouchPhase.Began && playerSlider.HitTest(touch.position)){
				
				touchPosition = camera.main.ScreenToWorldPoint(touch.position);
				fingerXPos = touchPosition.x;				
			}
			if (touch.phase == TouchPhase.Moved && playerSlider.HitTest(touch.position)){
			
				touchPosition = camera.main.ScreenToWorldPoint(touch.position);
				
				if (playerRigidbody.position.x > 415 && (touchPosition.x - fingerXPos) < 0){
					playerRigidbody.velocity = (new Vector3(1, 0, 0) * 1000 * (touchPosition.x - fingerXPos)/2 * Time.fixedDeltaTime);
				}
				else if (playerRigidbody.position.x < 850 && (touchPosition.x - fingerXPos) > 0){
					playerRigidbody.velocity = (new Vector3(1, 0, 0) * 1000 * (touchPosition.x - fingerXPos)/2 * Time.fixedDeltaTime);
				}
				else{
					playerRigidbody.velocity = (new Vector3(0, 0 , 0) * Time.fixedDeltaTime);
				}
			}
			else playerRigidbody.velocity = (new Vector3(0, 0, 0) * Time.fixedDeltaTime);
		}
	}
	else playerRigidbody.velocity = (new Vector3(0, 0, 0) * Time.fixedDeltaTime);

do you want to use realistic physics or just have collision detection?

the character controller can be used to obtain non phyics collision detection.

you may also use what you have which is accelerating the object but when it gets close to the finger set the velocity to very small
when it gets very very close set it to zero.

it’s not realistic but you dont appear to want realism (the object basically would be allowed to move at ludicrus speed across the screen and stop in basically no time)