Drag an object which camera follows

When you drag an object(Drag object relative to camera - Unity Answers) , while the program lets camera follow the object (by making the camera child object of the object or a script that follows the transform of the object), the dragged object moves in abruptly quickly.
Is there any methods which allow the dragged objects is followed by the camera in normal way?
I tried to limit maximum movement of the camera by using various ways such as Mathf.MoveTowards etc… but no success.
Thanks

For the dragged Object (Could be more sofisticated though):

Vector3 initialPos;
Vector3 curPos;
float speed = 3f;

void Update () {
	if(Input.GetMouseButton(0)){
		curPos = Input.mousePosition;
	}
}

void OnMouseDown() {
	initialPos = Input.mousePosition;
}

void OnMouseDrag() {
	if(curPos.x != initialPos.x ) 
	{
		transform.position = new Vector3(transform.position.x +(curPos.x - initialPos.x) * Time.deltaTime * speed,  transform.position.y, transform.position.z);
	}
}

For the camera:
public Transform target;
public float smoothTime = 0.3F;
private Vector3 velocity = Vector3.zero;

void Start () {
	transform.position = new Vector3 (target.transform.position.x,
	                                  target.transform.position.y, 
	                                  transform.position.z);
}

void Update () {
	transform.position = Vector3.SmoothDamp(transform.position, target.position, ref velocity, smoothTime);
	transform.position = new Vector3(transform.position.x, the y position that it should stay as this is horizontal scrolling, -10f);
}

Instead of moving the object directly with the finger drag, move an invisible mock object instead and then make the object follow that mock object’s position using SmoothDamp, or lerp, or anything similar. When adding smoothness to movement (either to a camera or any other object that doesn’t need to be time controlled) I prefer using the SmoothDamp option.

[Edit] Unless the object has a speed parameter (i.e. time controlled). Then use lerp instead, it’s easier to tweak.