Rotating object to face vector

I’m trying to rotate an object to align with a vector and when I originally searched, I seemed to find something that worked in one instance but it doesn’t work in another. So I’m hoping to get some help getting this to work with my other case.

I have 2 points that are moving away from each other and want to align my object so it faces along the line they form. When I spawn the object, I use this code to get its starting rotation, but if I continue to update the 2 points’s positions and rerun the same code, the object just spins in place very fast. What I’m using is

Vector3 tarVec = point2 - point1;
Quaternion facing = gameObject.transform.rotation;
gameObject.transform.rotation = Quaternion.LookRotation(Vector3.RotateTowards(gameObject.transform.forward, tarVec.normalized, float.MaxValue, float.MaxValue)) * facing;

This works exactly as I hoped when I first spawn my object, but as I modify point1 and point2 to move away from each other, the gameObject just spins like crazy.

Also, if I use OnDrawGizmos to draw a line between point1 and point2, I can see that they are moving apart exactly as I expect. So it’s not like these points are going haywire.

For the rotation i did this:

public Transform objOne;		// Point A
public Transform objTwo;		// Point B
public Transform centerObj;		// To visualize the center point

private Vector3 betweenPos;		// The position, between A and B
private Vector3 direction;		// Direction from look object to the middle point

public void Update()
	betweenPos = ((objTwo.position - objOne.position) * 0.5f) + objOne.position;	// Get mid point between objOne and objTwo
	direction = transform.position - betweenPos;									// Get direction, from curObj, to the point that it shoud look at.

	centerObj.position = betweenPos;				// Just a temporary object, to visualize the center, between the 2 points

	transform.parent.rotation = Quaternion.LookRotation(-direction, Vector3.up);	// Orient the parent of this Obj, to look at the middle point

For the orientation i’ve just parented the looking object into one empty gameObject and i’m orienting the parent. This way i can orient the child in the direction, that i want and avoid having more code for custom orientation…