Rotating object to face vector

I’m trying to rotate an object to align with a vector and when I originally searched, I seemed to find something that worked in one instance but it doesn’t work in another. So I’m hoping to get some help getting this to work with my other case.

I have 2 points that are moving away from each other and want to align my object so it faces along the line they form. When I spawn the object, I use this code to get its starting rotation, but if I continue to update the 2 points’s positions and rerun the same code, the object just spins in place very fast. What I’m using is

Vector3 tarVec = point2 - point1;
Quaternion facing = gameObject.transform.rotation;
gameObject.transform.rotation = Quaternion.LookRotation(Vector3.RotateTowards(gameObject.transform.forward, tarVec.normalized, float.MaxValue, float.MaxValue)) * facing;

This works exactly as I hoped when I first spawn my object, but as I modify point1 and point2 to move away from each other, the gameObject just spins like crazy.

Also, if I use OnDrawGizmos to draw a line between point1 and point2, I can see that they are moving apart exactly as I expect. So it’s not like these points are going haywire.

For the rotation i did this:

public Transform objOne;		// Point A
public Transform objTwo;		// Point B
public Transform centerObj;		// To visualize the center point

private Vector3 betweenPos;		// The position, between A and B
private Vector3 direction;		// Direction from look object to the middle point

public void Update()
{
	betweenPos = ((objTwo.position - objOne.position) * 0.5f) + objOne.position;	// Get mid point between objOne and objTwo
	direction = transform.position - betweenPos;									// Get direction, from curObj, to the point that it shoud look at.

	centerObj.position = betweenPos;				// Just a temporary object, to visualize the center, between the 2 points

	transform.parent.rotation = Quaternion.LookRotation(-direction, Vector3.up);	// Orient the parent of this Obj, to look at the middle point
}

For the orientation i’ve just parented the looking object into one empty gameObject and i’m orienting the parent. This way i can orient the child in the direction, that i want and avoid having more code for custom orientation…