Camera stuff is pretty tricky… I hear all the Kool Kids are using Cinemachine from the Unity Package Manager.
There’s even a dedicated forum: Unity Engine - Unity Discussions
If you insist on making your own camera controller, do not fiddle with camera rotation.
The simplest way to do it is to think in terms of two Vector3 points in space:
- where the camera is LOCATED
- what the camera is LOOKING at
private Vector3 WhereMyCameraIsLocated;
private Vector3 WhatMyCameraIsLookingAt;
void LateUpdate()
{
cam.transform.position = WhereMyCameraIsLocated;
cam.transform.LookAt( WhatMyCameraIsLookingAt);
}
Then you just need to update the above two points based on your GameObjects, no need to fiddle with rotations. As long as you move those positions smoothly, the camera will be nice and smooth as well, both positionally and rotationally.