I am not 100% sure about that, but changing values inside the Inspector are updated inside the Update method.
But your camera movement should be executed inside the LateUpdate methode after all Update logic is finished. The reason is that we want to render the scene after everything stopped moving. Thats why its jittering, because the other values are still changing after your camera moved to its destination.
Try, instead moving by inspector, using a script. You could use a simple script which allows you to control your camera by your cursor. There should be enough videos about that.