Mouse movement shakes camera.

Before you downvote this, yes, I have looked on all google answers, Vector Lerping is not the answer.
And before you say it, float imprecision isn’t it either.

I am making a solar system game but for some reason, the player movement isn’t working well. I have the physics set up and the planets rotate around the sun, but for some reason, I cannot get the mouse movement working.

Currently, this is what I have.

yaw += Input.GetAxisRaw ("Mouse X") * 100 / 10 * mouseSensitivityMultiplier;
		pitch -= Input.GetAxisRaw ("Mouse Y") * 100 / 10 * mouseSensitivityMultiplier;
		pitch = Mathf.Clamp (pitch, pitchMinMax.x, pitchMinMax.y);
		float mouseSmoothTime = Mathf.Lerp (0.01f, maxMouseSmoothTime, 0.2f);
		smoothPitch = Mathf.SmoothDampAngle (smoothPitch, pitch, ref pitchSmoothV, mouseSmoothTime);
		float smoothYawOld = smoothYaw;
		smoothYaw = Mathf.SmoothDampAngle (smoothYaw, yaw, ref yawSmoothV, mouseSmoothTime);
		if (!debug_playerFrozen && Time.timeScale > 0) {
			cam.transform.localEulerAngles = Vector3.right * pitch;
			transform.Rotate (Vector3.up * Mathf.DeltaAngle (smoothYawOld, smoothYaw));

I have maxMouseSmoothTime set at 0, (the shaking lasted longer when turned up)
This is what it looks like:

I have no idea why it is not working the way you want, maybe you should try replacing GetAxisRaw by GetAxis ?
I wonder why you multiply your yaw value per 100 before dividing it by 10. Wouldn’t it be simpler to multiply per 10 ?

Solution, and I am not sure why it is, but move the function above into FixedUpdate.