Rotating camera with quaternions; messing up if camera pitch is > ~+/- 75 degrees

Hi everyone!
We’ve contructed a rigidbody that works as a player character. The idea is then to have him skiing around our map tribes: ascend style. Currently we’ve run into an issue though which we haven’t been able to find a solution for.

We’ve got a sphere controlling the movement of the player and a camera inheriting the sphere’s position through a script. The Sphere is supposed to align itself with the ground underneath it, however, still inherit the local y-axis of the camera; meaning that the sphere’s y-axis should smoothly align itself with the normal vector of the surface it’s standing on while looking in the direction of the camera.

camera y-rotation → sphere, sphere position → camera

All this is fine and dandy and it works… until you put the camera into a high or low pitch situation. This will cause the camera to spin intensely whenever the sphere is rotating to align itself with the ground.

We’re currently using the following script to do this:

void Update () {
		if (!player.IsSkiing) {
			//align smoothly
			playerNormVector = Vector3.Lerp(playerNormVector, Vector3.up, cameraRotationSpeed*Time.deltaTime);
			projectedForward = playerCamera.transform.forward - (Vector3.Dot (playerCamera.transform.forward, playerNormVector)) *playerNormVector;
			transform.rotation = Quaternion.LookRotation(projectedForward, playerNormVector) *initialRotation;
		}
		else {
			RaycastHit hit;
			Vector3 rayDir;
			float rayDist;
			
			if (player.IsSkiing) {
				//set propper direction and length of the ray cast
				if (player.IsGrounded) {
					rayDir = -playerNormVector;
					rayDist = rayCastDistanceNorm;
				} else {
					rayDir = Vector3.down;
					rayDist = rayCastDistanceDown;
				}
				
				if (Physics.Raycast(transform.position, rayDir, out hit, rayDist)){
			    	//do the ray casts, either along invers norm or just down
					playerNormVector = Vector3.Lerp(playerNormVector, hit.normal, cameraRotationSpeed*Time.deltaTime);
					projectedForward = playerCamera.transform.forward - (Vector3.Dot (playerCamera.transform.forward, playerNormVector)) *playerNormVector;
					transform.rotation = Quaternion.LookRotation(projectedForward, playerNormVector) *initialRotation;
				} else {
					//if nothing hits, then orient the player along the last know axis
					projectedForward = playerCamera.transform.forward - (Vector3.Dot (playerCamera.transform.forward, playerNormVector)) *playerNormVector;
					transform.rotation = Quaternion.LookRotation(projectedForward, playerNormVector) *initialRotation;
				}
			}
		}
        //DEBUG STUFF
		Debug.DrawLine(transform.position, transform.position +rayCastDistanceNorm *playerNormVector, Color.red);
		Debug.DrawLine(transform.position, transform.position +rayCastDistanceDown *Vector3.down, Color.blue);
		Debug.DrawLine(transform.position,transform.position +projectedForward, Color.green);
	}

Any help would be greatly appreciated.

Found a solution!
I still have no idea what causes this, but our new setup has the same effect but does not cause the spinning.

Where we before had the camera position locked to the rotation of the sphere, we changed it so that the sphere only rotates itself, but the script attached to the camera handles the camera positioning above the normal vector of the sphere.

Basically doing the exact same thing, however, this does not seem to fuck up the camera… for some reason…