Keep Camera from player at a certain distance while orbiting around player

So far I have everything working perfectly with the following code.
The problem is, using transform.forward while keeping distance sets the camera’s
Y-axis to the same height as the player’s Y-axis. I want the camera to be a bit higher
than the player rather than equal to the player.

The reason I am not simply attaching the camera to the player is because the player is
a sphere that is constantly rotating. The method I am using keep the camera separate from the player while still keeping its distance.
The player’s movement is based on the direction the camera is facing, if that matters at all.

using UnityEngine;
using System.Collections;

public class RotateCamera : MonoBehaviour
{
	public float rotateSpeed = 300.0f;
	public float distance = 3.0f;

	public GameObject player;

	// Use this for initialization
	void Start ()
	{

	}
	
	// Update is called once per frame
	void Update () 
	{
		// If A key is pressed, rotate camera to the left
		// If D key is pressed, rotate camera to the right
		// if S key is pressed, rotate the camera 180 degrees around player
		if (Input.GetKey(KeyCode.A))
		{
			transform.RotateAround(player.transform.position, Vector3.down, rotateSpeed * Time.deltaTime);
		}
		else if (Input.GetKey(KeyCode.D))
		{
			transform.RotateAround(player.transform.position, Vector3.up, rotateSpeed * Time.deltaTime);
		}
		else if (Input.GetKeyDown(KeyCode.S))
		{
			transform.eulerAngles = Vector3.Lerp(transform.eulerAngles, transform.eulerAngles + 180f * Vector3.up, 100 * Time.deltaTime);
		}
	}

	void LateUpdate ()
	{
		// Keep camera a specified distance away from player
		transform.position = player.transform.position - transform.forward * distance;
	}
}

EDIT: basically I want to combine these two lines of code:

transform.position = player.transform.position - transform.forward * distance;

Combined with

transform.position = player.transform.position - transform.up * distance;

okay, so, have you tried to having a gameobject in the player that dosn’t rotate, then putting the camera in that?

you should tilt the camera down at the desired angle before hitting Play button