Camera jumps on rotating first time

I am rotating my Camera with mouse down and drag. The rotation works properly but sometimes another gameObject rotates my camera at a certain point. When I re-rotate my camera with the below script, the camera jumps to another rotation value. I feel the mouse position does not get updated which is why the camera jumps when it begins with rotation. How do I fix this?

    Ray ray;
    RaycastHit hit;
    public float RotationSensitivity = 35.0f;
    public float minAngle = -45.0f;
    public float maxAngle = 45.0f;
   
     //Rotation Value
    float yRotate = 0.0f;

    void Update()
    {
        ray = Camera.main.ScreenPointToRay(Input.mousePosition);
        if (Input.GetMouseButton(0))
        {
            if (Physics.Raycast(ray, out hit))
            {
                {
                    //Rotate Y view
                    yRotate += Input.GetAxis ("Mouse Y") * RotationSensitivity * Time.deltaTime;
                    yRotate = Mathf.Clamp (yRotate, minAngle, maxAngle);
                    transform.eulerAngles = new Vector3 (yRotate, 0.0f, 0.0f);
                  
                }
            }
        }
    }

It’s not the mouse’s fault. You’re explicitly setting the rotation of the camera using the yRotate value, so when the other script rotates your camera, this script is going to go right back to using the yRotate value it had set earlier when it runs again which is now out of sync with the camera’s rotation.