Input.anyKey still detecting mouse clicks after implementing workaround

I am trying to make sure my game does not start unless a key as been pressed, and am using anyKey for it. I implemented a workaround to make sure that it would only detect key presses, and not mouse button clicks, but it doesn’t seem to work. Any ideas on why this is happening?
using System.Collections;
using System.Collections.Generic;
using UnityEngine;
using UnityEngine.SceneManagement;

public class ChangeScene : MonoBehaviour
{
    private bool soundPlaying;
    public GameObject BackgroundSound;
    public AudioSource start;

    void Start() {
        BackgroundSound.SetActive(true);
    }

    void Update() {
        if (Input.anyKey) {
            if (Input.GetMouseButtonDown(0) || Input.GetMouseButtonDown(1) || Input.GetMouseButtonDown(2)) {
                Debug.Log("done");
                return;
            }
            else
            {
                BackgroundSound.SetActive(false);
                start.Play(0);
                soundPlaying = true;
            }
        }
        if (!start.isPlaying && soundPlaying) {
                SceneManager.LoadScene("HowToPlay");
        }
    }
}

Try using Input.anyKeyDown instead of Input.anyKey. The “Down” activates only on the frame you press it, while without it, it activates on every frame that it is down.

What is happening is that it detects the “anyKey” and then skips the “mouseButtonDown” properly. But then it immediately trips Input.anyKey again in the next frame, but this time “GetMouseButtonDown” is no longer triggered.