Yeah, it’s a Unity thing.
You can circumvent it by making a custom CM input handler that suppresses input when a certain key is pressed, or something like that. This post tells you how: How do I make a CinemachineFreeLook orbiting camera that only orbits when the mouse key is down?