GameObject with OnMouseDown method not blocked by UI?

So that’s most of the question - The UI panel is open screen-wide, but the (unrelated) GameObject with OnMouseDown method and a collider is still clickable.

How do I disable this clickability, without altering the Raycast layer and adding more logic to the code?