The click should not hit the background.

I have created a pause-menu panel that opens when we press the Escape button which pauses the game and panel is displayed. In that panel, I have buttons when I click on the button the background which is my game also responds to the click action.

Once the pause-menu is opened and your UI is shown, make a public boolean value that tells whether or not this menu is opened in the same script. Using an if statement, have the script that handles the clicking feature disable when the boolean value from the other script is true, and re-activate when it is false.