Prevent game object from getting selected on click in editor window

Hi there !
Is there any solution to prevent my “background” sprite from getting selected every time I try to select another sprite ? This is probably trivial but this simple thing is driving me crazy !

I have to go trough the hierarchy panel to be able to select the gameobject that I want, ALL THE TIME ! ^^