Allow raycast to go though some World Space UI but not Overlay UI

I’m having some trouble and maybe someone would be able to help…

My scene has some Overlay UI and some World Space UI.
The scene also has some 3D objects I need to interact with mouse clicks.

I have OnPointerEnter and OnPointerExit interactions with some of the World Space UI and because of that, I have them set to Raycast Target = true (otherwise the mouse interactions events won’t trigger.

In order to interact with the 3D objects I need to click on them (and that is working). However, the World Space UI is blocking the clicks on any object behind them.

When my script processes the mouse click from input, it checks for EventSystem.current.IsPointerOverGameObject() to prevent clicking through UI. I know this is what is “blocking” my clicks, but if I remove it, I’m able to click through the Overlay UI too… which I don’t want to.

Is it possible to make the clicks on the 3D objects be blocked by normal UI but not be blocked by World Space UI and allow the World Space UI to still use mouse interaction events (OnEnter, OnExit, OnClick, etc…)?

Additional Info:

  • My scene Main Camera has a Physical Raycaster attached to it, with Event Mask = "Nothing"
  • The canvas where I have the Overlay UI and World Space UI have all Graphic Raycasters
  • I have set my World Canvas UI in a separate layer from the normal UI (layer = “World UI”)
  • My physical raycast is using a layer mask that includes all the layers where I have objects that need to be clicked (and the “World UI” layer is not included in this mask)

Many thanks in advance.

If they’re on separate layers, just check for that too.