IPointerEnter/Exit handler works even though layer is to be ignored in collisions

Hi,
I have a prefab where the root has a script S_A and is in a layer A and some of its children are in a layer B, and each of the children has a script S_B attached to it.

Now I’ve implemented the IPointerXXXXHandler interfaces for both S_A and S_B scripts and basically the behaviour I want is, to only get pointer events for S_A and layer A until a close up is done on the object where I switch to getting only events for S_B and layer B.

I’m doing this by using a physics raycaster on the camera and switch the layer to get collisions from using:

raycaster.eventMask = (1 << LayerMask.NameToLayer("A"));
//or
raycaster.eventMask = (1 << LayerMask.NameToLayer("B"));

My problem is, there’s a popup canvas showing only when PointerEnter is achieved on the root object’s S_A script and it still shows when I hit one of the colliders of the children which use the S_B script.

Using VS debugger, both scripts’ PointerEnter functions are entered at the same time even after I’ve switched the raycaster eventMask to just layerB. But PointerClick is correctly disabled for S_A script and the parent object.

Is it a bug or is there some reason this happens?

EDIT: Forgot to mention, that its an early phase so for now I am instantiating the child colliders when the close up happens. But the object is still in layer B so it shouldn’t matter.

Figured it out, sort of, I actually omitted sth previously because I didn’t think it mattered.
The prefab root is in layer A and has the S_A script.
Under it , exists a plain GO named Units which is in the Ignore Raycast layer.
All the children with the S_B script attached with layer set to B are parented to Units object.

For some reason Units being in the Ignore Raycast layer affected things.
Even though the camera physics raycaster only checks collisions with layer B, somehow Units being in Ignore Raycasts affected things ,resulting in the Units go being in the hovered list of the PointerEventData, which fired the PointerEnter for the root.

Setting the Units go to layer A correctly stopped PointerEnter misfiring.

But I still don’t get why sth in a layer Ignore Raycast which by the way is checked off in the Layer Collision Matrix to not collide with layer A and layer B, produced the misfires.

Can someone shed some light?