2D collider mouse events not detected?

I'm using an overlay canvas and I want to use mouse events on 2d colliders.
I added a physics 2D raycaster to my camera but I can't seem to detect mouse events.

The only way I can get it to work is if the camera's frustum intersects with the actual canvas window in the 3d space, but then my camera is way out of place ofcourse.

If I use a graphic raycaster it does work on images tho.
Is it possible to get the same thing but with 2D colliders?

This seems like a common use case

Just shooting in the dark here;
What if you had an invisible object with 2d collider follow mouse position and use that?

I'm not sure what you mean, but what I want to do is have a circle collider catch mouse enter events.

The problem with the graphic raycaster is that everything is a square, the image can be whatever shape but it uses the encapsulated border for the mouse events.