Make collider behave like a trigger without isTrigger

I know this is a ridiculous question, but how can I make my player and all rigid bodies pass through a collider without using isTrigger? Basically I want everything to pass through my collider except particles

Can you just use layering? Otherwise, there is Physics.IgnoreCollision()