How to make small objects that are affected by physics, but don't hinder other things

How to make small objects that are affected by physics (and collision), but don’t hinder other things ?
Ideally it just doesn’t trigger on CollisionStay() and the like, on the other object it hits, and acts as if that other object is infinitely heavier (so it doesn’t hinder it’s motion/trajectory)

I have explosions instantiate debris, but they are often in the way of other objects, and cause rockets and mines to explode, and they work terribly with the character controller.
So I want to fix that, but can’t think of a good way to do this.

So any advice ?

Physics.IgnoreCollision ? or a particle effect

OK bear with me here, sorry if this is not a great thought:

say you create an empty gameObject, give it a collider, child it to the player, then set it to a unique collision layer that only sees the debris, on the same layer. Maybe then it wont affect the player physics? A collision layer would definitely solve the mine issue, at any rate…