One way ignoring collision

I use Physics.IgnoreCollision to ignore collision, but what if I want A to not ignore B but B to ignore A? (like clothes is Unity 5)

You must find a way to clarify the object wich gave the intention to collide. Something like: check wich of booth objects had more speed or wich has been moved by the player or so one.

Mass to System.Single 1e-7f.