IgnoreLayerCollision but then allow collision for some objects with that layer

Hey, I made a telekinesis action where you can grab objects using telekinesis. I made a kinematic collider on my player that should ignore litterally everything in the scene (so all layers), but when I grab an object, I want that collider to be able to collide with that object.

The point is to have my player be able to collide with the objects when he's not grabbing them, and have some "shield" preventing the grabbed objects to push him

I tried ignoring all layers collision in my player's Start() and then in the grab method, I allow collision with that object, but it doesn't work: the collision is not happening.

Anyone knows why ?

PS: At first I tried reducing a lot the mass of the grabbed object, and that fixed the "pushing the player" issue, but then they were unable to push the other objects (because their mass was to small) and so I dropped the idea)

I'm not sure why you're experiencing a problem, but an alternative approach could be to use contact modification to selectively ingore/ not ignore collisions, like these balls that ignore the collision going in, but pay attention on the way out:

So I've really tried understanding this but it feels really difficult to grasp.
For example, I'd like to simply do the following :

On collision, if the other collider has a certain script attached, then I want to ignore contacts (so that they effectively just ignore each other collision). I couldn't find a way to get this information. If I use the "OnContactModify" way, I get a "ModifiableContactPair" which holds stuff like colliderInstanceId and otherColliderInstanceId, which I don't know how to use to get said collider to make my condition of "has this script attached".

If I try directly writing code in the OnCollisionEnter method, then I have no obvious way to have access to the ModifiableContactPairs, so I can't do things like SetTargetVelocity or IgnoreContact.

Also, I couldn't find an explanation on how "pairs" work. Like, how would I set the velocity of only one of the contact points ? In a case for example where I have 2 rigidbodies both with a collider, and I'd want one of them to be unable to move the other ?

In my case, I don't want the kinesis object to be able to push the player. But I do want it to be able to interact normally (and thus push) other objects in the scene.

Modifying contacts feels super powerful, but I feel like I miss one or two pieces of information to make it work.

I guess you can't be doing this:
I feel like hopefully there's a quicker/ easier answer than contact modification tbh.
Or, can you move the kinesis object to another layer temporarily while the player is doing stuff with it?

I'm using contact modification kindof like this (this is hacked out from working code, may not quite work as-is):

//You don't need it to be a singleton if you make sure there's only one of it:)
public class MyContactModification : Singleton<WGContactModification> {
    public HashSet<int> TargetColliderInstanceIDs = new ();
    public HashSet<int> PlayerColliderInstanceIDs = new ();
    private void OnEnable() {
        Physics.ContactModifyEvent += ModificationEventDCD;
        Physics.ContactModifyEventCCD += ModificationEventCCD;

    private void OnDisable() {
        Physics.ContactModifyEvent -= ModificationEventDCD;
        Physics.ContactModifyEventCCD += ModificationEventCCD;
    private void ModificationEventDCD( PhysicsScene scene, NativeArray<ModifiableContactPair> pairs ) {
        ModificationEvent( pairs );
    private void ModificationEventCCD( PhysicsScene scene, NativeArray<ModifiableContactPair> pairs ) {
        ModificationEvent( pairs );
    private void ModificationEvent( NativeArray<ModifiableContactPair> pairs ) {

        foreach ( var pair in pairs ) {
            for ( int i = 0; i < pair.contactCount; ++i ) {
                //check pair.colliderInstanceID and/or pair.otherColliderInstanceID against TargetColliderInstanceIDs / PlayerColliderInstanceIDs
                //Then if you need to ignore the collision,
                //pair.IgnoreContact( i );

& I add/remove stuff to/from TargetColliderInstanceIDs and PlayerColliderInstanceIDs from FixedUpdate in various places in my game code, so that when ModificationEvent gets called, it knows which pairs of things to ignore contacts from. I think FixedUpdate happens before Contact modification, which I guess must itself be before OnTrigger/OnCollision etc.