Raycast hits rigidbody even without a collider?

I have a character model, with colliders on the bodyparts (nested GameObjects, with layer of type “PlayerBodyPart”), so that I can detect hits on arms, head, legs and so on.

I have a Rigidbody on the Root node, this GameObject has layer “PlayerCapsule”.

I’m doing a Raycast using a LayerMask with only “PlayerBodyPart” selected, so that should be the only thing I hit. But I’m hitting the root node, even though it doesn’t have any colliders on it, and it isn’t on the correct layer.

If I remove the Rigidbody on the root node, then I can hit the actual body parts.

Does a Rigidbody somehow own all nested colliders or something? I can’t really figure out what to do here.
I need the Rigidbody on the root to move my character, but I can’t figure out how to not hit the root, but instead only the body parts.
Do I have to add rigidbodies to all bodyparts? I’ve tried that, but it really messes with my movement, and just floats around.

Any suggestions?

I found out one of my PlayerBodyParts were on layer Default instead. So the root node capsule would collide with the Default layered gameobject and try to push the two colliders apart, I guess, but they’re nested, so it just made the character float around.
So I’ve added Rigidbodies to all PlayerBodyParts, and now I can hit the correct GameObjects.
So I guess a Rigidbody does own all nested colliders, which do not have a Rigidbody themself.

I found out I don’t need rigidbodies on everything. I can just call
rayCastHit.collider.gameObject to get the actual game object that I hit, instead of getting the root. I also learned something about compound colliders, which I hadn’t encoutered before.