How do I ignore some collisions but not others for some objects, and keep all collisions for other objects?

I have four types of objects currently. The player, bullets, enemies, and xp. Xp should only collide with the player, bullets should only collide with enemies, and the player should collide with everything except the bullets. Is there a way I can achieve this? If you need more information, let me know.

You can do it through layers by selecting with what they can collide

Here:

You could also do it by comparing the tag of the collided gameobject to the desired tag and then do stuff if it matches