Are you moving physics yourself? If so keep this in mind:
With Physics (or Physics2D), never manipulate the Transform directly. If you manipulate the Transform directly, you are bypassing the physics system and you can reasonably expect glitching and missed collisions and other physics mayhem.
Always use the .MovePosition() and .MoveRotation() methods on the Rigidbody (or Rigidbody2D) instance in order to move or rotate things. Doing this keeps the physics system informed about what is going on.
PS - your game reminds me a little bit of Descent… I love it with the little lights on the guys going into the tunnels!!