Is my 2D Raycast set up correctly? because it does not seem to hit anything when i use a layer mask.

Questions & Answers
1

I am trying to use RaycastHit2D to implement the attack for my enemy. It works fine when not using layers however it then hits things I don’t want it to hit (namely the enemy collider itself). And when using a layer mask, it doesn’t work at all.

I am fairly sure I have everything set up correctly, though I am only a novice so I feel I’m probably going wrong somewhere.

The player is set up like this; A game object with the sprite renderer, and a collider (not a trigger) on the feet, which is set to just the default layer because I don’t actually want the ray to detect this collider, it’s just for actual collisions with walls and things. Then I have another game object attached to the player that has only a Box Collider 2D set as a trigger, because I don’t want it to physically collide with anything, I only want the Raycast from the enemy attack to detect this collider. (my research has led me to believe that the ray should still be able to detect a trigger, is this correct?) Anyway, I have set this hit detection collider to be on layer 8, which I have named “Player” in the project settings, and I have set the GameObject to be on this layer.

It seems to work when I’m don’t use a layer mask, but when I use one, the ray from the enemy just does not detect a thing. The code is executed via an animation trigger on a frame in the enemy attack animation.

Here’s the code;

public void Attack()
{
RaycastHit2D hit = Physics2D.Raycast(this.transform.position, -this.transform.right, 5, 8);

 //Debug
Debug.Log(hit.transform.name);
Debug.DrawRay(this.transform.position, -this.transform.right * 5, Color.red, 10f);
}

In my understanding, this should fire a ray from the position of the enemy, in the direction of left, with a length of 5, and detect things on layer 8 (the players layer)

But it doesn’t work when I use the layer part for this method. Even my debugs stop working. I’m not really sure what I’m doing wrong, but it’s obviously something. I’ve been looking for a solution for hours but I’m completely stumped.

Any input is appreciated, thanks.

2

Your layer mask needs to go through a bit shift first. Documented here and explained very well here

So just declare your layer mask like this:

 public void Attack()
 {
 int playerLayer = 8;
 int layerMask = 1 << playerLayer;
 RaycastHit2D hit = Physics2D.Raycast(this.transform.position, -this.transform.right, 5, layerMask);
 
  //Debug

}