How to make a simple line of sight in a 2D top down shooter.

Questions & Answers
1

I’m making a top down shooter game, and I need to check if the player can see an enemy. At the moment, I’m using a raycast from the player to the enemy, and detecting if it hits the enemy or something else. But, for some reason, it doesn’t do anything. I’m using it in a function which returns my custom type ‘EnemyUnit’, but, even it the recast should hit something in the way, it just returns itself, but which it can’t actually see.
Here’s the block which has my raycast in it:

RaycastHit2D sightTest = Physics2D.Raycast(transform.position, transform.position - finalDetected.transform.position, detectionOptions.detectionRange);
		if (sightTest.collider != null) {
			if (sightTest.collider.gameObject != gameObject) {
				finalDetected = null;
				Debug.Log ("Rigidbody collider is: " + sightTest.collider);
			}
		}
		return finalDetected;

This isn’t the whole function, but it’s the part with the raycast in it.

finalDetected is a variable with type of EnemyUnit, you can - probably - guess what the rest are.

EDIT: I’ve just changed it a bit to make it not return anything, but now the problem is it just detects the map…

3

It seems you are casting the ray backwards. I think that if you want to cast the ray towards the enemy you should use for the end point:

finalDetected.transform.position - transform.position

…not the other way around. So, try:

 RaycastHit2D sightTest = Physics2D.Raycast(transform.position,  finalDetected.transform.position - transform.position, detectionOptions.detectionRange);
         if (sightTest.collider != null) {
             if (sightTest.collider.gameObject != gameObject) {
                 finalDetected = null;
                 Debug.Log ("Rigidbody collider is: " + sightTest.collider);
             }
         }
         return finalDetected;

I haven’t tested this myself, but this is something that “stands out”. Just try and see if it makes a difference.

4

Hi Zac

A few bits here. First up, as pointed out by pako, if what you want is a ray from the transform.position to finalDetected.transform.position, you have the maths the wrong way round. To calculate a vector that goes from A to B, the calculation is OFFSET = B - A.

A side note is that directions are typically normalized (i.e. of unit length), though its possible the raycast function does this for you internally.

First thing I would do here is use the debug utilities to draw your ray in the editor and make sure it is correct. See here: Unity - Scripting API: Debug.DrawRay

A quick blast at the code (without testing) would be something like:

//precompute our ray settings
Vector3 start = transform.position;
Vector3 direction = (finalDetected.transform.position - transform.position).normalized;
float distance = detectionOptions.detectionRange;

//draw the ray in the editor
Debug.DrawRay(start,direction*distance,Colors.Red);

//do the ray test
RaycastHit2D sightTest = Physics2D.Raycast(start,  direction, distance);
if (sightTest.collider != null) 
{
  if (sightTest.collider.gameObject != gameObject) 
  {
	  finalDetected = null;
	  Debug.Log ("Rigidbody collider is: " + sightTest.collider);
  }
}

With it drawn on screen you should know for certain if the ray is doing what you think it should be doing.

Following that, you might be seeing an issue as you are only asking the ray for the first object it hits (thats what Physics2D.Raycast does). So if it hits something else you won’t get your enemy. That’s fine if you definitely want the first object hit, but if there’s stuff you want to ignore and you aren’t using layer masks, you’ll need to handle manually.

You can limit this problem by filtering based on layer (see Cam’s answer), but if you still need to check all objects along the ray, use the raycastall function, and iterate over the results:

//precompute our ray settings
Vector3 start = transform.position;
Vector3 direction = (finalDetected.transform.position - transform.position).normalized;
float distance = detectionOptions.detectionRange;

//draw the ray in the editor
Debug.DrawRay(start,direction*distance,Colors.Red);

//do the ray test
RaycastHit2D[] sightTestResults = Physics2D.RaycastAll(start,  direction, distance);

//now iterate over all results to work out what has happened
for(int i = 0; i < sightTestResults.Length; i++)
{
	RaycastHit2D sightTest = sightTestResults*;*
  • //do stuff with this result? not sure exactly what your function needs though :)*
    }
    The results come out sorted (closest first), so you could identify:
    - stuff you wanted to ‘block’ the line of sight - break out of loop with ‘false’
    - stuff you don’t care about - just continue looping
    - stuff you want to detect hitting - break out of loop with ‘true’
    You could even use more debug draw functions to show where the ray cast hit to help you work out what is happening.
    Hope that helps!
    -Chris
2

Rather than checking to see if the raycast hits the GameObject performing the cast, you could assign all objects you want to check to a certain layer and cast against that.

To make that a bit clearer you could define two layers, one for enemies that you want to check line of sight against, and one for things that get in the way of line of sight like walls or obstacles. Then, when you do your raycast, you can pass it a layer mask to only check for collisions of objects on your “enemy” or “obstacle” layer. See here for how to set up and use layers.

The LayerMask parameter to Raycast is a bit weird if you don’t understand bitshifting (and I don’t, completely). It goes -something- like this:

-EDIT- I just tried to explain bitmasks, but it proved too long to write up in an answer. You can google it for a good explanation. Put REALLY shortly, if you want to mask against a single layer, you can use this as your LayerMask parameter:

1 << N

Where N is the layer you want to include.

For this example, if you set your “enemy” layer as layer 8 and your “obstacle” layer as layer 9 (the first 2 user-definable layers), I’m fairly sure that the LayerMask to include JUST layers 8 and 9 is:

11 << 8