Checking if a layer is in a layer mask?

Unity Engine
,
1

Hi there! I’m trying to check if a layer (int) is in a layerMask (LasyerMask) and I can’t seem to find a solution online that works for me.

I can use the LayerMask just fine inside of a Raycast:

bool hitBool = Physics.Raycast(ray, out hit, maxDistance, layerMask);

In this case, the layerMask is a [SerializeField] in the inspector so I can easily choose what layers I want to check for.

The problem is I was trying to check if the hit.transform.gameObject.layer was inside the layerMask later, mostly for debug purposes. but can’t figure out how to do it. From what I gathered online it involves bit shift but I can’t seem to get it to work.

Basically I want something like this:

if (hit.gameObject.layer "is found in" layerMask)
2

It is exactly bit shifts:

https://discussions.unity.com/t/802897/2

To ask, “does this mask contain this layer?” you would use bitwise-and (&) with bitwise shifting (<<):

if (( mask & (1 << layer)) != 0)
{
  // yup
}
else
{
  // nope
}

Now masks can be inclusive or exclusive, eg, that bit being 1 or 0 can mean exclude or include depending on where it is used.

15 Likes
3

Got it, thanks! I’ll read up the post too since this is something I’m not very familiar with!

1 Like
4

Because a LayerMask is a series of bits, what you really need to know is “is the Xth bit from the right, a 1?” Once you frame the question that way, all you need to know is the arcane syntax C# has for dealing with bits! The important operators here are:
Bitshift: << and >> which mean “take number A and shift its bits B spaces left or right”
Bitwise and/or: & and | These work just like && and ||, but they work on every bit in a value
Bitwise not: ~ just like above, it’s like the regular not (the ! operator), but on every bit.

So once you have that, you need to:

  1. Start with a 1 (which is 000000001 in binary)
  2. Shift its bit to match the layer you watch (so it’s now 000010000 for layer 4), which essentially creates a mask for that one layer
  3. Use “bitwise and” to merge this ask with the mask you’re checking against
  4. if this has result in anything nonzero, then the masks have some overlap - which means you’ve got a match

In code this looks like:

LayerMask checkingMask;

...
int thisLayer = 4; // probably comes from gmaeObject.layer
int thisLayerMask = 1 << thisLayer;
if (checkingMask & thisLayerMask != 0) {
//yes
}
8 Likes
5

This is one of the things that REALLY should have a built-in function.

Layermask.Contains() or something.

7 Likes
6

While it’s true that a built-in method for the Layermask type would be nice, you can always add extension methods to such types yourself.

Apart from that bitmasks in general are a quite common thing. So even if Layermask would have that feature, whenever you have another bitmask which is just an “int”, you would have the same problem again. Over here I posted a garbage free bitmask iterator which allows you to iterate through all layer indices that are contained in a bitmask.

bit flag checks are something that simple and common most people don’t really see a need for an explicit method. Though I do get that the code looks cleaner, but not necessarily more readable. Every method needs to have a name that should communicate its use as clear as possible. For example when you have a Contains method on the Layermask type, does it expect a layer mask or a layer index that should be checked? Both are possible. Sure, a layer index would make a bit more sense, but both would have their uses.

1 Like
7

That’s exactly the perils of something like a .Contains()… do you mean the bit? The bits? Or the layer? It will silently and mysteriously fail, and worse of all it might actually work when its incorrect and then break when additional (or different) layers are involved.

You could make a .ContainsLayer() and .ContainsMask() I suppose, which would help clarify the intent.

But it is unambiguous if you do the bit-flicking yourself in code, plus it helps your program clearly exhibit if you’re talking about bits or the numeric layer at any given instant.

And it also expands your brain and fills the new parts with numbers and boolean math. :slight_smile:

1 Like
8

For anyone who is looking for a method they can just copy and paste, here it is. We use this in production:

EDIT: integrated comment by @Bunny83 below.

public class Misc
{
    public static bool IsInLayerMask(GameObject obj, LayerMask mask) => (mask.value & (1 << obj.layer)) != 0;
    public static bool IsInLayerMask(int layer, LayerMask mask) => (mask.value & (1 << layer)) != 0;
}
4 Likes
9

Note that this may fail for layer 31 as we’re dealing with a signed integer and the most significant bit is the sign bit. So you would get a negative value for layer 31 (-2147483648 to be exact). So it’s usually better to use the condition != 0 instead of > 0. Essentially what Kurt posted in the second post. Though for layers 0-30 it should work.

Note that when you actually create a dedicated method for checking a gameobject against a layermask, it probably should be an extension method. Because having to call a static method in a utility class is in the end longer than just doing the bitwise and in place.

6 Likes