Howto: if layermask contains layer?

Unity Engine
1

Hi

I have searched a bit on the forum for how to see if a layermasks layer is enabled, although I can only find how you create a layermask.
The reason is that when building a custom inspector you can’t create a LayerMask field, so I’m trying to create one myself.

Is it something like

if (layermask == 1<<layer)

?

2 Likes
2

It’s not quite as simple as that, unfortunately. The layer mask is an integer, and you need to test just a single bit. You need to do something like:-

if (((layermask >> layer)  1) == 1) {
   ...
}

What this does is shift the bit that you want to test into the rightmost bit (ie, the digit that represents the value of 1). The bitwise-and operator then sets all bits except this one to zero. The value in the integer is then the value of the specific bit you want to test - it will be zero if false and one if true.

4 Likes
3

Hi

Thanks for the answer.
Although, like 10 seconds before you answered I found another solution:

if (layermask == (layermask | (1 << layer))) {

i.e if layermask equals layermask + my layer
And now I got a full LayerMask field for custom inspectors :smile:

public static int LayerMaskField (LayerMask selected) {
		ArrayList layers = new ArrayList ();
		ArrayList layerNumbers = new ArrayList ();
		string name = "";
		for (int i=0;i<32;i++) {
			string layerName = LayerMask.LayerToName (i);
			if (layerName != "") {
				if (selected == (selected | (1 << i))) {
					layers.Add ("> "+layerName);
					name += layerName+", ";
				} else {
					layers.Add (layerName);
				}
				layerNumbers.Add (i);
			}
		}
		bool preChange = GUI.changed;
		GUI.changed = false;
		int[] LayerNumbers = layerNumbers.ToArray (typeof(int)) as int[];
		int newSelected = EditorGUILayout.Popup ("Mask",-1,layers.ToArray(typeof(string)) as string[],EditorStyles.layerMaskField);
		if (GUI.changed) {
			if (selected == (selected | (1 << LayerNumbers[newSelected]))) {
				selected = ~(1 << LayerNumbers[newSelected]);
				Debug.Log ("Set Layer "+LayerMask.LayerToName (LayerNumbers[newSelected]) + " To False "+selected.value);
			} else {
				Debug.Log ("Set Layer "+LayerMask.LayerToName (LayerNumbers[newSelected]) + " To True "+selected.value);
				selected = selected | (1 << LayerNumbers[newSelected]);
			}
		} else {
			GUI.changed = preChange;
		}
		return selected;
	}
3 Likes
4

By the way, custom editor scripts CAN use the default inspector:
public override void OnInspectorGUI ()
{
base.OnInspectorGUI();
// …More code here
}

And for any custom fields, use [HideInInspector] right next to the decleration. :slight_smile:

5

Those are both “harder” than this more common method (and IMO also less intuitive):

if (layerMask & (1 << layer) != 0)
{
    Debug.Log("I belong in this mask.");
}

It works because the 1 << layer creates an int with a single bit shifted over layer number times (layer 1 gives you 0001, layer 2 gives you 0010, layer 3 gives you 0100, etc). Then by using the bitwise and operator (&) it will return any bits that exist in both integers. So, it gives you a 0 if no bits match.

3 Likes
6

at least in Unity 2020, this throws an error: error CS0019: Operator ‘&’ cannot be applied to operands of type ‘LayerMask’ and ‘bool’

1 Like
7

Not just in Unity 2020 ^^. Have a look at the Operators Precedence in C#. So the shift operator has the highest priority of the 3 operators we used. Therefore the brackets he used are pointless since the shift is done first anyways. However the issue is that the comparison operator != has a higher priority than the bitwise AND operator &. So when we break down his code it actually tries to do this

int mask = 1 << layer;
bool tmp = mask != 0;
bool result = layerMask & tmp;
if(result)

hopefully you see the result line makes no sense since we try to use the bitwise AND between a layermask and a boolean value. What you have to do is using brackets like this

if ((layerMask & 1 << layer) != 0)

This will result in this code

int mask = 1 << layer;
int tmp = layermask & mask ;
bool result = tmp != 0;
if(result)
1 Like