Layermasks took me a while to figure out and get working correctly, and I've seen a few people asking about them on the forums, so I thought I'd add a little something to help anyone who may be struggling with them.
The question is: How do I use/create layermasks to use with, e.g., Physics.Raycast or other Physics functions? For those who don't know: a layermask allows you to specify certain layers to exclude/include in certain physics functions.
layermasks work by setting bits in an integer value to 0 (false) or 1 (true), which represent whether or not to test against a certain layer. The first bit from the right is used for layer 1, the second for layer 2, etc. So a value of 00000101 (decimal value of 5), which has the first and third bit set to 1 (true), will test ONLY against layers one and three.
Using bit shifting to create a layermask
The easiest way to produce a layermask is using the bit shift operator (<<). What this does is move each bit in an value to the left. So if you have a value of 6 (binary value of 00000110), and shift it two places to the left, you will be given a value of 24 (binary 00011000).
Remember that a value of 1 in decimal is 00000001 in binary. So if you bit shift 1 by n places to the left, you will end up with the nth bit being 1 and all other bits being 0. Based on this, the following code will produce a layermask which can be used to only test against the given layer:
var layer : int = 3;
var layermask : int = 1 << layer;
layermask will turn out to be 4 in decimal (00000100). Notice that the third bit is 1 (true).
Combining layermasks
To produce a layermask which will test against more than one layer, you can use the binary OR operator (|) to produce a new layermask. A bit in the new layermask is true if the corresponding bit was true in either one of the layermasks it was made from:
var layer1 : int = 3;
var layer2 : int = 5;
var layermask1 : int = 1 << layer1;
var layermask2 : int = 1 << layer2;
var finalmask : int = layermask1 | layermask2; // Or, (1 << layer1) | (1 << layer2)
// The bits look like this:
// (00000001) First bit is true in layermask1
// (00000100) Third bit is true in layermask2
// (00000101) Both first and third bit are true in finalmask
(Note that adding layermasks together with (+) is usually not wanted. For example, adding two layermasks that both contain the same layer, such as 00000011 (layer 1 and 2) and 00000110 (layer 2 and 3) would produce 00001001 (layer 1 and 4), while with the bitwise OR operator (|) you get 00000111 (layer 1, 2, and 3) as expected.)
Inverting a layermask
To test for all layers except the given layer, just use the bit inverse operator (~). The bit invert operator takes every bit in an integer, and swaps it from 0 to 1 or vice versa. So ~00000010 equals 11111101, which would test for every layer except layer 2.
Note that if there are two layers you want to exclude, you need to combine the bitmasks with the OR operator before inverting:
var layer1 : int = 3;
var layer2 : int = 5;
var layermask : int = ~((1 << layer1) | (1 << layer2)) // NOT ~(1 << layer1) | ~(1 << layer2)
This would give you a value of 11101011.
I hope this helped somebody!
This is an attempt to simplify Stelimars insightful post.
LayerMask, or any "Mask" in Computer Science, is an integer which uses its 32 bits to represent different flags. While it is an integer, the value doesn't mean anything useful numerically. What matters is which flags are turned on or off.
In the case of LayerMask, each flag (each bit) represents one layer. If you select any game object from the editor you'll notice the Layer field in the top right corner of the inspector. Clicking this will bring up a list of different layers. There are some predefined Layers, but you can also add layers. Notice how you have 32 slots for your layers. This is because Unity uses an integer type to store the flags in each of the 32 bits that makes up an integer. As we can see, Layer 8 to 31 is available for us to define for our own game.
Since Masks use a single bit to represent if some option is true or false, a Mask can contain many different options. The same goes when thinking about Layers. A LayerMask specifies which Layers are enabled for any given mechanism. This enables us to tell the game that our GameObject "Don", for example, is thought to belong to the user defined layer "NPC" and "Bob" to belong to the user defined layer "SpellCaster".
Then, we make us of this information for example in a raycast call, specifying that the raycast should only return objects that belong to the "NPC" layer. An example when this could be used is when you want to target NPCs only with the click of the mouse. Then, imagine you have a game where you want to cast a spell, and that spell can only target Spell Casters. Then you'd only include the game objects that has the SpellCaster layer enabled. A third option would be to include both NPC and SpellCaster in the search. That way we would test both Don and Bob.
Finally some technical mumbo.
A LayerMask is represented as a 32bit integer. It also has a name table for naming the different bits so it is more readily understood by us game developers. Just to give some perspective, here's how a 32 bit integer would look like if it was written.
0000 0000 0000 0000 0000 0000 0000 0000
I put a space between every 4th bit to make it easier to read.
The first bit is actually located on the right side. Think of this as being the first layer. Here's how it would look if the first bit (or layer) would be set:
0000 0000 0000 0000 0000 0000 0000 0001
Since Unity predefines 8 layers, we can display the Mask for including all of those layers as such:
0000 0000 0000 0000 0000 0000 1111 1111
And if we like to search for NPC as from the example above, given we've put it in User Layer 8 (starting from 0, this is the 9th layer), we'd mask it as:
0000 0000 0000 0000 0000 0001 0000 0000 // NPC Layer
Just for completeness, this is how you should think a mask that include NPC OR SpellCaster should look like, if we assume the SpellCaster was set to User Layer 9:
0000 0000 0000 0000 0000 0011 0000 0000 // NPC | SpellCaster Layer
Allright, now that you see we're setting bits to enable the different layers, let's look at how we generate the bit field from scratch using C#. In my example, I will make three masks that can be used to query NPCs, SpellCasters, and "NPCs and SpellCasters". We'll use Physics.Raycast in our example.
Example 1: Raycast vs NPCs
int NPCLayer = 8;
// This is used to illustrate which Layer we want to use.
int NPCMask = 1 << NPCLayer;
// This is used to generate the Mask derived from NPC Layer.
if Physics.Raycast(fromPosition, direction, Mathf.Infinity, NPCMask)
{
// We cast a ray onto a NPC!
}
Example 2: Raycast vs SpellCasters
int SpellCasterLayer = 9;
// This is used to illustrate which Layer we want to use.
int SpellCasterMask = 1 << SpellCasterLayer;
// This is used to generate the Mask derived from SpellCaster Layer.
if Physics.Raycast(fromPosition, direction, Mathf.Infinity, SpellCasterMask)
{
// We cast a ray onto a spell caster!
}
Example 3: Raycast vs NPCs and SpellCasters
int NPCLayer = 8;
int SpellCasterLayer = 9;
// This is used to illustrate which Layers we want to use.
int NPCMask = 1 << NPCLayer;
int SpellCasterMask = 1 << SpellCasterLayer;
// This is used to generate the Mask derived from the layers.
int CombinedMask = NPCMask | SpellCasterMask;
// We use the binary OR operator to combine masks.
if Physics.Raycast(fromPosition, direction, Mathf.Infinity, CombinedMask)
{
// We cast a ray onto a NPC or a SpellCaster!
}
Stelimar also have excellent information in his posts that you should read.
While the information about layermasks in the other answers is good, it would be remiss not to mention the LayerMask type. Generally this is the easiest way to use them, since you don’t have to convert anything or know anything about binary math (although it’s a good idea to have that knowledge anyway, in case you want to do more complex layermask coding).
var myLayerMask : LayerMask;
if Physics.Raycast(fromPosition, direction, Mathf.Infinity, myLayerMask)
{
// Do something
}
This way you can simply choose the appropriate layers in the inspector.
The answers above do a great job of explaining how to use LayerMasks and the technical stuff behind it. This is just a little additional information I thought people might find helpful.
I find that working with LayerMasks by name instead of layer number makes the process a lot cleaner to read.
The LayerMask class has static methods
int NameToLayer(string layerName) // Returns Layer Number From Layer Name
string LayerToName(int layer) // Returns Layer Name From Number
When I build LayerMasks in my scripts, I alway use the names instead of the numbers. For instance…
lightBeamLayerMask = (1 << LayerMask.NameToLayer("World Geometry"))
| (1 << LayerMask.NameToLayer("Light Triggers"));
This way I don’t have to worry about whether “World Geometry” is Layer 8, or 10.
Also, I usually store LayerMasks as private variables and build them once in the Start() method, rather than creating them immediately before using them like the Unity Scripting Reference Examples.
Although you can always use the other mentioned methods to get the perfect layer-mask you desire. I’d like to point out another possibility to have the layermask. Which is actually putting the integer hardcoded. It’s not recommended though, unless you’ve got some problems with any other method, or may be if you’ve a doubt that your bitwise calculations are not correct.
For eg. if you want to include layer 2 and 4 in the layermask, the calculation should be
int mask = 1<<2 | 1<<4;
which will result into 0001 0100 which in decimal results into 20, so you can just put 20 as a layermask too.
int mask = 20;
So instead of putting the calculation in the code, you are doing calculations yourself and putting just the result. However since it’s hardcoding, I’d not recommend using this method except for debugging.
Make sure you’re not passing in the layer mask in the distance argument position.
Do This
if (Physics.Raycast(transform.position, transform.forward, out hit, Mathf.Infinity, SelectionLOSCheckMask))
NOT THIS
if (Physics.Raycast(transform.position, transform.forward, out hit, SelectionLOSCheckMask))
Honestly, after trying all these methods, I find the easiest to use ‘GetMask’ command:
You set your variable at the start:
int maskMouseRaycastDefault;
and then in the Awake function, define the names that create that mask. No need for bit shifting, or memorizing what number is what.
void Awake(){
maskMouseRaycastDefault = LayerMask.GetMask("Default", "Collision", "DynamicObjs");
}
The biggest advantage, and the reason why I stopped using the bit shift method, is that you can very clearly read what your layerMask is composed of, and if for some reason you later need to change your layer order, as long as you preserve the names it still works.
The easiest way of getting the masks is like this:
var layerNames = new string[] { "YourLayerName" };
var layerMask = LayerMask.GetMask(layerNames);
If you simply make a public LayerMask var, multiple layers can be selected simultaneously in the dropdown.
public LayerMask layerMask;
And then, for example:
var standing = Physics2D.OverlapCircle( position, radius, layerMask );
LayerMask _groundLayerMask = 1 << 10;
LayerMask _obstacleLayerMask = 1 << 20;
LayerMask _clearanceCheckLayerMask = _groundLayerMask | _obstacleLayerMask;
above is what worked for me when combining layers (c#)