Getting LayerMasks to work....

Physics
1

I have read all the threads on layerMasks I can find, as well as the thread on UnityAnswers, and I am still completely confused.

Here is the set up.

I have a disc that I want to raycast against, and nothing else. In my scene, the disc can be obscured by other objects with colliders. I want to cast a ray that ignores everything except the disc.

The disc is assigned to a layer called “Disc”, which in the Tag Manager is User Layer 10.

Now, I create a bitmask for the layer like so:

private var lm : int;
lm = 1 << 10;

First question, is that correct? Being that layers are a 0 based index, am I looking for 1 << 10, or 1 << 9?

Next, I use the layerMask thusly:

if (Physics.Raycast (ui_ray, ui_Arrow_hit, 1000.0, lm))

Now for clarity, everything about the code works perfectly, if I remove the layerMask.

The question is, does everything look correct for what I want to do? Cast a ray that only hits the object assigned to the “Disc” layer (User Layer 10).

I have also tried creating a public variable to set in the inspector for the layermask, but it still doesn’t work correctly.

I am so confused.

2

It’s easier if you use, as a public variable:

var layerMask : LayerMask;

–Eric

3

I tried the public var. The issue I see, is that the layermask isn’t doing anything.

Once the disc goes behind another collider, it fails to register the hit.

I will try to set up a basic scene and get it working there…

4

So maybe I have more information on what is going on.

In my scene where I need this to work, I am trying to run it from a OnMouseOver() event, and it doesn’t seem to work.

In my test scene I am running the same code in the Update() and it seems to be working.

Is casting a ray from the OnMouseOver not going to do it for me? What I need is an object to be drawn (for aiming purposes) when the mouse is over the disc collider.

Maybe I need to move everything into Update, and flip bools, and draw my object based on that, rather than the OnMouseOver…

Off to test. Input appreciated.

5

I think you need to have a rigidbody on the disc. This can have its isKinematic setting enabled if you don’t want the disc to have physical behaviour (falling under gravity, etc).

6

Thanks Andy, but unfortunately, I have a rigidBody on the disc in the scene I need it to work in, and in my test scene, I don’t have a rigidBody.

I think I may have another solution entirely, we shall see tomorrow.

7

Simply having a LayerMask variable by itself won’t do anything; you need to use the variable in a function, and OnMouseOver has no parameters that would allow layermasks.

(You don’t need a rigidbody for raycasting, only for physics behavior, bad andeeee. :wink: )

–Eric

8

Yeah, I was sure I’d seen that in the docs once upon a time and fixed a problem by using it. But I can’t find any such thing now, so bad andeeee indeed (or should it be “indeeeed”?) :wink:

9

Here are the pertinent parts of the code:

#pragma strict

var layermask : LayerMask;

function OnMouseOver () {
	
	if(BallControls.canDrawArrow){
		arrow.renderer.enabled = true;
		arrowInner.renderer.enabled = true;
	}
	
	ui_ray = Camera.main.ScreenPointToRay (Input.mousePosition);		
	
	if (Physics.Raycast (ui_ray, ui_Arrow_hit, 1000.0, layermask)) {
        //do a bunch of stuff here
}

Now, the layermask is set to the appropriate layer that I want to cast a ray onto, in this case the Disc layer. The only object assigned to that layer is the disc I want to hit.

If any other object gets in between the camera and the target, I no longer am able to hit the disc. As expected.

If I change the mask to cast a ray against everything except the layer that contains the interfering object, it doesn’t work. It does not ignore the collider that interferes.

I have attached a test scene that shows what I need to happen, but ain’t happening.

In the scene, I want the cube on top of the sphere to fade off when it gets between the camera and the sphere. I want the OnMouseOver to cast a ray at the sphere, and register a hit through the cube collider. You will notice that I print out the collider hit by the raycast, along with Time.time so I can see something changing, and the Time.time value stops updating when the raycast fails.

I have spent a few days fighting with this, and I am currently doubting if this is even possible.

332400–11704–$layermasktest_194.unitypackage (7.4 KB)