I must have a fundamental misunderstanding about how to linecast.

Unity Engine
1

Hey everyone, I’m having one of those moments where everything seems impossible. I have two squares with polycollider2d components attached, I need to linecast the collider path of one to find the intersect point on the other but I just can’t get it to ignore the primary collider. I’ve tried setting the object’s layer to ignore raycasts and filter that layer out in my linecast, as well as setting the objects z position outside the min/maxDepth test of the linecast and even tried disabling the collider entirely but it still hits it! Even all three methods in conjunction to try and ignore the primary object just don’t seem to work. I even tried moving the main collider a few hundred units away while I raycast and then back again but it still hits an object that shouldn’t even be there…

So the result I get at the moment is the linecast instantly hits the primary collider at the very start of the ray, regardless of everything I try.

transform.position = new Vector3(transform.position.x, transform.position.y, -2);
//mainCollider.enabled = false;

            Debug.Log("test all the points");
            for (int i = 0; i < mainColliderPathScaled.Length; i++)
            {
                Vector2 pointPosition = mainColliderPathScaled[i] + new Vector2(transform.position.x, transform.position.y);
                //wrap in the index values to avoid out of bounds errors
                int pindex = (i - 1 >= 0 ? i-1 : mainColliderPath.Length-1);
                int nindex = (i + 1 < mainColliderPath.Length ? i+1 : 0);

                Vector2 prevPointPosition = mainColliderPathScaled[pindex] + new Vector2(transform.position.x, transform.position.y);
                Vector2 nextPointPosition = mainColliderPathScaled[nindex] + new Vector2(transform.position.x, transform.position.y);               

                if (otherCollider.OverlapPoint(pointPosition))
                {
                    Debug.Log("Point inside other collider ");

                    RaycastHit2D rayTemp;
                    if (otherCollider.OverlapPoint(prevPointPosition))
                    {
                        rayTemp = Physics2D.Linecast(nextPointPosition, pointPosition, Physics2D.DefaultRaycastLayers, -1, 1);
                        Debug.DrawLine(nextPointPosition, pointPosition, Color.green, 5);
                    }
2

Why the need to perform multiple linecasts to find the intersect point? Can’t you just access the contacts property of the collision obtained in OnCollisionStay2D?

3

No, the two colliders dont’ actually collider. I’m coding in subtracting the shape of one object from the other.

4

So you’ve tried…

RaycastHit2D lineCastHit = Physics2D.Linecast(startPoint, endPoint, ~LayerMask.GetMask(new string[]{"raycastingObjectLayer"}) )

?

The syntax might be slightly off, but the idea is to negate the Layer that the Raycast originates from by using LayerMast.GetMask( string[ ] layers) to get the mask of the layer for the object(s) you want to ignore, and bitwise-negating the result so that all layers except the layer(s) you want to ignore is tested against.

I’m only recommending this suggestion because you said you’ve tried something like this, but I fell into a similar trap before where I thought the decimal number that resembled the layer in the Editor was the layer number, when that indeed isn’t the case.

Edit: You don’t have to use that exact line of course. It was just a quick-and-dirty way of doing it. For better performance, you can simply store the layer result value(s) for reuse.

5

Both objects are on the same layer so ‘~LayerMask.GetMask(newstring[ ]{“raycastingObjectLayer”})’ should ignore both objects although it still seems to hit the primary collider, however removing the ~ now the z depth test works and ignores the primary collider. Replacing it back with Physics2D.defaultRaycastLayers goes back to colliding with the primary box despite no change to it’s z. So it’s fix (for now)! But I still don’t understand why. Perhaps because the primary object’s layer in changed at the start of the function maybe it doesn’t take affect straight away? Still wouldn’t explain the Z part not working.

6

Sorry, I’m a bit tired. I’m curious about these lines of code:

                        rayTemp = Physics2D.Linecast(nextPointPosition, pointPosition, Physics2D.DefaultRaycastLayers, -1, 1);
                        Debug.DrawLine(nextPointPosition, pointPosition, Color.green, 5);

nextPointPosition is the startPoint of the line and pointPosition is the endPoint. Is this intentional, or did you mean to cast your line the other way around?

It’s really difficult to tell without seeing the logic for the rest of your code. If I’m way off the mark I apologize. Just trying to see why you’re getting weird results.