Picking up objects based on raycast distance creates incorrect results

Hi there.

I’m currently setting up a machine. It has a single button, and when you press it, it instantiates a little test object on top of itself. The test objects in question are circles that you can pick up, walk around with, and drop, using this script:

public class CarryItem : MonoBehaviour
{
    public Transform holdPosition;

    private RaycastHit hit;

    Ray ray;

    int layerMask = 1 << 8;

    private void Awake()
    {
        layerMask = ~layerMask;
        ray = Camera.main.ScreenPointToRay(Input.mousePosition);
        holdPosition = GameObject.Find("HoldPosition").transform;
    }

    void OnMouseDown()
    {
        if (Physics.Raycast(ray, out hit, 4, layerMask))
        {
            GetComponent<Rigidbody>().velocity = Vector3.zero;
            GetComponent<Rigidbody>().useGravity = false;
            GetComponent<Rigidbody>().Sleep();
            this.transform.position = new Vector3(holdPosition.position.x, holdPosition.position.y, holdPosition.position.z);
            this.transform.parent = GameObject.Find("HoldPosition").transform;

            Debug.Log(hit.distance);
        }
    }

    void OnMouseUp()
    {
        GetComponent<Rigidbody>().useGravity = true;
        this.transform.parent = null;
    }
}

The section in OnMouseDown using Physics.Raycast is my attempt to not let the player pick the object up if it’s too far away (in this case, farther away than the raycast shoots). I also have it printing the hit.distance to see exactly what the issue is, and no matter how far away I pick up the object from (even if it’s from across the entire plane), it always allows me to, and prints seeminly unrelated and only slightly variable numbers:
6131915--668672--Screenshot_2.png
How can I tweak this code to allow it to work, and why isn’t hit.distance working as it should?

Have you tried logging hit.collider.name? I strongly suspect you’re hitting a different object than you think you are hitting. Maybe even yourself.

1 Like

I gave it a try. The two names that showed up were Player and Button. I’m not sure how I managed to have the ray hit the player, because the number it showed was far smaller than any other numbers I had seen (~0.0118). The Button hit was at a similar distance (~2.1).

I tried removing the layerMask = ~layerMask; line, but once I did that, I couldn’t pick anything up and it wouldn’t print anything in the console, no matter if I was close or far.

Rather than trying to do bitwise operations to try to get the appropriate mask, try just making it a public variable and setting up the layermask in the inspector:

public LayerMask PickupMask;

...

if (Physics.Raycast(..., ..., ..., Pickupmask.value) { // etc.
1 Like

Since you can’t apply the inverse operator to a LayerMask, how would I go about only allowing the ray to consider one layer instead of only allowing it to ignore one layer?

LayerMasks are “whitelists”, for the purpose of Raycasting. So if you have a layermask with only one layer selected, raycasts using that mask will only hit that layer. Here’s a screenshot from one of my projects where I have a LayerMask in an inspector. If I use this layermask for a raycast, it will only hit objects in the “Default”, “Terrain” and “Signal Blocking Surface” layers:

6132002--668684--upload_2020-7-25_19-7-9.png

In your case, all you have to do is select only the single layer you care about in this dropdown. You also can use the inverse operator - all you have to do is plug ~myMask.value as the layermask parameter to the raycast. If I did that with the mask in my screenshot, then the ray would be able to hit everything EXCEPT those checked in the list.

1 Like

I feel like something might be wrong with the object itself. If it’s useful to know, the objects I’m testing carrying with are instantiated during gameplay at the press of the button, not at the very start.

I have the layer mask set to the exact same layer that I have set the prefab to be on, but it’s still not doing anything at all.

Is the collider directly on the root of the prefab? Or is it on some child object of the prefab? If it’s on a child, what layer is the child in?

1 Like

The colider is directly on the prefab.

One thing I’m noticing is that you only ever calculate the ray once, in awake:

ray = Camera.main.ScreenPointToRay(Input.mousePosition);

Maybe you should put that line of code into the OnMouseDown()? Otherwise the ray will always be in the same place, which only is related to wherever the mouse/camera were when this object was first created.

It finally worked after I put that line at the top of OnMouseDown. Thank you so much!

1 Like

Due to a refactor, the system is once again broken, but I’m quite close to finding a fix. Figured you might be able to help me out.

I changed it to work on a toggle system so you don’t have to hold down the mouse in order to carry something. This is my code as of current:

using UnityEngine;

public class CarryItem : MonoBehaviour
{
    public Transform holdPosition;

    public bool isCarrying = false;

    public LayerMask layerMask;

    Ray ray;
    RaycastHit hit;

    private void Awake()
    {
        holdPosition = GameObject.Find("HoldPosition").transform;
    }

    private void Update()
    {
        ray = Camera.main.ScreenPointToRay(Input.mousePosition);
        if(Physics.Raycast(ray, out hit, 4, layerMask.value))
        {
            Debug.Log(hit.collider.name);
            Debug.Log(hit.distance);
            if(Input.GetMouseButtonDown(1))
            {
                if(isCarrying)
                {
                    GetComponent<Rigidbody>().useGravity = true;
                    this.transform.parent = null;
                    isCarrying = false;
                }
                else
                {
                    GetComponent<Rigidbody>().velocity = Vector3.zero;
                    GetComponent<Rigidbody>().useGravity = false;
                    GetComponent<Rigidbody>().Sleep();
                    this.transform.position = new Vector3(holdPosition.position.x, holdPosition.position.y, holdPosition.position.z);
                    this.transform.parent = GameObject.Find("HoldPosition").transform;
                }
            }
        }
    }
}

The Debug.Log statements show that the ray is detecting what it’s hitting and how far away it is perfectly well, but clicking the left mouse button does nothing.

You may kick yourself after reading this but Input.GetMouseButtonDown(1) is the right mouse button. The left would be Input.GetMouseButtonDown(0)

You were, in fact, correct about me kicking myself. Thanks.