Raycasting on Navmesh?

I’m trying to get my click to move setup working with Navmesh, but I’m having some problems and not entirely sure I’m doing it right. As it stands the raycast doesn’t seem to hit anything.

What I want to do is when I click on my terrain, send a ray from the camera to the navmesh, and set the destination of that point to where the raycast hit the navmesh. Below is my code - also I don’t know quite how to use the passableMask parameter (I put 0 as my integer, hoping to access “Built in Layer 0”):

// Moves the Player if the Left Mouse Button was clicked
		if (Input.GetMouseButtonDown(0)&& GUIUtility.hotControl ==0) {

			Ray ray = Camera.main.ScreenPointToRay(Input.mousePosition);
			NavMeshHit hit;
			
			if (NavMesh.Raycast(ray.origin, ray.direction, out hit, 0)) {
				Vector3 targetPoint = hit.position;
				destinationPosition = hit.position;
				
				targetRotation = Quaternion.LookRotation(targetPoint - transform.position);
				Debug.Log ("NavMesh hit!");
			}
			Debug.Log (destinationPosition);
		}

If anyone wants to know how to do a raycast to the navmesh and is having trouble using the answer here, here’s the code that works for me (my navmesh lies on the xz plane):

function raycastToNavmesh(screenPosition : Vector3) : NavMeshHitResult{
	var ray : Ray = Camera.mainCamera.ScreenPointToRay(screenPosition);
    	
    var positionOnGroundPlane : Vector3 = ray.origin - ray.direction / ray.direction.y * ray.origin.y;  //collide with plane at y=0
    	
    var navMeshHit : NavMeshHit;
    var result = NavMesh.SamplePosition(raycastHit.point, navMeshHit, 1, 1);
    return new NavMeshHitResult(result, navMeshHit);
}

Or if you have a terrain or other 3D geometry on which your navmesh lies on top of (such as hills, etc) you can use Physics collisions after adding your collider components, but you need to map that collision point to a point on the navmesh:

function raycastToNavmesh(screenPosition : Vector3) : NavMeshHitResult{
    	var ray : Ray = Camera.mainCamera.ScreenPointToRay(screenPosition);
        	
        var raycastHit : RaycastHit;
        if(Physics.Raycast(ray.origin, ray.direction, raycastHit)){
        	var navMeshHit : NavMeshHit;
        	var result = NavMesh.SamplePosition(raycastHit.point, navMeshHit, 1, 1);
        		return new NavMeshHitResult(result, navMeshHit);
        }
}

Where NavMeshHitResult is just a script I made to contain the results:

class NavMeshHitResult{
	var result : boolean;
	var navMeshHit : NavMeshHit;
	
	function NavMeshHitResult(result : boolean, navMeshHit : NavMeshHit){
		this.result = result;
		this.navMeshHit = navMeshHit;
	}
}

Why don’t you use a regular raycast? I am not familiar with this NavMesh.Raycast but by the look at the docs, I am confused with your ray.direction. It seems NavMesh needs a start and end point but you are giving a start and direction. I would assume that if your direction is Vector3.forward(0,0,1) your ray is from start to that point and then not getting through the ground.

I would recommend raycast hit instead

Camera other = GameObject.FindObjectOfType(typeof(Camera)) as Camera;
Raycast hit;
if(Physics.Raycast(other.ScreenPointToRay(Vector3(Input.mousePosition)), out hit)){
  Vector3 targetPoint = hit.position;
  // other codes
}