RaycastHit.triangleIndex always returning -1

Questions & Answers
1

So I’ve been looking into how to get the material of a certain triangle on a certain mesh by raycasting and then attempting to get the triangle index and search through the object’s submeshes to find the triangle that was hit and thus, it’s index which should be used to get the material BUT for some reason I am always getting a “-1” for the triangle index return when raycasting onto my mesh. I looked around and didn’t see anyone else with “unity raycast.triangleIndex returning -1” issues so I came here.

Sorry if I missed something someone posted before but thought I’d ask to see if anyone knew what my issue might be. Below is the code I currently have.

Vector3 HitCoords(){
		//Returns Vector3 of the mouse's click location
		Ray ray = Camera.main.ScreenPointToRay(Input.mousePosition);
		RaycastHit hit;

		//Raycast to mouse point
		Physics.Raycast (ray, out hit, 100);

            //set burstMaterial to hit triangle's material
	FindTriangleMaterial (hit);

		return hit.point;
	}

	void FindTriangleMaterial(RaycastHit hit){
		triIndex = hit.triangleIndex;      //<-- this is always -1
		mesh = GetComponent<MeshFilter>().sharedMesh;
		subMeshCount = mesh.subMeshCount;
		matIndex = -1;

		for (int i = 0; i < subMeshCount; i++) {
			int[] curTris = mesh.GetTriangles(i);
			for(int j = 0; j < curTris.Length; j++){
				if(curTris[j] == triIndex){
					matIndex = i;
					break;
				}
			}
		}
		if (matIndex == -1) {
			burstMat = null;
		} else {
			burstMat = GetComponent<Renderer>().materials[matIndex];
		}
	}

Thanks for anyone who can help.

EDIT:
So in looking around, I have found that convex mesh colliders wont allow triangleIndex to ever work? I saw nothing on this anywhere else but if this is the case, that’s a real bummer. Verification would be much appreciated.

3

Of course it doesn’t work with a convex mesh collider since a convex mesh represents the convex hull of your actual mesh. So it’s triangles have no relation to the actual mesh at all. A convex mesh collider also works way different from a normal mesh collider. A convex mesh collider represents a volume while a normal mesh collider just represents a surface.

Keep in mind that you can use two meshcolliders if you want, one convex and one non-convex. Just setup the layers accordingly to define which one can collider with what.

However if your mesh contains submeshes the triangleIndex won’t help much i think. The MeshCollider of course doesn’t use submeshes. So the triangle index won’t tell you to which submesh the triangle belongs.

Your code also makes no sense since you compare the triangle index to a vertex index. The triangle index is simply the position of a triangle within the triangles array divided by 3 (since a triangle is made by 3 vertices)

What you can do is doing something like this (This might even work):

  • use the world space position of your raycast hit point and transform it into local space of your mesh (Transform.InverseTransformPoint).
  • Iterate through all submeshes and through all triangles (take 3 indices at a time)
  • use the vertices array with the 3 indices to get the actual positions of the corners.
  • use the 3 points to calculate the face normal (cross product)
  • use the face normal and one corner of the triangle to project our hit point onto the triangle.
  • Check if the distance between the projected point and the actual hitpoint is very small (so the point actually lays in the plane of the triangle).
  • Calculate the barycentric coordinate of the projected hit point for each triangle. See this post over here
  • With the barycentric coordinate you can easily check if the hitpoint is within the triangle
4

Just replace your primitive Collider with a MeshCollider.
RaycastHit.triangleIndex only works with those.

6

If there is a rigidbody attached to the transform of the hit meshCollider (non-convex), the meshCollider will not return triangle indexes. Set the rigidbody to isKinematic before raycasting and then back to whatever it was.

2

Physics.Raycast returns true when the raycast actually hits something. You’re not checking this, are you sure there was a hit? Maybe you’re using the “hit” variable without any hit happening. Check that return value and maybe add a Debug.Log line to make sure it’s hitting what you think it’s hitting:

Debug.Log(hit.rigidbody.name);
5

@Escapade
Just Update MeshCollider for each hit. Assign the updated mesh to MeshCollider component. This issue may be solved. but if any others finding for solution my suggestion may be helpful.