An optimizing problem in finding vertices inside drag area

Hi everyone,

Many of you will be familier with various 3D tools like 3D max, blender, etc. I’m trying to make a one similar function of them in Unity. The function is mouse drag → select vertex.

I don’t need to highlight the vertices. I just need to get the mesh.vertex indices which are inside a rectangle.

I’ve made the process in this order :

  • Make a rectangle by mouse dragging
  • Convert all vertice in mesh to Screen position
  • loop the screen position array and check whether it is inside or not

This works, but the problem happens when I have huge meshes. The given 3d mesh has more than a million of vertice. This make the whole process to be complete in about 0.3~0.4 seconds, but I need this to happen almost immediately.

After my trial, I realized that the normally used 3d modelling tools are very rapid compared with my approach. Does anyone have a good idea in optimizing this??

Make a rectangle by mouse dragging
Convert all vertice in mesh to Screen
position loop the screen position
array and check whether it is inside
or not

That’s not a good approach. It’s way faster to do the opposite. Transfer your selection rectangle into the local space of your mesh and just check the vertices against the 4 boundary planes of that frustum. To get the frustum planes, you would use:

  • [Camera.ScreenToWorldPoint][1] with all 4 corners. Make sure you pass a positive, non zero z value. Usually the near clipping plane, but it doesn’t really matter
  • Do the same again with a large z distance (usually the far plane distance)
  • Use [InverseTransformPoint][2] of your target object and convert those 8 corners into local space.
  • Now you can simply construct 4 [Planes][3] out of 3 corner positions to form a mathematical plane. Be careful, the order of the 3 corner positions matter when you pass them to the Plane constructor. If you have the wrong order, the plane would face the other direction. You want the planes to face outwards. So the winding order of the corners should be clockwise when you view the side of the selection cube / frustum from the outside. At least I think that should be right :slight_smile:
  • Once you have the 4 planes, you can simply iterate through the vertices and check the “side” with [GetSide][4]. If a certain vertex is in front of any plane, it’s not in the selection volume. So you can directly skip that vertex. Otherwise if all 4 planes GetSide calls return false, the point is in the volume

Note: For performance it’s important that you actually cache the vertices array of the mesh in a local variable. Never use something like mesh.vertices*. Everytime you access the vertices property of the Mesh, you would create a copy of the vertices array. So if you have a mesh with 1 million vertices and you iterate through all of them and each iteration you access mesh.vertices, it means you create 1 million copies of that array.*
Maybe that’s already your main issue? We don’t know since you didn’t show a single bit of code.
Of course if you need / want to do the check often on the same object, you probably should cache the vertices array in a member variable of your class. So you don’t read / create that array each time. Planes are extremely cheap. They are just tiny structs. Essentially just 4 float values like a Vector4. They don’t allocate any garbage.
So assuming we have already have the vertices array:
// given
Vector3[] vertices;
Transform obj;
Vector2 screenStart;
Vector2 screenEnd;
// can be cached as well
Vector3[] corners = new Vector3[8];
// the main camera or whatever camera you want to use.
Camera cam = Camera.main;

// get the 8 corners in screen space.
corners[0] = corners[4] = new Vector3(screenStart.x, screenStart.y, 1f);
corners[1] = corners[5] = new Vector3(screenEnd.x, screenStart.y, 1f);
corners[2] = corners[6] = new Vector3(screenEnd.x, screenEnd.y, 1f);
corners[3] = corners[7] = new Vector3(screenStart.x, screenEnd.y, 1f);
corners[4].z = corners[5].z = corners[6].z = corners[7].z = 10f;
// transform those 8 corners into local space of “obj”
for (int i = 0; i < corners.Length; i++)
corners = obj.InverseTransformPoint(cam.ScreenToWorldPoint(corners*));*
// create the 4 planes (p0=top, p1=right, p2=bottom, p3=left)
Plane p0 = new Plane(corners[0], corners[4], corners[1]);
Plane p1 = new Plane(corners[1], corners[5], corners[2]);
Plane p2 = new Plane(corners[2], corners[6], corners[3]);
Plane p3 = new Plane(corners[3], corners[7], corners[0]);
for(int i = 0; i < vertices.Length; i++)
{
var p = vertices*;*
if (p0.GetSide(p))
continue;
if (p1.GetSide(p))
continue;
if (p2.GetSide(p))
continue;
if (p3.GetSide(p))
continue;
// vertex is inside the selection volume.
// maybe add it to a list?
// list.Add(p);
}
Note this code assumes that screenStart is the top left and screenEnd the bottom right corner. If you want to allow an inverted selection (so start is bottom left and end is top right), you have to first “sort” those coordinates. Keep in mind that screen space has its origin at the bottom left while GUI space has its origin at the top left and is flipped vertically.
*[1]: https://docs.unity3d.com/ScriptReference/Camera.ScreenToWorldPoint.html*_
[2]: https://docs.unity3d.com/ScriptReference/Transform.InverseTransformPoint.html*_
_[3]: https://docs.unity3d.com/ScriptReference/Plane.html*_

_*[4]: https://docs.unity3d.com/ScriptReference/Plane.GetSide.html*_