Grouping vertices?

Hey everyone.
I have a model that, when in Max has only 1,002 vertices. When exported into Unity, it has 5,722. This is (from what I’ve researched) due to the vertices requiring multiple normal-facing directions to allow it to render hard edges and flat faces (exported intentionally without any smoothing groups).

Basically what I’ve written so far shoots a raycast from the mouse position to the mesh, and from the hit point, gathers the closest, say 80 verts. It then iterates through all the verts and grabs any that share the exact position of the chosen 80. All these verts are then stored in a different array and operated on.

What I’d like to know is if there is any way to group all shared vertices in something simple so I can simply reference them all at once? Ideally, something like a hash-table that stores multiple values per key. All I can think of so far is creating a 2D array to store their int position in the main vertices array, but that would still require iterating through the entire thing to find what group a vert belongs to.
Any ideas?

So it seems like (strangely enough) the most efficient way is to just not group vertices and iterate through them every time. I’ve tried practically everything that I could find (lists of lists, multidimensional arrays, jagged arrays, arrays of arrays), but none of these options run anywhere near as fast according to the profiler.