I have 1 array for the enemies to find the player units this is an RTS game.
My current way of finding the nearest target because there is just so many units in the game is using a list like so:

void DistanceToTarget()
	{
		//Sort so nearest enemy is always Enemies[0]
		PlayerUnits.Sort(delegate( Transform t1, Transform t2){
			return Vector3.Distance(t1.transform.position,transform.position).CompareTo(Vector3.Distance(t2.transform.position,transform.position));
		});
	}

void TargetedEnemy()
	{
		DistanceToTarget();
		//after sorted nearest enemy will be 0
		SelectedTarget = PlayerUnits[0];
	}

The problem with this is my threshhold is only around 80 units on both enemies/player sides before the game starts to eat bricks. I use a collision box around the enemies for detection range and when entering this happens so if my units stay away from the 80 enemy units the game runs fine…but i need this to work IN combat and run fluidly.

Edit: here is things already tried based on using list method…
1: LateUpdate to help on performance (still goes slow but the fastest so far)

void LateUpdate()
	{
		//if (PlayerUnits.Count != 0)
			//TargetedEnemy();
	}

2: OnTriggerStay causes a complete system lock up…

//Causes app lock up?!
	/*void OnTriggerStay(Collider other)
	{
		if (other.tag == "SelectableUnit" && PlayerUnits.Count != 0)
			TargetedEnemy();
	}*/

This is why i would like to see an actual different method entirely I do not know of any other way to do this :(…

3: Fastest by a long shot but causes enemy to target last playerunit first not the nearest playerunit…

	bool isRunning = false;
	IEnumerator AlreadyRunning(Collider other)
	{
		isRunning = true;
		EnemyAITarget = GetComponentInParent<EnemyAI>().SelectedTarget;
		if (EnemyAITarget != other.transform)
		{
			EnemyAITarget = other.transform;
			GetComponentInParent<EnemyAI>().SelectedTarget = EnemyAITarget;
			other.GetComponent<NavMeshAgent>().Stop();
		}

		yield return new WaitForSeconds(0.5f);
		isRunning = false;
	}
	//PhysX
	private void OnTriggerEnter(Collider other)
	{
		if (other.tag == "SelectableUnit")
		{
			if (!isRunning)
				StartCoroutine(AlreadyRunning(other));
		}
	}

	//PhysX
	private void OnTriggerExit(Collider other)
	{
		if (other.tag == "SelectableUnit")
		{
			if (!other.GetComponentInParent<RTSController>().isMoving)
				other.GetComponentInParent<RTSController>().isMoving = true;
		}
	}

With some assumptions:

  1. All the units are on the same plane XZ.
  2. I will name “Hero” the unit that will try to find the nearest enemy to target.
  3. Our hero only wants to target units inside a 1x1 square around him.

Something you could try:

  1. Keep all your enemy units sorted on a list. But sort only comparing X coordinates on the world.
  2. If your world starts at (0, 0), the first units on the list would be the ones with X near 0.
  3. It should be only one list like this, accessible to all your heroes. And the list should be sorted only once per frame.
  4. If your hero is at position (Hx, Hz), search the list for the first enemy with X: (Hx - 1 <= X <= Hx + 1).
  5. Now, for each enemy afterwards, start looking for those with Z: (Hz - 1 <= Z <= Hz + 1), and compare distances only between those that match the criteria and are inside your square. Stop advancing through the list when the enemies’ X positions go outside your limit (Hx -1 <= X <= Hx + 1).

Maybe this could be tweaked a little more, but the basic idea is discard several units with simple math before doing more complex distance measures.

Also, measuring a Vector3 distance involves a square root operation, which is slow if used frequently. Sometimes you could avoid it by computing your own vector length without the square root and comparing the result to distance * distance, or using Manhattan Distance.

---- EDIT:

There are some good advices in the comments and answers. I believe that you went for the solution of continuing to use the OnTriggerEnter and OnTriggerExit functions, adding some extra logic to avoid re-targeting new units. Is that right? It would be nice to know since this seems an interesting question.

So, I suppose the physics engine inside Unity is optimized in such a way to avoid checking for collisions between units that are far away, and this could work in your case, avoiding extra math operations against all enemies. As an additional note, read about collision layers (here and here), because they can help you out to differentiate friendly units from enemies and avoid checking for combat proximity against undesired objects.

I had a roughly similar problem: i had many units in scene with their own AI, and each of them had to know who is the nearest enemy to attack or flee. I solved it by following steps:

  • i was searching nearest enemy not per-frame but once per some seconds (scan period was chosen experimentally, 2…3 seconds was enough usually). all other frames i used cached target (usually it doesnt matter if i choose not first but second nearest object, if they are near each other)
  • at startup (in awake) I randomized scan period to +/-20% for each object to avoid periodical freezes when all units trying to scan nearest enemy at same time.
  • to scan space around i used SphereOverlap with layer-mask (to ignore non-unit objects for reduce physics calculations), because i could call it just when I need (once per some seconds, as I wrote). this was significally cheaper than trigger-colliders (because each collider have to check collision every fixedUpdate).