Enemy AI gets stuck in corner when fleeing.

I have a very simple script for making my enemy AI flee from the player. However, it flees into the corner and gets stuck. As the objective of the game is to catch the enemy AI, I want it to avoid being caught by the player as long as possible.

I do have a sphere Collider in the enemy, but do not know how to use it to aid in pathfinder/avoidance. I am using Nav Mesh Agent for navigation and pathfinding. I am new to unity, so I appreciate any help you can give.

    NavMeshAgent agent;
public Transform target;
private SphereCollider notice;


void Start () 
{
	agent = GetComponent<NavMeshAgent> ();
}
	
void Update () 
{
	Vector3 direction = (target.position - transform.position).normalized;
	float fleeDistance = -10; //How far AI will flee
	agent.destination = (transform.position + direction * fleeDistance);
}

}

While baking the navmesh make sure the “Min Region Area” option under the Advanced settings is equal to or bigger than your enemy AI’s collider radius.

no, unfortunately the object still moves for the corner and and stays there until the player is standing in practially the same spot as it is. after which of course it will flee for the next available corner.

When Fleeing to not get stuck at Walls and Obstacles I solved it, see below:

Some conditions:

  1. You need to tag the obstacles that you want to avoid. In the code example the obstacle objects are tagged as “Wall”, but feel free to change to your preference.

At the end of the below code your enemy will get exactly the same SetDestination result, just it will not go to corners it will rotate away from them.

And the code:

//We will check if enemy can flee to the direction opposite from the player, we will check if there are obstacles
                bool isDirSafe = false;

                //We will need to rotate the direction away from the player if straight to the opposite of the player is a wall
                float vRotation = 0;

                while (!isDirSafe)
                {
                    //Calculate the vector pointing from Player to the Enemy
                    Vector3 dirToPlayer = transform.position - Player.transform.position;

                    //Calculate the vector from the Enemy to the direction away from the Player the new point
                    Vector3 newPos = transform.position + dirToPlayer;

                    //Rotate the direction of the Enemy to move
                    newPos = Quaternion.Euler(0, vRotation, 0) * newPos;

                    //Shoot a Raycast out to the new direction with 5f length (as example raycast length) and see if it hits an obstacle
                    bool isHit = Physics.Raycast(transform.position, newPos, out RaycastHit hit, 3f, layerMask);

                    if (hit.transform == null)
                    {
                        //If the Raycast to the flee direction doesn't hit a wall then the Enemy is good to go to this direction
                        nvAgent.SetDestination(newPos);
                        isDirSafe = true;
                    }

                    //Change the direction of fleeing is it hits a wall by 20 degrees
                    if (isHit && hit.transform.CompareTag("Wall"))
                    {
                        vRotation += 20;
                        isDirSafe = false;
                    }
                    else
                    {
                        //If the Raycast to the flee direction doesn't hit a wall then the Enemy is good to go to this direction
                        nvAgent.SetDestination(newPos);
                        isDirSafe = true;
                    }
                }