Enemy evade sight of player

I have coded of far: an Enemy is randomly spawned in an area around the player on a terrain. After the spawn it standing around in idle mode until the player see the enemy and then the enemy is chasing the player. If the player is too far away from the enemy and still standing in idle mode, he will despawn and respawn on a new location around the player. Now i want to add: when the player aim with the flashlight onto the enemy, the ai should find the fastest way out of the player sight. (Like go behind a tree)
I have a cone trigger on the flashlight and when the enemy is inside the trigger it fires a raycast from the enemy to the player and checks if something is between them. If not, it should go behind the nearest obstacle or terrain tree. How can i archive this? Thanks in regard.

How this is usually done in AAA games depens on your pathfinding implementation.

The simplest is using A* or a similar grid system. In such a grid system you would simply raycast from the player to every point in the grid, called nodes, in a certain range to determine wether or not this node is hidden. Now you just tell your AI to go to the nearest node that is marked as hidden.

Instead of using a bool for when a node is hidden, you want to use tags. This allows for more conditions, such as “this node is behind an obstacle” and "this node is outside the flashligh but it is the FoV of the player. Then let your AI decide the most desireable node based on time to reach, and safety from the light.

If you are using a navmesh, which is what the builtin unity AI uses, you will have to use some complicated math to split up the navmesh into sections that are hidden and not. Still your AI would not behave how you think, as it doesn’t differentiate between 1m beside the flashligh cone, and behind some obstacle.

If you are using navmeshes, I would suggest getting some grid based system, like A* Pathfinding by Aron Granberg on the asset store, and use it ontop of your navmesh, or replace your navmesh system entirely. It’s not very difficult! I have included some old links from when I have answered this question before.

Unity Answers
Better explained, overly detailed implementation