NavMeshAgent-obstacle avoidance but permit agent to pass by each other.

I know this might sounds ridiculous but what i need is the agent actually performing obstacle avoidance but at the same time they would not collide and stuck.Any clue?

Set the radius of each NavMeshAgent’s (in your prefabs) to a very small number

Don’t forget to use NavMeshSurface - was introduced in 2017, by Unity team.
The trick is to bake several surfaces for various agents (there is now an additional “Agents” tab in Navigation window). In there, you could assign ‘fake’ radius to each such an agent, say 1.6, etc, but in reality make each navMeshAgent’s prefab have something like 0.0001 radius.

This will ensure the agents with tiny radii do not walk too close to the walls.
Also, a little more info about NavMeshSurface sampling:

Also, you might encounter issues when radius is very small (0.00001). It’s the best value to use, however unity will complain that agent wasn’t placed on the NavMesh.
To solve this, first allow character to reach the surface, and build the first nav-path. Once it did this, you can set the radius to 0.00001 because character is guaranteed to be on the surface.