Navmesh agent ignores collider

As you can see in the video. When the agent does not have a destination, or chasing, it collides with the walls
I put in front of it. However, as soon as I give it a target, it will ignore and go through the walls.
The walls were not baked with navmesh as I plan on spawning obstacles in runtime later to stop the AI from chasing.

In fact, even baked walls collisions get ignored when I do knock backs or other actions on the agent.

how do I fix this problem? I just want it to obey basic collisions.