Should I use NavMesh Agents or Rigid bodies?

I'm writing a 3rd person action game with somewhat dynamic terrain, and I'm looking for some insight into the NavMesh agent issues. The players uses rigid bodies (which works nicely).

Ideally what I'd like to do is have a "brain" script which controls characters with the ability to swap out a player brain for an AI brain and vice versa.

For that reason I figured I'd like to have the AI just send instructions to its character instead of the seemingly RTS-like behavior of the NavMesh agent controller, which as far as I've understood it doesn't work with rigid-bodies.

Has anyone tried something similar? Something I should watch out for here? Is it possible to get "Navmesh.CalculatePath" to work with off-link paths? How about Area masks?

Found this thread:

So I figure I will create a NavMesh agent, let it run away, and chase it with a Rigid Body-based AI controller. As long as I can make sure the NavMesh agent doesn't run away too far it should give the result I desire.