NavMeshAgent or alternative that has no collision

Just starting out with NavMesh and was trying to find an answer to a simple question. I want to use NavMesh to give me paths to points around obstacles in a fairly simple world but when testing it all out the collisions are being done with the navmesh and not the object colliders, is there a way to get a path and move my character around which does not collide with the navmesh but with game objects?

I guess I need to write my own NavMeshAgent script which will look at the path and move the object accordingly, is there an example of this anywhere?

I simply made the navmeshagents the same size as the colliders on my units