Of course right after I create this, I find this other post
Think this might be what I am looking for? It looks like I misunderstood what the NavMeshAgent radius setting does and what the Navigation radius setting does.
I assumed that the NavMeshAgent radius would respect the boundaries of the mesh surface but looks like that might only be the case for other objects using NavMeshAgent?
I think in order to get rid of the clipping I am seeing, seems like I would just need to tinker with the radius in the Navigation window