NavMeshAgent won't move at all if it's destination point is a Not Walkable area (And if intended, how to find the closest "Walkable" point?)

I’m new to Unity and trying out NavMeshAgents for pathfinding in an Isometric ARPG style game. So far I have it so that whenever I left click on the Ground my Player will navigate towards that point, avoiding anything in the way. However, if that point is on a “Not Walkable” baked area (An Obstacle for example) then the player won’t even attempt to move there at all. Is this intended? And if so, how would I go about finding the closest “Walkable” area to that point? I’m really struggling with this, and I’ve searched around a bunch and can’t seem to find an answer.

Here is the code so far. How would I get the closest “Walkable” point to my SetDestination(point)?

if (Input.GetMouseButtonDown(0))
        {
            Ray castPoint = Camera.main.ScreenPointToRay(Input.mousePosition);
            RaycastHit hit;
            if (Physics.Raycast(castPoint, out hit, Mathf.Infinity, groundLayer))
            {
                var distPoint = new Vector3(hit.point.x, hit.point.y, hit.point.z);
                point = distPoint;
                playerAgent.SetDestination(point);
                playerAgent.transform.LookAt(point);
                isNavMoving = true;
            }
        }

Did you try to precalculate the Path. It works fine for me. You just write like

if (Input.GetMouseButtonDown(0)) { navAgent.CalculatePath(point,navAgent.path); /*bool*/ pathComplete = false; }
Then out of the box you ask

if(pathComplete == false) {
if(navAgent.status == NavMeshPathStatus.PathComplete) {
navAgent.SetDestination(point);
pathComplete = true;
   }
}