Nav Mesh Agent won't fall off the platform?

I did the Unity tutorial “Survival Shooter” which used Nav Mesh Agents (http://unity3d.com/learn/tutorials/projects/survival-shooter).

Now I’m using tools that I learned to create a different game. Problem is that I can’t get the blue cubes with Nav Mesh Agents to fall from my platform. I have tried using the Rigidbody AND the Mesh collider but neither makes it work. The blue cubes stop to a invisble wall, even though there is nothing to stop them from falling (but I guess the baked Nav Mesh somehow makes them stop at the edge).

Is it possible to use Nav Mesh and still use physics to make them fall from the ledge? Idea is that the blue cubes should follow the red cube, but they should be able to pump to one another and fall off the platform.

I also tried to expand the baked Nav Mesh over the platform but with no results; the cubes float in the air…

Attach a rigidbody with collider to the navmesh agent. Mark the rigidbody as kinematic = true and useGravity = false. When the navmesh agent is supposed to fall, disable the navmesh agent, make the rigidbody isKinematic = false, and add a force to it to push it outside or useGravity = true.