Occlusion Culling baking on very large outdoor scenes?

Been looking into ways to improve the performance of our project, and I believe objects in the distance are not being culled.

The scenes we are are very large and outdoors, and the player object has several cameras, and some have large far clipping planes to accommodate the narrow FOV for a close zoom. I observed when the player is facing a section of the map with several buildings, but obscured by terrain, the FPS take a hit, and then comes back up when the player turns to face away from them.

I attempted to bake the culling data, after flagging all the buildings as Static Occulder, but had to abort that process after 40 hours of the dialog spinning.

Is there a better setting to apply to the buildings so they can be culled by the terrain?

Thanks