Culling inside an inverted cube

I have a question about culling, if the camera is inside an inverted cube which acts as a house with no hole (windows), the objects outside the cube are still rendered, how can I fix it?

Use occlusion culling.
Unity’s solution is a bit CPU heavy for VR, but works.
There are some solutions on the asset store which are faster (check for VR compatibility).

If you work with seperate rooms manually enabling and disabling the whole gameobject is the fastest