Rendering objects behind only Behind or Inside Stencil

As the title suggests, I need to render objects only when behind or inside a gameobject (in this example I use a cube mesh that writes 1 to the stencil buffer). I am making a top down game with an isometric like camera except I can rotate it around the player. When the player enters a room I want to change renderer to have an interior renderer that culls everything outside the visibility of the room. to achieve this I use a stencil buffer that is written on by a cube gameobject that takes the entire place inside the room.

The only problem now is that objects that are outside the room but pass the stencil test as they are between the room and the camera, are still being renderer. Lets say I have a building outside the room, if this building is in front of the room itll get renderer.

Here is a video of something similar to what I am trying to do :

Here is a picture of my game and some problem I have, Id love to show more pictures for more context but I cant :confused: