Unity is not just a “representation technology”. It’s a game engine. This includes rendering, of course, but also things like physics simulation, pathfinding, useful data structures for gamedevelopment (like Vectors and all available methods on them, but many more) and so on.
If you want to programm a game independantly of Unity, nothing stops you from doing so. You can just create a game the ‘old school way’, programming the OOP object hierarchy yourself. If you want it to be truly independant and only use Unity for rendering purposes tho, you will need to not make any use of things like Vector3, or other tools like pathfinding. You will then also need a way to convert your custom Vector types to Unity types for displaying them in Unity. You will also needs to create one interface / manager object, which reads your custom game data and converts it into gameobjects. So sure, it’s possible. But if you want to go these lengths, why not simply add a renderer and create your own engine?
Are you familiar with normal object oriented programming? Whatever you do, you will have a Main() method somewhere. Everything that happens, happens inside of this main methode. The rest of the OOP code is just structures to help you. you can simulate this by simply using the Start() method of a single gameobject to replace a Main() method. You can additionally use Update() to call some other code to tell your own game structure that one frame passed.
You wrote that you’d like to have some scripts running in the background and just visually update whats happening. However, how would that work? You either calculate things as fast as possible, or need to pause some time to reach some desired target updates per second. The first would just waste all kinds of ressources, which is bad design. The second is basically… what Unity does. A frame is nothing more than calculating each thing that needs to happen regularly once, then optionally wait until some time passed to reach a lower target framerate. Then repeat this cycle. You may want to take a look at this:
Each object needs to be initialized and then have some update cycle. What you are talking about (things happening ‘independant of the framerate, and visuals just updating what’s happening’) is basically what FixedUpdate does. The physics simulation happens in fixed timesteps and not necessarily every frame. This is an optimization. If you want to develop your own backend for a game and only use Unity for visualizing this, you would also have to think about all of this yourself and implement your own solutions. The effort required for this would be huge.
I also believe you have some general misconceptions. You wrote “this would allow to have entities be active without needing to add them to the scenegraph”. Meshes already only get rendered when they are seen by any camera. So invisible objects, objects behind the camera, or faraway objects already will not be rendered. However, if you plan on fully disabling them, that would be equal to disabling the whole gameobject. If you do not want to waste CPU on some entitiy, then you will also not update it. Thus you cannot know if it walked into the camera view again, or depending on how you do it, you wouldnt even know if you moved the camera to look at it, so you had no idea when to render it again, or would potentially render an un-updated version of the object.
So to conclude my answer: you can write all the custom data structures you want and recycle them, perhabs even between different engines. At some point however, you will need to interact with Unity. It’s also probably a good idea if you think about how you would implement these things differently - because just having scripts run in the background without any limiters is hardly any good design. These objects / scripts will need to have an update rate. This update rate ideally needs to be in sync with other update rates, and in a fixed order relative to them. This is what a framerate is.
I believe perhabs you have the wrong idea about how games work in general (which is a weird thing to say, i know). Games dont exist between frames. Nothing happens between these snapshots we see, and if something happened there, we wouldnt see it. Games are literally just snapshots of something calculated in the background, determined by some samplerate which is the framerate. The game world does not change between two frames, it changes only with each new frame.
