Memory management with extreme amount of data

I am creating a universe with billions of galaxies. Each galaxy has billions of stars. Each star has its own set of planets and other orbiting objects, each planet having their own characteristics (mineral and atmospheric makeup, orbits, moons, possible lifeforms, etc.)

However to even have a one dimensional array tracking just the name (in floats) of 1 billion systems:

1 billion elements * 8 bytes per float = 7.45 gigabytes of memory

So unless I am missing something pretty fundamental this is impossible. The only possibility I can think of is not having the entire universe loaded into memory, but loading the local area that the player is working on into memory from a file. I don’t have much experience with this kind of structure. There would be an awful lot of reading from file, loading it into memory, making the changes, then overwriting the source data, loading entirely new section of data, wash rinse repeat. As far as the visual aspect of it, I was just planning on having the thing that the player is working on visible, so there would be very little requirements there.

Is this a bad idea? Are there any examples of games that deal with such extreme amounts of data or is it just outside the boundaries of whats possible?

I think you may be approaching this from the wrong angle. What you are looking for may be something more like Elite, where the entire system is generated randomly. It’s based on a fixed Seed so the generated systems are reproducable, but then you have functions that, instead of reading an array or a file, return a random value.

It’s a little more complex than what I’m saying but this is probably how you should approach it. It’ll reduce your dataset massively.

You can still random generate if you are scripting. You’d need to be very careful with your random gen code, but you’d have a stable system afterwards. This said, you would need to script your game around the generated system. Developing species is more of a story thing- don’t expect to create an AI that genuinely develops itself whilst the player gets through the story.

OR, Create your “Fixed” parts of the system, and Randomly generate the rest- the parts where no story happens.

FOr positioning AI that “moves” with time- You should perhals invoke the random generator in such a way that at X universal time, the AI is at <x,y> random place, but at Y Universal time (which should be significantly after X), the AI is at <x2,y2> - for all times in between, use a bit of tweening to get a position between these two points. This way the AI will appear to move between points as time progresses, without you needing to worry about their actual position.

Good luck.