I am quite new to Unity and C# and am looking for some general advice on how to deal with large numbers of game objects.
What my software needs to do:
-
Track 1000s+ unique game objects which represent real life (live) satellites in space.
-
Users need to be able to click into a graphical representation of each satellite to access data about them. The scene itself will be a 3D representation of space.
What is the best Unity way to store the data in about each satellite in memory which enables quick lookup access and can handle the potential high number of satellites without memory bottlenecks? Which data structure(s) would be most effective in this scenario when using Unity?
Please note that the number of satellites being tracked may vary dramatically from moment to moment at the data to be displayed will depend on live data and very high numbers may need to be processed at any given moment. Perhaps I have missed the appropriate tutorial/thread. Can anyone help point me in the right direction?
There is no appropriate toturial/thread for something like this that I know of. 
You should choose a native array for this, I think. But it requires that you take a decision on the upper bound of the number of elements to be stored. Even if you don’t know how many satellites will be tracked at a given time, do you know what the worst case scenario is? If you know that your program will never track more than, say, 5000 satellites at a time, it will likely be more efficient to just declare an array that big to begin with. This Wikipedia-article, for example, states that about 3600 (man-made) satellites are currently in orbit, including non-functional ones. Maybe that number would be a good starting point?
It might seem a little silly at first to just declare an array big enough for all of them all at once, instead of going with a List of maybe 100 and then expand it as your need grows. But there is a very specific reason for this suggestion: Unity’s garbage collector. Since we’re still on Mono 2.6, Unity’s Garbage Collector is unable to compact and defragment de-allocated memory so it can be returned to the OS. It will remain allocated to the process as your program runs, and just have huge holes in the process’s heap memory. Therefore, Unity developers should be very careful not to constantly re-allocate large amounts of memory, because doing so will cause Unity to ask the OS for more memory for its heap as it won’t be able to fit the new requirement for contiguous memory into any of the holes in its heap. Therefore, Generic Lists that contain 1000’s of elements run the risk of fragmenting the heap and continually increase memory usage whenever they re-allocate their internal array to change size, because Unity’s garbage collector is unable to return the previously used memory to OS.
A native array does not have this serious drawback. It is allocated once, and then just sits there for the entire lifetime of the object. You may then clear the array and re-fill it with new objects as much as you please; it will never be re-allocated unless you yourself do so with the ‘new’ operator.
In addition to this advantage, native arrays do not concern themselves with keeping elements sequential like Lists do. So, as long as you can keep track of the indices of your individual satellites, a native array will offer you both O(1) lookup and O(1) addition/removal, which you will definitely need if you’re going to be editing this array a lot based on incoming data.
So, go with an array.