Hello, I’m working on a strategy game, and after some research it appears that serialization is going to be a big problem with my current code.
It is a building game with lots of references to class instances:
Buildings have references to the road instances that are connected to them, and vice versa
Roads have references to the RoadSegment instances they’re made up of, and vice versa
Roads have references to the Car instances that are currently driving on them, and vice versa
Cars have references to their home building and destination buildings, and vice versa
etc. etc.
Am I right in thinking that these nested and circular references are problematic for serialization? My plan to simplify this is to keep all instances in flat lists. Looking at buildings:
There would be one big list of all building instances
Every object that needs a building reference stores a list index instead
When buildings are removed, a ListChanged event is broadcast
Every object that holds a building reference subscribes to this event and decrements their list index by one if it’s greater than the deleted index
I’ve started implementing this, but it’s really cumbersome. Because I have a lot of custom classes and would thus need a lot of these flat lists.
And I can never forget to subscribe to all of the relevant events. Doing it this way, I can’t quickly add references anymore. Every time I introduce a reference I’ll have to add a ton of code to make sure that the list indices get updated, it seems excessively complex. Is there a better way to do it?
Instead of depending on indexes, you can use dictionaries. Look up an item using a key, string key is probably the most common. Your individual objects can then hold a string key to their reference instead. This way you are nolonger index dependent, making the removal step a little smoother.
Of course dictionaries can’t be serialized, so you’ll need a custom class to store that data, though using 2 lists should be sufficient. On load you can then populate the actual dictionary you are using with the list pair.
Ideally a save system only runs twice, on save and on load. There is no reason to update this data at all, and new data is submitted when a save is requested. If the references in-game are class references, I can’t picture how indexes will matter. You can just rebuild the lists on save; new saves generate new indexes/index links, and none of the indexes matter after loading.
Don’t use the binary formatter/serializer: it is insecure, it cannot be made secure, and it makes debugging very difficult, plus it actually will NOT prevent people from modifying your save data on their computers.
When loading, you can never re-create a MonoBehaviour or ScriptableObject instance directly from JSON. The reason is they are hybrid C# and native engine objects, and when the JSON package calls new to make one, it cannot make the native engine portion of the object.
Instead you must first create the MonoBehaviour using AddComponent() on a GameObject instance, or use ScriptableObject.CreateInstance() to make your SO, then use the appropriate JSON “populate object” call to fill in its public fields.
A flat list with a string or int “index” on each object is the best approach. This is basically what Unity does with its GUIDs, but you can’t control a GUID on start so you’ll have to essentially implement your own GUID.
This may be the main issue here. If you’re reimplementing this for every type of class, that’s a bad approach. You’d be better off with a single* list containing all of your objects, and the classes should derive from a common type. This way, you only need to implement this once.
*You might, for optimization, later divide up your list, perhaps by region or something. But, you wouldn’t divide it by type.
The way I had it planned was to not use class references anymore, the idea was to directly work with the lists (or dictionaries as you suggested) in-game. Not sure if this is a silly way to go about it, it just seemed that since I’ll have to convert the data for saving anyway, I might as well work with the same structure at runtime.
Currently (very simplified):
class Car{
Building homeBuilding;
Building destinationBuilding;
}
Planned:
class Car{
int homeBuildingID;
int destinationBuildingID;
}
class GlobalData{
List<Building> allBuildings;
Building GetBuilding(int index) {
return allBuildings[index];
}
}
I hear you about the bad approach. I started converting my code yesterday and realized how deep this rabbit hole goes with how many custom classes I have.
I like the idea of having one big list, it would simplify things a lot. My concern is that I already use certain lists in-game, for instance allBuildings or allRoads which I sometimes have to iterate over during gameplay. If these are tucked into one huge general list I won’t be able to iterate over them anymore. Any tips on how I could maintain the ability to have “sub-lists” like that?
Your own GUID system is a must. Another key is to have a separate serialization surrogate for each object that you use for saving and loading. Finally, for all your circular references, loading should be done in three phases: one for internal setup, one for external setup, one to finalize (if you have to calculate data using references of references, for example).
Typed this here but should help illustrate what I mean:
struct RoadSerializationSurrogate
{
public GUID guid;
public List<GUID> segments;
public List<GUID> cars;
}
class Road
{
private GUID guid;
private List<RoadSegment> segments;
private List<Car> cars;
public void Load(LoadingPhase phase, RoadSerializationSurrogate surrogate)
{
switch(phase)
{
case LoadingPhase.Internal:
guid = surrogate.guid;
break;
case LoadingPhase.External:
// we know that all other objects have their GUIDs assigned at this point
// so we can re-build our references
segments = new List<RoadSegment>();
foreach(GUID guid in surrogate.segments)
segments.Add(World.FindByGuid<RoadSegment>(guid));
// ... and all the other data
break;
case LoadingPhase.Finalize:
// any special logic here
break;
}
}
public RoadSerializationSurrogate Save()
{
RoadSerializationSurrogate surrogate = new RoadSerializationSurrogate();
// fill in the data
return surrogate;
}
}
Then saving your game is just a matter of looping through every major object in your game, creating a big list of surrogates, and dumping all that to a file. That’s what I did for my 4X game at least.
public IEnumerable<T> IterateByType() where T : YourBaseType {
foreach (YourBaseType obj in masterList) {
if (obj is T typedObj) yield return typedObj;
}
}
...
foreach (var roadSegment in IterateByType<RoadSegment>()) {
// do something with the segment
}
(untested code, the syntax may be slightly off, but I think it’s right)
You could also add a condition delegate to check in your iterate function if you want to filter by something besides the type.
Or, you can maintain copies of the list for your important types, and just generate those copies after loading. That would be less simple and more bug-prone, but slightly more performant when dealing with large lists.