I’m experimenting using Unity’s ECS for a game that requires determinism for its network play. I’ve built a checksum system that will checksum ArchetypeChunks of a component I wish to keep track of.
Starting two worlds at the same time the internal structure will remain the same through the many update ticks of the simulation (my checksum system gives me pretty good confidence of this!)
For this game we will have players hot join a match which means that we serialize a world and send it over the network, deserialize, and checksum the players against each other. The checksum system I built is giving me different checksums for two players. The reason appears to be after a deserialization of a world there doesn’t appear to be a guarantee that the archetype chunks remain in the same order and given that the checksum system combines checksums from one Archetype into another means I get a different checksum.
I can sort the checksums before combining them to make it order irrelevant but given the deterministic goal of the Unity ECS I’m curious if this is the desired behavior of the serialization system? I could imagine doing an ArchetypeChunkJob that needs to use a EntityCommandBuffer.Concurrent and then diverging the two systems since the concurrent usage of EntityCommandBuffer.Concurrent would be non-deterministic then. Or if I did a single threaded job in archetypes accumulating a number in some non-determistic way (which I suppose is what my checksum system is doing).
Or is this unexpected and I might be doing something wrong that’s causing these to diverge?
The current Serialization code actually changes entity ID’s to be tightly packed. So it is not deterministic at this point.
There are two approaches:
Make every player that is connected serialize and read the world again. Thus everyone will be on the same checksum. This can be done without changing Unity.Entities
Change SerializeUtility.DeserializeWorld to have a flag that keeps all entity id’s tightly packed.
I’m interested in solution two but it’s not immediately obvious to me looking at the code how to accomplish that (I also don’t fully follow the issue).
Is a deterministic serialization something that Unity plans to support some point in the not-crazy-distant-future? If so solution one is alright as a temporary solution as it would unblock development.
I’ve looked at the SerializeWorld and DeserializeWorld functions but it’s not immediately clear where I would need to make a change to make it not tightly pack the entities. Could I be given a pointer here?
Check out GenerateRemapInfo on serialization side, thats where it packs generates a remap for packing the entities to be consecutive.
Then on the deserialize side you have to fix all the fall out from the ids no longer being consecutive. (AllocateConsecutiveEntitiesForLoading)
There is a bunch of code that depends on that assumption later on in that deserialize function, all has to be patched up. Hard to predict which ones, I expect many unit tests will fail when you get it wrong though
Definiately not a simple change if this is your first time modifying the guts of Unity.Entities.
I might toy with getting it working or go with solution one for now but I feel like I might possibly be asking a dumb question but would I copy the files (and tests) over from the preview package location in the library folder? I didn’t see a github repository for Entities.
