I’d like to sync the state of a large, modifiable voxel world made up of millions of voxels between a server and multiple clients.
In a test project, I setup a ghost with a DynamicBuffer of 100 int3 (represents the positions of voxels). Every 2 seconds on the server, I assign random values to the entire buffer (represents potential changes to the world each frame). This works but I can see some issues. First is the SnapshotDynamicDataBuffer on the client ghost is massive(65MB). Second is the packet size.
I know I can prolly use RPC or ICommandData to avoid these issues. But I’d also lose the ability to rollback ? afaik, the predication / correction only works on ghosts
How should I go about syncing large data set while preserve the ability to rollback?
Thank you
hey!
so give that you sending 100 int3 we are speaking about 10034 bytes that Is 1200bytes of data (that is 9600 bits).
Then we need to add: 300 bits for the change mask, plus another 2 control bits, plus any change masks for other component present on that entity (for sure the 1 bits for the position, 1 bits for the buffer len) plus quaternion and position (that does not change but the first time they are sent).
So we are around 10000 bits, not encoded. We apply Huffman coding. The encoding bits are tailored for small data ranges (that uses less bits). But for larger values there aren’t particular gains (actually more bits in certain cases).
If the buffer size remain constant (so always 100), we are kinda able to delta compress the content but if the len change we don’t.
When you change the data in the buffer with a new set of random values, even though we are delta compressing the values against the latest confirmed packet received by the client, the delta it is not going to be small if the value are completely random and not correlated.
As such I can say it is like not having delta compression at all and every time a new full set of data is sent (so always 10000 bits pretty much).
However, for the frames in between the data change, we should be able to send less data (so I’m expecting at some point the size should be a little smaller).
Until the server does not receive from the client the ack for the latest packet with the new changes, the sent data will continue to be high.
With local loopback, that should be in general vary in between 1-4 ticks (because we don’t know when the OS decide to deliver the data).
But since you are speaking about 2 seconds my expectation would be a spike (10000 bits)) for a couple of frames and then lower (close to 400 bits or so).
If you set the block ghost to be static this can be optimised even further (since no data at all is changed until you change the buffer).
The size of snapshot buffer on the client though seem really too big though. We round it to the closest power of two, so would expect 65KB or even 512KB but not 65MB. That need some investigation.
Can you share please your ghost component setup?
1 Like
you are right. it’s indeed 65KB instead of MB. was a brain fart on my part.
and data won’t be completely random in the real project.
so I may just make this work with ghosts.
thanks very much