Better Network Serialize commands

Multiplayer & Networking
1

I’m posting this here cause I’m curious about what other Unity programmers feel about this.

Oftentimes I need to send a bulk of data to the client to initialize his data. I have 2 options for doing this: RPC (Remote Procedural Call) or a OnSerializeNetworkView function.

The problem with using an RPC is you pass the data using arguments. My data includes arrays of classes so using an RPC is problematic.

I can use instead a OnSerializeNetworkView function and serialize the data into a stream. The problem with the current OnSerializeNetworkView function is you can’t control when exactly it gets called. OnSerializeNetworkView is meant to be called continuously, and has to be attached to a certain Network View.

I need a Serialize function that would get called only when I tell it to, much like an RPC.

Having worked with the Torque Game Engine, Torque has what they call a NetEvent, basically a one-shot serialized data transmission from/to Server and Client.

This is perfect for initializing the Client with data, or when the Client needs to submit a list of information to the Server (like when buying/selling multiple items from/to a shop).

And please, if this gets implemented, please make it available to the Unity free version.

My guess is allowing a BitStream to be a valid argument to pass in an RPC is the most straightforward way to do this.
Right now I’m faking serialization by turning my data into one long string and sending that to the client. This, I guess, takes up a lot of bandwidth since strings in Mono are 16-bit each character because of Unicode support.

That’s ok, I tell myself. Just imagine the client is loading a web page. Heck, maybe even the average web page is larger than what I’m transmitting. And I’m not sending it every loop or something; its for initializing client data, so a one-time, large bulk of data transmission is quite allowable.

It makes me miss my days when coding in Torque Game Engine. You can squeeze all the bits of data you want as small as possible there. They have their own version of the BitStream: Like a normal stream you serialize your data, but the special thing is the compactness of the serialization. Transmitting a boolean requires only one bit, strings can be compressed to save space, and you can specify by how much bits an integer or float is going to take when serialized into the stream.

For example, say you have an integer variable health, which in your game goes only in the range 0 to 100. You can then specify the health variable to be serialized to only 7 bits (essentially creating a 7 bit integer, which goes in the range 0-127), since its value goes only up to 100. You saved 25 bits, assuming it was a 32-bit integer, a whopping 78% of the original size.

They even have what they call a StringTableEntry. Basically used for strings that get sent over the network frequently, like usernames or preset error messages. What happens is the first time its sent, it gets sent as a normal string, but the next time, it gets sent only by a reference ID number to save bandwidth.

I made a formal request about the whole thing here: http://feedback.unity3d.com/forums/15792-unity/suggestions/824649-better-network-serialize-commands

2

I just now realize how important is this thread of yours, to avoid the ridiculous amount of effort and time needed to transmit the most stupid and basic info from server to client and viceversa in Unity.
I have supported your request.

3

Supported.

4

RPC supports sending a byte[ ] (which is pretty much as good as a bitstream). I learned this not too long ago, when someone posted a package on the asset store for a struct serializer/deserializer that used it. So just serialize/deserialize your arrays to byte[ ]. I’m using it in my current project, so I know it works.

5

Yeah, you can send byte[ ] in RPC.

6

As previously stated - simply serialize it yourself and send the bytes. Using the binary formatter in the .net framework isn’t hard (though it may take a few hours).

7

This must be something new. I actually tried doing this a while back and it wasn’t supported. It’s a welcome addition.

This route is very interesting and you can go really far with it until you hit some crippling limitations. To obtain a minimum of ordering, all your RPCs must be sent this way and you must use reliable delta compressed. Not trivial, but can be done. Then, you must create your own implementation of Network.Instantiate using this new kind of RPC. This is a bit tricky since there’s no simple way of serializing references to prefabs.
Finally, you must realize that, with this method of communication, you can’t send a RPC strictly to one peer - all messages will be broadcast to everyone due to the way synchronization works, wasting a lot of bandwith in these situations (the only situation that works is when the client sends a message to the server only).
If even then you still want to venture forth, you shall not use state synchronization with reliable delta compressed for anything else IF you want players to join your game in the middle of a stage. In these situations, the server will send the object’s state before you have a chance to instantiate it using your custom RPC because you MUST instantiate synchronized objects during the server’s OnPlayerConnected. After this event, Unity sends the object’s initial state to the player that just connected, and if it’s not instantiated, it will lose the initial object’s state and it’ll not synchronize further. This is problematic even if you buffer the instantiated objects and instantiate them yourself later.
There are a lot of other tweaks that must be done for this to work in order to keep latencies in check. Also, I suspect that there is a somewhat low limit on the total number of bytes that you can send during OnSerializeNetworkView, limiting the number of RPCs that you can bundle together. Anyway, I implemented this and got lower latencies than Unity’s native RPC (around 2-3 ms for a round trip, nothing spectacular) and around 80% the bandwith usage, disregarding the broadcast problem cited above, with many benefits in usability: I was able to serialize component references, gameObject references, data transfer objects, lists, maps, etc.

8

Yes, I’ve also read about the byte[ ] trick for RPCs. Thanks for sharing it here also guys!

9

i don’t get that how can i send a gameobject instance over network via RPC?

10

I’m looking at this situation as well, my though for serializing objects across the wire is that you could use a tag or just use a lookup table (digest) that allows you to map objects to key values. My issue is that I want two separate versions of prefabs for server and client, using a single codebase and jamming all this code in it is one solution, but a messy and wasteful one. I’d rather use two separate projects and on the server side have one sec of objects (i.e. monsters will have AI scripts, collision detectors for LoS, etc), while on the client another (i.e. same monster prefab will not have any scripts, be only a model and animations for instance with some prediction code perhaps for networking).

Seems to me the only way to handle this atm is to use a dictionary and attach to a scene object, then have it map string → prefab for doing lookups.

11

The other thing you can do is have a script that checks on instantiation if it’s a server or not, and then just do a Destroy(scriptreference) to change the behaviour of the prefab.

so like:

var serverScripts : MonoBehaviour[];
var clientScripts : MonoBehaviour[];

function OnNetworkInstantiate()
{
    If (Network.IsServer)
{
   for (var s in clientScripts)
{
   Destroy(s);
}
}
   else if (Network.IsClient)
{
    for (var s in serverScripts)
{
    Destroy(s);
}
}
}
12

Of course, this is spaghetti code at it’s finest though - mixing client/server semantics within the same class AND muddling behaviours of each so you’re left with one mess.

I’ll mock something up over the weekend and see what it looks like.