Hi,
I see that prime’s gamekit plugin has a method to send and receive byte arrays. So I would like a really robust way to populate this array and of course sort it out at the other end again.
Has anyone got any information for me about this process?. Obviously it’s not just for strings 
Thanks in advance
It’s a very simple process called serialization, .NET (and Mono) has built in classes for this that will both do XML (which you can then turn into a byte[ ], but this is very inefficient) and for raw Binary (which will output a tightly packed byte[ ]).
This processes is in 99% of all cases done by simply reflecting over the class(es) you wanna serialize using the System.Reflection namespace and then attaching the proper meta data to the serialized values so you know what types they should be de-serialized as on the other end.
However, using reflection is pretty slow (relatively speaking) and when you have a limited problem space like a game I would recommend doing manual serialization of your objects, simply by defining an interface like this:
public interface ISerializable
{
byte[] Serialize();
void Deserialize(byte[] data);
}
And then implementing it on all the types that you have control over yourself, and then creating some sort of fallback for types you do not control (built in primitives, everything you don’t have the source code to, etc.) over. Either by reflecting over them as I previously mentioned or creating some utility class which allows you to register a serializer/deserializer delegate pair for a type, something like:
void RegisterSerializer(Type type, Func<object, byte[]> serializer);
And then doing something like this:
RegisterSerializer(typeof(int), (v) => System.BitConverter.GetBytes((int)v));
Obviously the delegate/lambda function would have to be more advanced for types more complex then a mere integer, but you get the point.
That is really cool, guys. Thank you both. I’m going to take a couple of days to fully digest all this and see what I can come up with.
???
I wrote a post, but deleted it 5 seconds later because yours was better. Hippos are apparently faster than I gave them credit for.
–Eric
I also wanted to mention that I’ve built a custom serializer for my SlimNet project, it’s 100% consistent between Mono and .NET and supports more advanced features (like polymorphic arrays and properties) then the built in serializer(s) in Mono/.NET. It currently only does XML but I plan to add binary in the future. It’s also open source (though the source is not available, it will be when I put up SlimNet 0.2)
You can also look in to the BitStream project (CodeProject hosts it I think) that does some relatively low-impact storage of data. Or you could write your own. But yeah, what fholm said, if you are not storing massive amounts of generalised data, e.g. not an MMO or strategy game, generic serialization can be overkill.
I would also recommend you check out some of the third-party serializers such as YAXLib that extends the basic serializes that .NET offers**. I cannot recall if YAX is available through NuGet or CodeProject.
** Or you could just break in to fholm’s network and steal his source codez. 
Depending on access permissions, I like to use a bit of Marshal to make things /really/ slick.
/// <summary>
/// Pack a struct into an array. The struct must be a primitive type or implement only primitive types.
/// </summary>
/// <typeparam name="T">A primitive type T.</typeparam>
/// <param name="val">The value to serialize.</param>
/// <returns>A byte array containing the serialized struct.</returns>
public static Byte[] Pack<T>(T val) where T : struct
{
Int32 len = Marshal.SizeOf(val);
Byte[] arr = new Byte[len];
IntPtr ptr = Marshal.AllocHGlobal(len);
Marshal.StructureToPtr(val, ptr, true);
Marshal.Copy(ptr, arr, 0, len);
Marshal.FreeHGlobal(ptr);
return arr;
}
/// <summary>
/// Unpack a struct from an array. The struct must be a primitive type or implement only primitive types.
/// </summary>
/// <typeparam name="T">A primitive type T.</typeparam>
/// <param name="bytearray">The array containing struct data.</param>
/// <returns>A struct of type T.</returns>
public static T Unpack<T>(Byte[] bytearray) where T : struct
{
try
{
Int32 len = bytearray.Length;
IntPtr i = Marshal.AllocHGlobal(len);
Marshal.Copy(bytearray, 0, i, len);
var val = (T)Marshal.PtrToStructure(i, typeof(T));
Marshal.FreeHGlobal(i);
return val;
}
catch
{
return default(T);
}
}
This automates the whole process of serializing /any/ struct, without any additional bits of code. The restrictions though, is that it requires System.Runtime.InteropServices.Marshal (unavailable on WebPlayer) and the struct can only be comprised of primitive types.
The problem with a lot of the already existing serialization libraries is that they are not 100% tested on Mono always, which at least makes me not use them. More so with the limited profile that Unity has (especially in WebPlayer and iOS/Android).
This is a very neat trick, but as you say it does not work in WebPlayer (or iOS or Android IIRC) which makes it sort of limited.
Really, if you’re building a game a need high peformance by far the best way is to manually serialize your objects by hand.
There is an even more advanced way to do serialization, and that is by reflecting over the types you need to serialize, and then generate in memory IL wrapped as a delegate method and then caching it (performance hit only comes the first time you see a type). This will give you the best of every world: Serialization of any type. You don’t need to write any code for each type or implement an interface. It’s very fast (it will run at the same speed as hand written code specific for each type). It is however pretty (really) complex to implement.
Good point.
I do know from personal experience that YAXLib and BitStream are Mono compatible – at least to the extent I have used particular features in my own Unity projects. YMMV.
http://unity3d.com/support/documentation/ScriptReference/MonoCompatibility.html implies that it would work on iOS and Android as well.
Just to elaborate for the benefit of other users who come across this thread, the most important element of serialization is choosing the parts of the game state which need to be serialized. State which does not change needn’t be serialized because it’s redundant, the game already knows that state. Highly detailed state can often be simplified, for example serializing the state of a particle emitter including the location of all particles would bloat the data. It would be simpler to just serialize it’s position and parameters, this is true for many types of objects. The key is finding the minimum amount of data which can reasonably recreate the same state.
Thank you for all your responses. I’ve been looking into this serialization stuff and it seems like overkill. Let me know what you think of the following structure of data:
Each enemy or object in the game will send the following information in this same order:
(byte) id
(byte) action
(16bit float) x
(16bit float) y
This would be repeated about up to 8 times. There’s really only 8 enemies or items including the other player max on screen, and I feel that this amount of data would probably be enough - I would pack all 8 of these into one byte array and send it.
I can always send more detailed string based commands for setup, but for general networking flow I would just need the above repeated 8 times. Is this too much data to send at 15 frames a second?
And would I need to bother serializing it?
The gist is I would just need to send the positions of the enemies and their current action each frame to the client who is merely sending back his position and current action. The game is a final fight style beat em up and it’s for co-op play. One player will act as server, make decisions, send all enemies and player positions (as above, precision could be 16 bit just fine) and in return just receive player 2 in the same manner.
Is this a good or bad network design? I haven’t done multiplayer to this extent before, only turn based stuff with strings.
Well, no matter if you actually call it “serialization” or not, packing some data structure into another structure is serialization. Also, unless you do manual compression each float is 32 bits, not 16.
A couple of years ago, Bungie collected data from XBox owners, and established that their target would be 8kb/s. Your network traffic comes out to 10 b/f * 15 f/s * 8 = 1200 bytes/sec. I say you’re in the clear.
Thanks guys.
I’m thick as a post. I don’t understand a DAMN thing about serialization. All I want to do, is keep it simple. I want to put my ints, floats and bytes into a byte array and send it.
Then when I get a byte array, I want to decode it the same way. I know the order of the data and how much there is. My brain isn’t grasping all this new-fangled .net stuff.
Is there a very simple example which says: hey this is how you put a bunch of ints and floats into a byte array. And Hey, this is how you decode it?
Would be much appreciated. Otherwise I’ll just have to buy some more books and delay this a bit. Thanks for any replies.
Investigating BitConverter Class (System) | Microsoft Learn at the moment…
Thinking I will probably need to first allocate a byte array, then put BitConverter results into the array and increment myself as I go along.
And do the reverse for decoding it. Hope I am on the right track.
Well, maybe I’ll put my post back. 
System.BitConverter.GetBytes(x) will convert x, whatever it might be (float, int, etc.), to an array of bytes. That would be an array of 4 bytes for int and float. Then, going the other way, System.BitConverter.ToSingle (byteArray, i) will convert part an array of bytes, starting at index i, to a float, and BitConverter.ToInt32 does the same for ints. So you can take those arrays of bytes and mush them into one big array to send, then parse the byte array on the other end.
One possible sticking point is that BitConverter does nothing for endian issues. If you’re 100% sure that the two players will always both be using a system with the same endianness, then just do the above. Otherwise you have to take care of endian order yourself. Here is a link to PlayerPrefsX, which does this; what you’d need is the functions at the end.
–Eric
If you always know the order of the data, and the data sizes, e.g. four 32-bit floats, followed by two 32-bit integers, followed by three Booleans, and you don’t care too much for error detection, i.e. your network protocol will detect and correct the errors, or the data is saved and restored to disk and the possibility of corruption is reasonably low, then a simple bit stream serialization system works just fine. You can take the raw bits or bytes of each piece of data, stick it in to a byte array, and then do whatever you need to with it. The BitStream project I linked to will encapsulate all of that for you very easily, or you can do it manually.
If you are not sure of the order of the data, or the data sizes, or if the data may or may not include certain pieces, i.e. it is context sensitive, sometimes there is a Boolean to indicate some effect in your game and sometimes there isn’t, you can use the more traditional serialization techniques of .NET.
My recommendation is that you do not worry about network optimising or format optimising or coding optimising your serialization system at this time until you have it working just so.
Serialization is just the fancy modern word for sticking data in a container at one end and pulling it out of the container at the other end, but with the added implication that the serialization is more opaque to the programmer, and more forgiving of changing data formats and also different machine architectures. In the bad old days, if I wanted to store a few integers, I would have to worry about whether they were 16-bit, 32-bit or 64-bit integers, whether the machine I was on was using little-endian or big-endian architecture and floats and doubles might be even worse, because IEEE 754 might be a very fine standard for storing floating point numbers in a computer, but slight variations in implementations between different central processing units can cause all sorts of issues.
[System.Serializable]
public struct DataPacketSerialization
{
int health;
int level;
// we are assuming that Vector3 is a simple data structure
// containing intrinsic value types and is therefore can be
// automatically serialized by .NET without any extra work
// on our part
Vector3 pos;
}
public struct DataPacketBadOldDays
{
int health;
int level;
Vector3 pos;
byte[] PackItUp()
{
byte[] data = new byte[20];
data[0] = (byte)((health >> 24) 0xFF);
data[1] = (byte)((health >> 16) 0xFF);
data[2] = (byte)((health >> 8) 0xFF);
data[3] = (byte)(health 0xFF);
data[4] = (byte)((level >> 24) 0xFF);
data[5] = (byte)((level >> 16) 0xFF);
data[6] = (byte)((level >> 8) 0xFF);
data[7] = (byte)(level 0xFF);
// obviously this is C# and this kind of byte manipulation won't work
// we would instead use BitConverter.GetBytes to extract the individual bytes of floats,
// but this code illustrates the nastiness that was "serialization" in C++ in the bad old days
data[8] = (byte)((pos.x >> 24) 0xFF);
data[9] = (byte)((pos.x >> 16) 0xFF);
data[10] = (byte)((pos.x >> 8) 0xFF);
data[11] = (byte)(pos.x 0xFF);
data[12] = (byte)((pos.y >> 24) 0xFF);
data[13] = (byte)((pos.y >> 16) 0xFF);
data[14] = (byte)((pos.y >> 8) 0xFF);
data[15] = (byte)(pos.y 0xFF);
data[16] = (byte)(pos.z >> 24) 0xFF;
data[17] = (byte)(pos.z >> 16) 0xFF;
data[18] = (byte)(pos.z >> 8) 0xFF;
data[19] = (byte)(pos.z 0xFF);
return null;
}
void UnpackIt(byte[] data)
{
health = (data[0] << 24) | (data[1] << 16) | (data[2] << 8) | (data[3]);
level = (data[4] << 24) | (data[5] << 16) | (data[6] << 8) | (data[7]);
pos = new Vector3();
pos.x = (float)((data[4] << 24) | (data[5] << 16) | (data[6] << 8) | (data[7]));
pos.y = (float)((data[4] << 24) | (data[5] << 16) | (data[6] << 8) | (data[7]));
pos.z = (float)((data[4] << 24) | (data[5] << 16) | (data[6] << 8) | (data[7]));
}
}
All of this nice-ness and syntactic sugar that .NET adds comes with a price. The overhead of the serialization itself. Whereas the bad old days version takes up 20 bytes of data total, the serialization version will take up some arbitrary number of bytes more memory space simply because it needs to store type information too.
In .NET you can also version your serialization data, so that should your underlying data structure change, such as a new data member is introduced at a later time, but this new data needs to work with older versions of your code, you can say “I am using serialization version 1.7, this is what I expect” and the .NET serialization code will say “Okay, this is what I have, and here are some sane values for the bits I am missing.”
With serialization you can also easily change the format of the serialization, be it XML, which is reasonably readable by humans, especially if you make use of YAXLib to properly format your data, you can save it out as binary, which still includes the type information, error correction, etc all the way down to a bitstream, where the type information is thrown away and you, as the programmer, have to explicitly know what order the data members in, and what type they are, so that you can de-serialize the bitstream. With a pure bitstream you also need to be careful, as previously stated, about endian issues, Intel, Morotola, ARM, etc. Different implementations of floating point numbers (even if they follow the standards), or Booleans, can also cause issues.
You can also marshal your entire data structure by pinning it and then getting the raw underlying bytes in one easy .NET operation, this will give you the raw bytes, which you are then free to throw across a network to an identical machine architecture, and reconstitute at the other end with a similar marshal operation. I use this technique for reading memory out of non-managed applications such as Unity or World of Warcraft, when I want to go poking around in the innards of an application that has no functionality to do what I want. Andorov touched on this, but it is recommended you stay away unless you really have a strong reason to do this as you are relying on how the platform, Mono in this case, arranges its data in memory, though you can use ExplicitLayout on your structs if you need to.
Edit: Eric5h5 has mentioned the BitConverter class, which I also touched on, that can do much of what is needed to extract or inject bytes in to an intrinsic value type so you do not have to go mucking around with the bit-operations I showed in the example code that is more indicative of a C or C++ solution rather than C#.