Are native containers serializable?

Unity Engine
, ,
1

I am procedurally generating huge map for my game. I would like to store it in streaming assets folder split into chunks of Native Arrays. Is that possible?

2

Not directly, but if you have a .NET binary writer, and don’t mind doing a little unsafe code, it’s doable. I can toss together an example if you’d like.

2 Likes
3

It’d be great if you can show an example

1 Like
4

@burningmime I also would be interested. That would extend the examples serializations you already give me into a more “native to dots” area, so I’d be curious where the differences are.

5

I don’t have my real computer, just work computer (long story…) so I can’t test this until next week some time:

private static unsafe Span<byte> NativeArrayToByteSpan<T>(NativeArray<T> array, int lengthBytes, bool readOnly) where T : struct {
    Unity.Assertions.Assert.IsTrue(lengthBytes == (int) array.Length * (int) UnsafeUtility.SizeOf<T>());
    void* ptr = readOnly ? NativeArrayUnsafeUtility.GetUnsafeReadOnlyPtr(array) : NativeArrayUnsafeUtility.GetUnsafePtr(array);
    return new Span<byte>(ptr, lengthBytes); }

public static void WriteNativeArray<T>(System.IO.BinaryWriter writer, NativeArray<T> array) where T : struct {
    int lengthBytes = (int) array.Length * (int) UnsafeUtility.SizeOf<T>();
    // we don't need to write both, but it's just for sanity checks/debugging (and in case the size of the type changes)
    writer.Write((int) array.Length);
    writer.Write(lengthBytes);
    writer.Write(NativeArrayToByteSpan(array, lengthBytes, true)); }

public static NativeArray<T> ReadNativeArray<T>(System.IO.BinaryReader reader) where T : struct {
    int length = reader.ReadInt32();
    int lengthBytes = length * (int) UnsafeUtility.SizeOf<T>();
    int checkBytes = reader.ReadInt32();
    if(lengthBytes != checkBytes)
        throw new Exception($"Corrupted/invalid data -- expected to read constant {lengthBytes}, but actually read {checkBytes}");
    NativeArray<T> array = new NativeArray<T>(length, Allocator.Persistent, NativeArrayOptions.UninitializedMemory);
    bool fail = true; try {
        Span<byte> span = NativeArrayToByteSpan(array, lengthBytes, false);
        int readBytes = reader.Read(span);
        if(readBytes != lengthBytes)
            throw new Exception($"Stream too short -- expected to read {lengthBytes} bytes, but actually read {readBytes} bytes");
        fail = false; return array; }
    finally { if(fail) array.Dispose(); } }

EDIT: @snacktime pointed out that there’s also an Entities API for this, if you’re already using that package: https://docs.unity3d.com/Packages/com.unity.entities@0.11/api/Unity.Entities.Serialization.BinaryReaderExtensions.html . You’ll still need to serialize the length separately, but it would work something like this:

public static void WriteNativeArray<T>(Unity.Entities.Serialization.BinaryWriter writer, NativeArray<T> array) where T : struct {
    int lengthBytes = (int) array.Length * (int) UnsafeUtility.SizeOf<T>();
    // we don't need to write both, but it's just for sanity checks/debugging (and in case the size of the type changes)
    writer.Write((int) array.Length);
    writer.Write(lengthBytes);
    writer.WriteArray(array); }

public static NativeArray<T> ReadNativeArray<T>(Unity.Entities.Serialization.BinaryReader reader) where T : struct {
    int length = reader.ReadInt();
    int lengthBytes = length * (int) UnsafeUtility.SizeOf<T>();
    int checkBytes = reader.ReadInt();
    if(lengthBytes != checkBytes)
        throw new Exception($"Corrupted/invalid data -- expected to read constant {lengthBytes}, but actually read {checkBytes}");
    NativeArray<T> array = new NativeArray<T>(length, Allocator.Persistent, NativeArrayOptions.UninitializedMemory);
    bool fail = true; try {
        reader.ReadArray(array, length);
        fail = false; return array; }
    finally { if(fail) array.Dispose(); } }
1 Like
6

Entities package has had this for some time see BinarySerialization.cs.

3 Likes