Serialization [0.6.3-preview] Package Performance Issues

[Soaryn]

Is there a way currently to read in a json string and deserialize to a data struct containing a NativeString?
I was hoping the new JsonSerialization utility was the solution, but it seems that NativeString is only currently deserialized from its specific structure: (length, Byte30, etc) rather than from also a string using the implicit conversion.
EDIT: It now deserializes as shown below; however, there is a rather large performance impact utilizing this. Thread title changed to better represent the current problem.

Example:
Struct:

public struct DataStruct : IComponentData {
    public NativeString32 Value;
}

Json:

{
    "Value" : "Test"
}

[Soaryn]

This actually worked, but would love confirmation if this is how we should be handling json translation from string to NativeString.

TypeConversion.Register<SerializedStringView, NativeString32>(view => new NativeString32(view.ToString()));

[Soaryn]

After some experimenting, the Serialization package for json is not ready quite yet. The results are showing 100 measurements of 100 iterations. This is purely reading the same json line. These values are HOPEFULLY either a debug test is running in the background of the serializer, or I am using it wrong.

The code used to test these two is below:

    public class JsonTests {
        private const string Basic = @"{""Value"":""JarJar""}";

        public struct BasicData {
            public string Value;
        }

        [Test, Performance]
        public void TestJsonUtility() {
            Measure.Method(() => {
                    var data = JsonUtility.FromJson<BasicData>(Basic);
                })
                .Definition("JsonParsing_JsonUtility")
                .WarmupCount(1)
                .MeasurementCount(100)
                .IterationsPerMeasurement(100)
                .Run();
            PerformanceTest.Active.CalculateStatisticalValues();
        }


        [Test, Performance]
        public void TestJsonSerializer() {
 
            Measure.Method(() => {
                    var data = JsonSerialization.DeserializeFromString<BasicData>(Basic);
                })
                .Definition("JsonParsing_Serializer")
                .WarmupCount(1)
                .MeasurementCount(100)
                .IterationsPerMeasurement(100)
                .Run();
            PerformanceTest.Active.CalculateStatisticalValues();
        }
    }

[Soaryn]

So far, it has only been the deserialization where performance has gone totally awry. Which incidentally is what I need more heavily ;). This actually seems backwards from what I expected, but still not to this degree.

25 iterations per measurement. 100 measurements:

[Prodigga]

Unrelated, sorry, but I was wondering what you are using for your benchmarking?

[Soaryn]

The performance testing package (using Unity.PerformanceTesting; )which is included as a dependency for ECS 0.2.0

[MNNoxMortem]

@Soaryn , have you experimented with other serializers, e.g. Odin, GitHub - MessagePack-CSharp/MessagePack-CSharp: Extremely Fast MessagePack Serializer for C#(.NET, .NET Core, Unity, Xamarin). / msgpack.org[C#], etc?

However, are you sure you really need the string values inside of your burst compiled jobs, or would it be sufficient to use whatever the fastest kind of serializer available and then run the job with blittable data?

[Soaryn]

So I have NativeStrings being my end target data container inside my structures which is why I was looking into the new serializer. JsonUtility doesn’t handle customs very well. NativeStrings are fine to use in Bursted jobs so long as I don’t go back into managed string land

I’d prefer not to have a 3rd party library added for this if I can at all avoid it as that adds another dependency to this which is already 1 too many. The Json I am receiving is not of my construction unfortunately. The data to placed as an entity component is not necessarily the full json though. I am currently just wanting to serialize a provided json to a custom data struct.

[Soaryn]

I still have yet to get the Serialization package to work as performantly as it would suggest. As a thought, I decided to test to see if batching the json into a single large string would improve the performance, and even then it takes a minimum of ~1.97ms (1/8th of a frame) to deserialize a single struct that contains a single field. This does improve the more you add; however, this is not ideal if receiving say 1 message per second and you need to be reactive to it rather than buffering messages.

Test cases are jsonString, with batch size. 1 iteration per measurement, 100 measurements.

[TestCase(@"{""Value"":""Barry""}", 1)]
[TestCase(@"{""Value"":""Barry""}", 10)]
[TestCase(@"{""Value"":""Barry""}", 100)]
[Test, Performance]
public void TestJsonBatched_Serialize(string json, int count = 10) {
    var jsonBatch = $"{{\"batch\":[{string.Join(",", Enumerable.Repeat(json, count))}]}}";
    Debug.Log(jsonBatch);
    Measure.Method(() => {
            var batch = JsonSerialization.DeserializeFromString<BasicDataBatch>(jsonBatch);
        })
        .Definition("JsonParsing_Serializer_DeserializeBatch")
        .WarmupCount(1)
        .MeasurementCount(100)
        .IterationsPerMeasurement(1)
        .Run();
    PerformanceTest.Active.CalculateStatisticalValues();
}

Regardless of the use case, this deserialization method is far from performance by default

Any Unity staff able to look into why performance of this package is awry on deserialization? Serializing I would think would be the slower aspect, but somehow it is night and day difference to the deserialization :\

[Soaryn]

Same test as before with the JsonUtility used instead for comparison.

[Prodigga]

Just wanted to say good job and keep at it! Hopefully you get some eyes on this soon from someone at Unity.

[Dale_Kim]

Could you share your test project with us?

[Soaryn]

I can; however, it is purely using the scripts above. I don’t mind proving a package later tonight however

[martinpa_unity]

Sorry for the delay before answering. We’ve just been made aware of this post (context: this package is used by entities, but wasn’t developed for entities specifically, and since we are a “non-visible” preview package, we don’t really have a dedicated forum).

We have re-created this on our side and we are working on it. Thanks for raising this up!

[Soaryn]

@martinpa_unity

Sorry for the delay before answering. We’ve just been made aware of this post (context: this package is used by entities, but wasn’t developed for entities specifically, and since we are a “non-visible” preview package, we don’t really have a dedicated forum).

We have re-created this on our side and we are working on it. Thanks for raising this up!

Hey @martinpa_unity no worries! I was curious if there has been any progress in tracking down what might be causing the long interval of time when deserializing?

[martinpa_unity]

Yes, the massive difference in timing is due to a number of different things, and we’re working on them at the moment. We’ll do a post about the timings we were able to achieve this week. Here are some of the things to keep in mind:

• JsonUtility operates on the native side and has direct access to Mono types, so it will be very hard to beat, but we’re confident we can get it down to something in the 2x-3x range for the same use-cases.
• Because we support streaming data in, we were using safe views over the data pretty much everywhere. In the cases where we deserialize from a complete Json string, we can use unsafe views over the data. This shaved off quite a lot of time.
• Reading/loading in memory and tokenizing is very fast and is on par with what JsonUtility can do. This is because we are using Burst to do this part.
• Having the Jobs debugger, the safety checks and leak detection on will chip at our results in the editor. Same goes for the profiler, of course.
• Creating new instances and transferring the data in those take the bulk of the time, because we cannot use Burst here, since we are dealing with arbitrary types. This is where JsonUtility will outperform us. We’ll optimize this part as much as possible, but as this is a C# only library, we’ll be bound by its limitations as well. Deserializing into an existing instance will most likely help a lot.
• This last step is done using another library, which is being re-written and should be more performant as well.

On top of all of the above, there are some design decisions that we took that we believe will add more flexibility and more features, at the cost of some overhead.

Polymorphic serialization
We decided to include this from the start. The overhead here is evident, as we need to ensure that we create new instances of the correct type. JsonUtility only supports this for UnityEngine.Object types.

Enhanced collection support
We decided to support more collection types and with the next version, we’ll be able to support lists, arrays, dictionaries and hash sets. This also includes nesting collection types.

Support for generic types
Since we are not tied to current Unity serialization, we can more easily dig in generic types.

Separation between serialization and UI
JsonUtility is tied to the way Unity is serializing the data, which means it is also tied to what would be shown in the default inspector. With this library, we’ll be able to separate the two cleanly, even though the default behavior will resemble what Unity does today. For example, you could mark a field to be serialized and use a property for UI, so that you can validate the data in a more natural way.

Migration support
Migration is currently possible in Unity at the engine level and through a limited number of attributes (i.e.: FormerlySerializedAs), but general migration is somewhat lacking for users.

We aim to allow users to control how their data change across time. We’ll support name changes, moving types to a different assembly/namespace, type changes and type conversions and manual migration. Not all of these will be supported at first, but this is the direction we are going.

Validation
When parsing the Json file, we can run different levels of validation to ensure that it is, in fact, a valid Json string or file. In the case of an error, we can output where the errors were found (line and character) as expected.

During deserialization, we can also inform the users about different problems that may have occurred, such as an instance could not be created because no default or implicit constructor was found, as well as give an indication as to how users can fix this problem.

Streaming
The low level parser is designed as a hybrid between SAX (forward-only read) and DOM (document-object-model). This gives us the ability to stream in large amounts of data while still working with discrete chunks.

Low-Level support
While we provide some high-level APIs to make it very easy to use, users are able to use the SerializedObjectReader to do custom deserialization by walking the tokens manually.

The in memory views returned from the reader can be consumed in bursted jobs for more specialized code paths.

---

Most of the above will add overhead compared to JsonUtility, but we believe these features are worth it.

As mentioned above, we are rewriting one of the packages that this one depends on, so the next update will probably take some time.

Let us know what you think.

[Prodigga]

Very excited for low level support. The only reason I am using Newtonsoft JSON over JSONUtility is because it lets you write your own json converters. Our usecase: We have a database asset in our project that references all the Database elements in the project (Both are ScriptableObjects). Every element has an ID that never changes so you can globally lookup a database element from anywhere given an int ID. We wrote a Json converter for type DatabaseElement that just writes out that elements ID when serialising (0 for null) and when deserialising it does a database lookup for that element. There are other ways to achieve the same thing though this is just so pleasant to work with. You just serialise a class that has references to a bunch of SerialisableObjects and when you deserialise it the references to those SerialisableObjects are automatically populated.