What is a good way to generate/change data for optimization purposes?

[Ardenian]

Hello,
I am looking for an answer to changing or generating data in the editor when optimizing objects. Before I get into details or my own research results, I would like to give a quick example that hopefully explains my issue.

Example: I have a MonoBehavior component that looks like this:

public class MyComponent : MonoBehavior {
    [SerializeField]
    private List<MyScriptableObject> data;
}

This component has a list of data objects (ScriptableObjects), which contain configurations. My designer can now easily choose which objects to add to this list and everything works fine. However, inside my code where I use the configurated data, I need to have a different collection, in this particular case a Dictionary.

How do I display the list in the inspector, but transform it into a different object for runtime purposes, but in the editor, explicitely not transforming it at runtime?
Research: Having made my own research, I found two solutions, being ISerializationCallbackReceiver and OnValidate. I am not going too much into detail what these things do and more into the problems that I have with them.

Let’s start with the easier one, OnValidate. This is a method that is being called every time that something in the inspector is being changed. Sounds perfect, since I want to do something when my designer alters data in the inspector of a component.

public class MyComponent : MonoBehavior {
    [SerializeField]
    private List<MyScriptableObject> data;

    private Dictionary<MyKeyType, MyScriptableObject> organizedData;

    public void OnValidate(){
        organizedData.Clear();
        foreach(var item in data){
            // fill organizedData based on data.
        }
    }
}

The problem that I have here is that I don’t know if this works with serialization and if the data generated for organizedData will persist through sessions and at runtime. ISerializationCallbackReceiver does exactly that, it seems:

public class MyComponent : MonoBehavior, ISerializationCallbackReceiver {
    [SerializeField]
    private List<MyScriptableObject> data;

    private Dictionary<MyKeyType, MyScriptableObject> organizedData;

    public void OnBeforeSerialize(){}

    public void OnAfterDeserialize()
    {
       organizedData = new Dictionary<MyKeyType, MyScriptableObject>();

        for (int i = 0; i != data.Count; i++){
           // fill dictionary
        }
    }

}
}

However, I do not understand why I have to write code into both directions. I only want to fill the dictionary, why do I have to implement OnBeforeSerialize(), too? It is a one-way transformation, isn’t it, so why would I want to transform the dictionary back into the list?

Additionally, in both solutions, I end up with more member per class that I need. At runtime, there should be no list with the data, only the dictionary. Is there any way around this, having only the members at runtime that one actually needs? I thought about using #UNITY_EDITOR for members that are only relevant in the editor, but I don’t know if at runtime, OnAfterDeserialize() is being called or something like that.

Are there better ways to transform members into different members in the editor, so that one ends up with optimized objects and not more members than necessary in the object?

[eses]

Hi @Ardenian

This part sounds understandable:

“How do I display the list in the inspector, but transform it into a different object for runtime purposes…”

But the rest of the sentence doesn’t make sense to me.

If you just need to convert a list of scriptable objects into dictionary of <string,scriptableObject> for runtime use, can’t you just do this in Start, so that you just fill the Dictionary from a List?

I don’t see how using ISerializationCallbackReceiver would help in this case, but then again, I didn’t quite get what you want, as it sounds like you only need one way transform from List of SOs to Dictionary.

[Ardenian]

Thank you for your respone, @eses !

@eses

Hi @Ardenian
(…) can’t you just do this in Start (…)

This is exactly what I would like to avoid and what I tried to explain in my original post. Think about it this way. I already know how my dictionary will look like at runtime, based on the list that I have in the editor of the same object. Therefore, there is no need to wait for runtime to fill the dictionary.

If thousands of objects did that at the same time, think about performance, every time an object of this type is created, it would execute Start() and thus do a calculation that should have already been done in the editor beforehand. I hope that this explanation clears up with what I try to achieve.

How do I avoid calculating something, a member for instance, at runtime when I already know in the editor how it is supposed to look like and it could therefore be calculated in the editor instead, lowering the impact on performance at runtime?

[grizzly]

You can’t serialize a Dictionary in Unity, so there’s no way around building it from a List at runtime as you are already doing.

[Ardenian]

Thank you for your respone, @grizzly !

@grizzly

You can’t serialize a Dictionary in Unity, so there’s no way around building it from a List at runtime as you are already doing.

Does this means that even ISerializationCallbackReceiver only does it at runtime, when the object is deserialized and prepared for the world to be spawned into, every time an object of the type is spawned?

Hm, looks like it is the case. From the Scripting API by Unity:

@Scripting API

(…) when using Instantiate() to clone an object, Unity serializes and deserializes the original object in order to find internal references to the original object, (…)

So it looks as if there is no way to actually calculate something with native inspector and store it, so it doesn’t have to be calculated at runtime, every time when deserializing the object.

Makes me wonder whether tools like Odin do allow you to do that, since they can serialize types like dictionaries.

[grizzly]

@Ardenian

Does this means that even ISerializationCallbackReceiver only does it at runtime…

No, ISerializationCallbackReceiver works both in the Editor and at runtime.

@Ardenian

Makes me wonder whether tools like Odin do allow you to do that, since they can serialize types like dictionaries.

I believe Odin is using the same ISerializationCallbackReceiver interface to handle Dictionaries in the same way.

PS: You can read more about Serialization in Unity here.

[Ardenian]

Thanks for the link, @grizzly , after reading a bit through it:

(…) the best approach is to use serialization callbacks. They allow you to be notified before the serializer reads data from your fields and after it is done writing to them. You can use this to have a different representation of your hard-to-serialize data at runtime than when you actually serialize (…)

So when objects are instantiated, it does:

When you call Instantiate() (…) we serialize the object, then create a new object, and then we “deserialize” the data onto the new object.

Which means there is zero way to not calculate something at runtime, concerning serialization?

[grizzly]

@Ardenian

Which means there is zero way to not calculate something at runtime, concerning serialization?

You can calculate whatever you want in the Editor, but can’t avoid the copy if you want to use a Dictionary in this way.

[Antistone]

In some sense ALL data structures have to be built at run-time, no matter how they were serialized. The data on your hard disk is not literally a C# object instance, even when Unity does it. The data still goes through a conversion process; it’s just hidden.

If you are performing some sort of expensive transformation process on the data, then there’s probably some performance savings you could get by (somehow) saving precomputed results. But if the thing you are trying to save is simply copying entries from a List to a Dictionary, it’s not obvious to me that you could do anything significantly faster even if Unity offered a more convenient feature-set. Any data saved to a file is presumably going to be serialized to some kind of linear format. It’s not inconceivable that there’s a way of parsing that file that is faster than putting its contents into a List before transferring them to a Dictionary, but it’s also not obvious that one must exist.

You could, of course, write your data to your own separate file, and then at runtime you could read and parse that file in any way you like. But again, it’s not clear that would offer any savings.

[Ardenian]

@Antistone

it’s not obvious to me that you could do anything significantly faster

Yeah, I agree. I implemented ISerializationCallbackReceiver for one particular object and in total, it saved me around 10 ms (2% better performance, time-wise) in test runs, so it ain’t be nothing, but certainly not the most important place to improve performance.

I asked this question because I perform a lot of data requests from lists, reading out configuration data. Since I only use native editor UI so far, I can only use lists and arrays and have to perform calculations at runtime to transform my lists into data structures that are more suited for my needs, which does take a toll if done repeatedly. Thus my prompt to investigate if there are ways to perform such calculations once (and again on changes) without it impacting runtime for players.