Are there any hooks available for invoking code before Unity’s serialization system deserializes any assets after compilation/domain reloading in the editor? I’ve tried the following without luck:
AssemblyReloadEvents.afterAssemblyReload
Static class with [InitializeOnLoad] and a static constructor
The issue I’m trying to solve is that I’m using MemoryPack for byte serialization. I need to be able to register these formatters from a static context BEFORE any asset gets ISerializationCallbackReceiver.OnAfterDeserialize() called on them. The [InitializeOnLoad] works in all cases except for when that class itself is modified. For whatever reason, its static constructor in that scenario will get called after everything is deserialized and my deserialization code will fail because the formatters haven’t been registered yet.
For now, I just detect this editor-only scenario and defer all the side effects from OnAfterDeserialize(), OnEnable(), and OnValidate() and queue them up in EditorApplication.delayCall. However it’s incredibly hacky and delicate.
Surely Unity offers some way to invoke code prior to all deserialization? Any thoughts?
If you want to use such a package, you should not apply this to Unity-serialized data but rather employ your own kind of serialization to your own file format so you have full control over when and how to read/write that data.
I parsed the docs for MemoryPack. It looks like it WANTS to be applied to Unity’s serialized data. I must say the caveats and known issues and the “black box” design with attributes clearly sound like something to steer away from. In particular if you have no pressing issues (as in: make or break the project kind of issues) to improve the serialization of your data.
If you have that kind of make or break issues, it’s still best to write your own serializer!
MemoryPack is the kind of package you either do not even look at because it locks you into the way it wants you to do things, which has apparent limits. Or you need to dig deep to understand how it’s designed and works so you can fix the problem you are having (thus wasting a lot of times with potentially no resolution). Or you write your own serializer as I said that is perfectly designed to cover your use case and thus blows every generic serializer out of the sky, no matter what they promise.
To that end, look into the Unity Serialization package’s binary serializer. I’ve had a blast with this one. Easy to use, once you get the idea, and extremely efficient, even burstable.
You asked for “any thoughts” and that’s what you get.
Granted, we don’t have the time to write serializers that we only need for convenience. To me, that’s Odin Serializer and Dictionary types. But when you need to serialize interfaces or polymorphic objects at runtime I’d say that’s a sign of a broken architecture (data not separated from code, using the wrong types like multidimensional arrays or trying to serialize black boxes like Unity Mesh).
We do and ought to have the time to write serializers that are the core of our game, without which the game would suffer for all eternity due to either using too much disk space or CPU cycles. The kind of thing where we would need the speed and efficiency of a thing like MemoryPack. We cannot afford to outsource that to a black box library because it’ll come back and bite us, hard.
I guess this is the latter case here.
When I think of serialization, I always think of it in terms of data, never anything code related. IMO there’s nothing in a game that ought to need serialization on the level of interfaces, delegates, events, polymorphic classes, or 3rd party types that we don’t have any control over. I kinda feel that’s where serialization in general took a wrong turn, trying to cover those use cases that we needn’t have to begin with.
You have some absolutely ridiculous and absurd notions, sorry. There’s plenty of reason to be serialising interfaces and polymorphism in particular (alongside by-reference serialisation in general). Why the hell do you think Unity introduced [SerializeReference]? Because it’s useful, and it’s powerful! Not using it is just shooting yourself in the foot over what only amounts to pride.
And no we don’t have the time to write our own serialisers. Most of us just want to be working on the meaningful part of our projects, not faff about with problems that have already been solved.
But we’re also talking different things here, I try to point out the differences.
There’s the part where you work from within Unity and have serialization needs where SerializeReference can be very useful (polymorphism). To me, that is editor serialization, that’s the thing where Unity provides us with a stable experience. And where serialization is primarily to support our editing needs, and it’ll magically, transparently also work at runtime (ie the Inspector assigned reference is not null).
What I would warn against is to rely on the Unity serialization system to save data that would be better accommodated by a custom serializer. The kind of data that typically ends up in a savegame or persistent (modifiable) world. The kind where you end up having to write a custom serializer anyway because Unity’s serializer cannot write at runtime.
I often see the former being used, then a custom runtime data serialization being tacked onto it.
But due to the way both systems are incompatible, you end up with a requirement that comes from Unity’s serializer because it does allow for something like polymorphism that you suddenly find you cannot read/write at runtime. That’s at least neglect or inexperience, to not consider the need for runtime serialization of parts of the data. To not write code in a way that it can instantiate the necessary references from plain value types.
Say, you have tiles in your game, and they are GameObjects. In Unity, no problem. At runtime, you need to be able to say that you got a “123” for position 11,-5 and then instantiate a GameObject at that position and give it the graphics for “123”. It’s a simple problem and a simple solution, yet what is often attempted is to actually save the GameObject including its visualization. That’s super wasteful all around, disk space, save/load time, development time (because it poses many avoidable issues).
The bad idea is to try and make the game’s serializable data fully serializable with the Unity serialization system, including custom C# classes with [Serializable] attribute and so on. If the game’s data layout is very complex or provides opportunities (or the necessity) for optimization, say Minecraft style 3D chunks in a large world - it’s just not going to run performant with Unity’s serializer. Regardless of whether you add MemoryPack on top. For that kind of thing you need 100% full control over the serialization of your data.
I hope that makes it clearer. I’m mostly concerned with preemptively considering how data gets written and loaded at runtime because that dictates how the code is architected. If that’s done right, it’s really not a big time sink to write a serializer for your tile world chunks. Or the savegame.
No one does that. A serialiser is standalone. You take your data in memory and serialise that. There’s none of this ‘unity serialising data with another serialiser tacked on’ on that you’re making up.
Both MemoryPack and the Odin Serializer are standalone serialisers. Neither of them require or are specific to Unity. All they do is serialise data with their particular method and API. Odin is just a particular case in that it just has a lot of existing Unity support, and follows the same patterns as Unity as to what it will and won’t serialise (though this is customisable as well).
You cannot serialise a Unity object to disk, full stop. I know that, and plenty of other people know that. No one is doing it because it’s not possible. You still always have to serialise raw data to disk and back again when it comes to save games.
Just feels like you’re fabricating issues here to prove a point.
Not fabricated but arguably mixing a few past experiences together.
My worst experience is trying to do custom serialization of whatever sort alongside with Unity’s serialization in the editor much like the OP seems to be doing. I came to the conclusion that this won’t ever work satisfactory no matter what you are trying to do. The editor simply serializes data far too often, eg for GUI needs, and it does so on a per-object basis which is not deterministic.
On the other hand, being able to load your custom data file and have GameObjects get spawned in the editor (but not saved to the scene) and being able to do so at runtime with the exact same code, deterministic, with versioning and full debuggability. That was not all that much engineering work but prevented a lot of headaches where otherwise the saving of objects into the scene had gotten in the way. And when you have that, you think of many things differently … like not having a UnityEvent on each GameObject but hooking up your events some other way.
I don’t really see what this has to do with your serialiser of choice, though. This is just a matter of managing your data; basically a matter of model-view. Keep the model separate to the view. So long as you have a pure data representation that’s free of direct Unity object references, you can use any serialiser at that point and you certainly don’t need to write your own.
Someone asking about editor hooks and coming in and telling them to write their own serializer is an absolutely ridiculous response, sorry.
Yeesh…Let’s clear the air. I’m sorry I didn’t choose your binary serializer @CodeSmile. To be fair, I had never even heard about it.
Anyway, @spiney199 is right. I don’t have time to write my own serializer and MemoryPack is not a package you just “steer away from”. It has 3.1k stars and is authored by arguably the leading serialization expert in C# (having written 4 successful serializers, each one better and more performant than the last). I was able to modify a 3rd party plugin from JSON serialization to byte serialization in just a few days. This is literally the only problem I’m having, and seems to be more a Unity problem anyway for not exposing deterministic editor lifecycle hooks to users. Don’t blame this on MemoryPack.
That being said, I’m still open to hearing solutions to my original question. In the meantime, this is my hacky solution:
void ISerializationCallbackReceiver.OnAfterDeserialize() {
#if UNITY_EDITOR
//Hack: Fixes a race condition where MemoryPackSettingsSO is modified in the editor, then after
//compilation/domain reload, the graph attempts deserialization before the formatters have had a chance
//to register with the MemoryPackFormatterProvider.
if ( !useJsonSerialization && !MemoryPackFormatterProvider.IsRegistered<Node>() ) {
//initialize the static class (we must use reflection because the class lives in a different assembly)
var memoryPackSettingsType = Type.GetType("MemoryPackSettingsSO, Assembly-CSharp, Version=0.0.0.0, Culture=neutral, PublicKeyToken=null");
var methodInfo = memoryPackSettingsType.GetMethod("Initialize", BindingFlags.Static | BindingFlags.Public);
methodInfo.Invoke(null, null);
}
#endif
SelfDeserialize();
}
FWIW I don’t feel like your “hack” warrants a better solution. It’s absolutely okay to resort to reflection to do something just before Unity does something else.
Although your race condition issue smells like it’s an issue of MemoryPack to begin with. It’s kind of odd that it would not be able to handle this itself, unless you are using it outside of its intended use cases. Perhaps it doesn’t support in-editor serialization? Or it’s as simple as the package not properly supporting assembly definitions? I’m just guessing.
the off-topic stuff
That’s why I feel like mentioning it every time something even remotely serializable comes up.
It’s a slim, really well-engineered package. I did not know about this package 2 years ago either and it existed since at least 2019 by that point. It’s not in the Unity registry for some reason, but it’s not pre or exp.
Sorry for derailing the thread though.
Exactly my point. You put your money on a horse you don’t ride. That’s called a gamble.
That is … if you jump on it based only on popular opinion and promises on how fast it is and how many times it has been rewritten.
I can’t really assess it at this point. I just know for core features like serialization I’m on edge to entrust this to other people’s code. If after an hour I can’t understand what their code does, I won’t use it because at some point I will likely have to debug it, and dig deep.
I don’t! I think of it more as a general issue of tacking general-purpose serialization tools onto Unity. These tend to open up even more issues, one way or another. Including Odin. I used to love it but never even looked at it after that one time many years ago where it killed editor performance, and also lost us a good amount of data in the process of unwinding it.
