Impossible Null Reference?

I’m using ISerializationCallbackReceiver to display a HashSet in the editor. For some weird reason, I’m seeing this NullReferenceException error:

ArgumentNullException: Value cannot be null.
Parameter name: array
System.Array.Clear (System.Array array, System.Int32 index, System.Int32 length) (at <437ba245d8404784b9fbab9b439ac908>:0)
System.Collections.Generic.HashSet`1[T].Clear () (at <351e49e2a5bf4fd6beabb458ce2255f3>:0)

Here’s my code:

[System.Serializable]
public class SHashSet<TKey> : HashSet<TKey>, ISerializationCallbackReceiver
{
    [SerializeField] private List<TKey> keys = new List<TKey>();

    public void OnBeforeSerialize()
    {
        keys.Clear();

        foreach (TKey key in this)
        {
            keys.Add(key);
        }
    }

    public void OnAfterDeserialize()
    {
        this.Clear();

        foreach (var key in keys)
        {
            this.Add(key);
        }
    }
}

This shouldn’t happen at all. It happens whenever I save my changes in a script and switch back to Unity (when it reloads or whatever)

That’s probably not going to work.

Your sHashSet inherits from HashSet. HashSet stores it’s data, internally, in an array.

When your SHashSet gets deserialized by Unity, that inner array is not going to automatically be initialized correctly. Some of the data will be set up correctly, some won’t. Reading through the decompiled version of Clear(), it looks like this:

    [__DynamicallyInvokable]
    public void Clear()
    {
      if (this.m_lastIndex > 0)
      {
        Array.Clear((Array) this.m_slots, 0, this.m_lastIndex);
        Array.Clear((Array) this.m_buckets, 0, this.m_buckets.Length);
        this.m_lastIndex = 0;
        this.m_count = 0;
        this.m_freeList = -1;
      }
      ++this.m_version;
    }

Both m_slots or m_buckets are not initialized before they’re needed, and Unity won’t be able to automatically serialize them. It seems like m_lastIndex is automatically serialized, though, causing the if-statement to pass.

Two suggestions for fixing this;

  • HashSet implements IDeserializationCallback, which has an OnDeserialization method. You could try invoking that at the start of OnAfterDeserialize.
  • The way we usually implement serializable versions of collections is not to inherit from them, but instead wrap the collection, and expose the methods that are needed. Then we new() the wrapped collection and add elements to it as needed in OnAfterDeserialize.
2 Likes

@Baste Thanks for the response. Everything is a bit clearer now and I’ll try the first suggestion tomorrow.
Could you please give me an example of the second suggestion? I’m having trouble understanding what you meant exactly. Thanks again!

[System.Serializable]
public class SHashSet<TKey> : ISerializationCallbackReceiver
{
    [SerializeField] private List<TKey> keys = new List<TKey>();
    
    private HashSet<TKey> hashSet;

    public void OnBeforeSerialize()
    {
        keys.Clear();

        if (hashSet != null)
            foreach (TKey key in hashSet)
            {
                keys.Add(key);
            }
    }

    public void OnAfterDeserialize()
    {
        if (hashSet == null)
            hashSet = new HashSet();
        else
           hashSet.Clear();

        foreach (var key in keys)
        {
            hashSet.Add(key);
        }
    }

    public void Add<TKey>(TKey value) => hashSet.Add(value);
    // etc. etc. for all methods
}

We do this with out SerializableDictionary implementation.

The biiiig downside here is that you can’t send this to a method that expects a HashSet. If you have SHashSet implement ISet and the other things that HashSet implements, you get a bit more flexibility.

So I’d for sure go for suggestion 1, since that doesn’t have those downsides.

1 Like

@Dextozz
One other workaround that seems to work correctly: add a default constructor that calls a base constructor from HashSet that calls the Initialize method. Here’s my implementation:

[Serializable]
public class SerializedHashSet<T> : HashSet<T>, ISerializationCallbackReceiver
{
    #region Properties

    [field: SerializeField]
    private List<T> SerializedValues { get; set; } = new();

    #endregion

    #region Constructors

    /*
     * Without a call to specific base constructors,
     * some underlying arrays won't be initialized correctly.
     * The constructor that takes a capacity calls Initialize, which fixes this issue.
     */
    public SerializedHashSet() : base(4) { }

    #endregion

    #region Methods

    /// <inheritdoc />
    void ISerializationCallbackReceiver.OnBeforeSerialize()
    {
        SerializedValues.Clear();
        SerializedValues.AddRange(this);
    }

    /// <inheritdoc />
    void ISerializationCallbackReceiver.OnAfterDeserialize()
    {
        Clear();
        this.AddRange(SerializedValues);
        SerializedValues.Clear();
    }

    #endregion
}
3 Likes

This doesn’t work for me, getting this warning:

Objects are trying to be loaded during a domain backup. This is not allowed as it will lead to undefined behaviour!

UnityEditor.Graphing.GraphObject:OnBeforeSerialize ()

And if I try to add 8 or more items to the hashset in the inspector I get this:

IndexOutOfRangeException: index + length > size
System.Array.Clear (System.Array array, System.Int32 index, System.Int32 length) (at <590dfe5623e34b4a95eb2713418f5b37>:0)
System.Collections.Generic.HashSet`1[T].Clear () (at <bb6ef06d183b4e69988b5217a4bef8fb>:0)
SerializedHashSet`2[T,ST].OnAfterDeserialize () (at Assets/Scripts/SerializedHashSet.cs:33)
IndexOutOfRangeException: Index was outside the bounds of the array.
System.Collections.Generic.HashSet`1+Enumerator[T].MoveNext () (at <bb6ef06d183b4e69988b5217a4bef8fb>:0)
SerializedHashSet`2[T,ST].OnBeforeSerialize () (at Assets/Scripts/SerializedHashSet.cs:24)
UnityEditor.InspectorWindow:RedrawFromNative() (at /home/bokken/build/output/unity/unity/Editor/Mono/Inspector/Core/InspectorWindow.cs:241)

I’m guessing that hardcoding the base(4) here isn’t a good idea.

I doubt that this 4 makes any difference. My guess is that because during domain reload Unity actually serializes private fields as well, all the internal state variables of the HashSet are serialized and restored as well, except for the actual “_entries” array. So when you deserialize this you always end up with a wrongly initialized class. So I would suggest you do what @Baste had suggested and create a wrapper class. In such a wrapper it’s usually also a good idea to do

[System.Serializable]
public class SHashSet<TKey> : ISerializationCallbackReceiver
{
    [SerializeField] private List<TKey> keys = new List<TKey>();
    
    [System.NonSerialized] private HashSet<TKey> hashSet;

This ensures that Unity won’t serialize anything of the actual hashSet.