Issue with UIToolkit when using sprites in text

Unity Engine
, , , ,
1

I am encountering an issue with UIToolkit when using sprites in text in Unity. So far, I haven’t been able to create a minimal reproducible example, as the problem only occurs within a larger scope.

The issue arises in the key rebinding menu when displaying buttons with icons in text using button.text = "<sprite name=...>". In this menu, UIToolkit breaks, starts throwing the following errors, and some images fail to render properly—only the <sprite name=...> tag is displayed instead.

Reproducible in 6000.0.43f1 and 6000.1.0b11.

ArgumentException: An item with the same key has already been added. Key: 1
System.Collections.Generic.Dictionary`2[TKey,TValue].TryInsert (TKey key, TValue value, System.Collections.Generic.InsertionBehavior behavior) (at <7c8bdf03079d46eabc06e15c1257780a>:0)
System.Collections.Generic.Dictionary`2[TKey,TValue].Add (TKey key, TValue value) (at <7c8bdf03079d46eabc06e15c1257780a>:0)
UnityEngine.TextCore.Text.SpriteAsset.UpdateLookupTables () (at <e493e4ea7412422f9b7eec7c629e14e8>:0)
UnityEngine.TextCore.Text.SpriteAsset.GetSpriteIndexFromHashcode (System.Int32 hashCode) (at <e493e4ea7412422f9b7eec7c629e14e8>:0)
UnityEngine.TextCore.Text.SpriteAsset.SearchForSpriteByHashCode (UnityEngine.TextCore.Text.SpriteAsset spriteAsset, System.Int32 hashCode, System.Boolean includeFallbacks, System.Int32& spriteIndex, UnityEngine.TextCore.Text.TextSettings textSettings) (at <e493e4ea7412422f9b7eec7c629e14e8>:0)
UnityEngine.TextCore.Text.TextGenerator.ValidateHtmlTag (UnityEngine.TextCore.Text.TextProcessingElement[] chars, System.Int32 startIndex, System.Int32& endIndex, UnityEngine.TextCore.Text.TextGenerationSettings generationSettings, UnityEngine.TextCore.Text.TextInfo textInfo, System.Boolean& isThreadSuccess) (at <e493e4ea7412422f9b7eec7c629e14e8>:0)
UnityEngine.TextCore.Text.TextGenerator.PopulateFontAsset (UnityEngine.TextCore.Text.TextGenerationSettings generationSettings, UnityEngine.TextCore.Text.TextProcessingElement[] textProcessingArray) (at <e493e4ea7412422f9b7eec7c629e14e8>:0)
UnityEngine.TextCore.Text.TextGenerator.PrepareFontAsset (UnityEngine.TextCore.Text.TextGenerationSettings generationSettings) (at <e493e4ea7412422f9b7eec7c629e14e8>:0)
UnityEngine.TextCore.Text.TextHandle.PrepareFontAsset () (at <e493e4ea7412422f9b7eec7c629e14e8>:0)
UnityEngine.UIElements.UITKTextJobSystem+PrepareTextJobData.Execute (System.Int32 index) (at <51d3c247e402427c8c078d62a0e45cb0>:0)
Unity.Jobs.IJobParallelForExtensions+ParallelForJobStruct`1[T].Execute (T& jobData, System.IntPtr additionalPtr, System.IntPtr bufferRangePatchData, Unity.Jobs.LowLevel.Unsafe.JobRanges& ranges, System.Int32 jobIndex) (at <c007129f0c8f49e5b1f7e125061a5718>:0)

ArgumentException: An item with the same key has already been added. Key: -955380455
System.Collections.Generic.Dictionary`2[TKey,TValue].TryInsert (TKey key, TValue value, System.Collections.Generic.InsertionBehavior behavior) (at <7c8bdf03079d46eabc06e15c1257780a>:0)
System.Collections.Generic.Dictionary`2[TKey,TValue].Add (TKey key, TValue value) (at <7c8bdf03079d46eabc06e15c1257780a>:0)
UnityEngine.TextCore.Text.SpriteAsset.UpdateLookupTables () (at <e493e4ea7412422f9b7eec7c629e14e8>:0)
UnityEngine.TextCore.Text.SpriteAsset.GetSpriteIndexFromHashcode (System.Int32 hashCode) (at <e493e4ea7412422f9b7eec7c629e14e8>:0)
UnityEngine.TextCore.Text.SpriteAsset.SearchForSpriteByHashCode (UnityEngine.TextCore.Text.SpriteAsset spriteAsset, System.Int32 hashCode, System.Boolean includeFallbacks, System.Int32& spriteIndex, UnityEngine.TextCore.Text.TextSettings textSettings) (at <e493e4ea7412422f9b7eec7c629e14e8>:0)
UnityEngine.TextCore.Text.TextGenerator.ValidateHtmlTag (UnityEngine.TextCore.Text.TextProcessingElement[] chars, System.Int32 startIndex, System.Int32& endIndex, UnityEngine.TextCore.Text.TextGenerationSettings generationSettings, UnityEngine.TextCore.Text.TextInfo textInfo, System.Boolean& isThreadSuccess) (at <e493e4ea7412422f9b7eec7c629e14e8>:0)
UnityEngine.TextCore.Text.TextGenerator.PopulateFontAsset (UnityEngine.TextCore.Text.TextGenerationSettings generationSettings, UnityEngine.TextCore.Text.TextProcessingElement[] textProcessingArray) (at <e493e4ea7412422f9b7eec7c629e14e8>:0)
UnityEngine.TextCore.Text.TextGenerator.PrepareFontAsset (UnityEngine.TextCore.Text.TextGenerationSettings generationSettings) (at <e493e4ea7412422f9b7eec7c629e14e8>:0)
UnityEngine.TextCore.Text.TextHandle.PrepareFontAsset () (at <e493e4ea7412422f9b7eec7c629e14e8>:0)
UnityEngine.UIElements.UITKTextJobSystem+PrepareTextJobData.Execute (System.Int32 index) (at <51d3c247e402427c8c078d62a0e45cb0>:0)
Unity.Jobs.IJobParallelForExtensions+ParallelForJobStruct`1[T].Execute (T& jobData, System.IntPtr additionalPtr, System.IntPtr bufferRangePatchData, Unity.Jobs.LowLevel.Unsafe.JobRanges& ranges, System.Int32 jobIndex) (at <c007129f0c8f49e5b1f7e125061a5718>:0)

IndexOutOfRangeException: Index was outside the bounds of the array.
System.Collections.Generic.Dictionary`2[TKey,TValue].TryInsert (TKey key, TValue value, System.Collections.Generic.InsertionBehavior behavior) (at <7c8bdf03079d46eabc06e15c1257780a>:0)
System.Collections.Generic.Dictionary`2[TKey,TValue].Add (TKey key, TValue value) (at <7c8bdf03079d46eabc06e15c1257780a>:0)
UnityEngine.TextCore.Text.SpriteAsset.UpdateLookupTables () (at <e493e4ea7412422f9b7eec7c629e14e8>:0)
UnityEngine.TextCore.Text.SpriteAsset.GetSpriteIndexFromHashcode (System.Int32 hashCode) (at <e493e4ea7412422f9b7eec7c629e14e8>:0)
UnityEngine.TextCore.Text.SpriteAsset.SearchForSpriteByHashCode (UnityEngine.TextCore.Text.SpriteAsset spriteAsset, System.Int32 hashCode, System.Boolean includeFallbacks, System.Int32& spriteIndex, UnityEngine.TextCore.Text.TextSettings textSettings) (at <e493e4ea7412422f9b7eec7c629e14e8>:0)
UnityEngine.TextCore.Text.TextGenerator.ValidateHtmlTag (UnityEngine.TextCore.Text.TextProcessingElement[] chars, System.Int32 startIndex, System.Int32& endIndex, UnityEngine.TextCore.Text.TextGenerationSettings generationSettings, UnityEngine.TextCore.Text.TextInfo textInfo, System.Boolean& isThreadSuccess) (at <e493e4ea7412422f9b7eec7c629e14e8>:0)
UnityEngine.TextCore.Text.TextGenerator.PopulateFontAsset (UnityEngine.TextCore.Text.TextGenerationSettings generationSettings, UnityEngine.TextCore.Text.TextProcessingElement[] textProcessingArray) (at <e493e4ea7412422f9b7eec7c629e14e8>:0)
UnityEngine.TextCore.Text.TextGenerator.PrepareFontAsset (UnityEngine.TextCore.Text.TextGenerationSettings generationSettings) (at <e493e4ea7412422f9b7eec7c629e14e8>:0)
UnityEngine.TextCore.Text.TextHandle.PrepareFontAsset () (at <e493e4ea7412422f9b7eec7c629e14e8>:0)
UnityEngine.UIElements.UITKTextJobSystem+PrepareTextJobData.Execute (System.Int32 index) (at <51d3c247e402427c8c078d62a0e45cb0>:0)
Unity.Jobs.IJobParallelForExtensions+ParallelForJobStruct`1[T].Execute (T& jobData, System.IntPtr additionalPtr, System.IntPtr bufferRangePatchData, Unity.Jobs.LowLevel.Unsafe.JobRanges& ranges, System.Int32 jobIndex) (at <c007129f0c8f49e5b1f7e125061a5718>:0)

Occasionally, after the errors mentioned above occur, the overall text rendering breaks (see video), leading to additional errors listed below.

uitoolkit

ArgumentException: The state of the provided MeshGenerationNode is invalid (entry isn't empty).
UnityEngine.UIElements.MeshGenerationContext.Begin (UnityEngine.UIElements.UIR.Entry parentEntry, UnityEngine.UIElements.VisualElement ve) (at <51d3c247e402427c8c078d62a0e45cb0>:0)
UnityEngine.UIElements.UITKTextJobSystem.AddDrawEntries (UnityEngine.UIElements.MeshGenerationContext mgc, System.Object _) (at <51d3c247e402427c8c078d62a0e45cb0>:0)
UnityEngine.UIElements.UIR.MeshGenerationDeferrer.Invoke (UnityEngine.UIElements.UIR.MeshGenerationDeferrer+CallbackInfo ci, UnityEngine.UIElements.MeshGenerationContext mgc) (at <51d3c247e402427c8c078d62a0e45cb0>:0)
UnityEngine.UIElements.UIElementsRuntimeUtilityNative:RepaintPanels(Boolean)

NullReferenceException: Object reference not set to an instance of an object
UnityEngine.TextCore.Text.TextGenerator.ParsingPhase (UnityEngine.TextCore.Text.TextInfo textInfo, UnityEngine.TextCore.Text.TextGenerationSettings generationSettings, System.UInt32& charCode, System.Single& maxVisibleDescender) (at <e493e4ea7412422f9b7eec7c629e14e8>:0)
UnityEngine.TextCore.Text.TextGenerator.GenerateTextMesh (UnityEngine.TextCore.Text.TextGenerationSettings generationSettings, UnityEngine.TextCore.Text.TextInfo textInfo) (at <e493e4ea7412422f9b7eec7c629e14e8>:0)
UnityEngine.TextCore.Text.TextGenerator.GenerateText (UnityEngine.TextCore.Text.TextGenerationSettings settings, UnityEngine.TextCore.Text.TextInfo textInfo) (at <e493e4ea7412422f9b7eec7c629e14e8>:0)
UnityEngine.TextCore.Text.TextHandle.UpdateWithHash (System.Int32 hashCode) (at <e493e4ea7412422f9b7eec7c629e14e8>:0)
UnityEngine.UIElements.UITKTextHandle.UpdateMesh () (at <51d3c247e402427c8c078d62a0e45cb0>:0)
UnityEngine.UIElements.UITKTextJobSystem+GenerateTextJobData.Execute (System.Int32 index) (at <51d3c247e402427c8c078d62a0e45cb0>:0)
Unity.Jobs.IJobParallelForExtensions+ParallelForJobStruct`1[T].Execute (T& jobData, System.IntPtr additionalPtr, System.IntPtr bufferRangePatchData, Unity.Jobs.LowLevel.Unsafe.JobRanges& ranges, System.Int32 jobIndex) (at <c007129f0c8f49e5b1f7e125061a5718>:0)
2

I was finally able to reproduce the problem in a very simple example. :partying_face:

It is quite interesting, because it depends on many states and conditions.

Basically, I just want to display icons (like gamepad button icons) on buttons (UIElements.Button). I’m doing this (deliberately) using sprites in rich text tags.

In certain situations, as mentioned in the post above, there’s a problem with displaying the sprites. In more complex user interfaces, this further leads to a complete breakdown of rendering some elements.

To display an icon, I have a sprite and a corresponding SpriteAsset (sprite mode: multiple) with defined icons as in the image below.

image
image768×439 52.4 KB

SpriteAsset is assigned as default sprite asset in PanelTextSettings. To display the icon on a button, I use a rich text tag (for example, <sprite name="buttonSouth">), as described here.

image
image832×344 45.5 KB

To create buttons with icons, I use the following code. Basically, I iterate through the sprite names that are serialized and display them on the button.

If I run this example, errors appear in the console, and the sprites are broken. However, if I, for example, remove the label (which is completely useless and can even be empty), the sprites render correctly. Similarly, if I print the number of sprites in the sprite asset to the console (by accessing spriteAsset.spriteCharacterTable’s getter), the problem is fixed.

public class Test : MonoBehaviour
{
    public SpriteAsset spriteAsset;
    public string[] gamepadButtonNames;

    private void OnEnable()
    {
        var container = GetComponent<UIDocument>().rootVisualElement;

        // When I remove this line/label, the sprites are displayed correctly without exceptions.
        container.Add(new Label("This label magically breaks everything."));

        // When I add the following line (and therefore access the sprite asset's spriteCharacterTable),
        // the sprites are displayed correctly without exceptions.
        //Debug.Log(spriteAsset.spriteCharacterTable.Count);

        foreach (var gamepadButtonName in gamepadButtonNames)
        {
            var button = new Button();
            button.text = $"{gamepadButtonName}<sprite name=\"{gamepadButtonName}\">";
            container.Add(button);
        }
    }
}

The problem is also somehow font-dependent. I’ve included two fonts in the project, with one the rendering breaks, while with the other it’s fine.

image
image1011×507 36 KB

It’s reproducible in Unity 6000.0.44f1 and 6000.1.0b12. Please note that this problem occurs very frequently, but not in 100% of cases.

A minimal example project is attached here: UIToolkitTextSpritesBug.zip (217.5 KB)

P.S. I’ve also reported the bug through Unity Hub: Unity Issue Tracker - “ArgumentException” errors thrown when adding Buttons with sprites to a VisualElement that has a Label