Creating Custom Runtime Visual Elements

Hi, I’m trying out the workflow for creating custom VisualElements and have got a certain way from the docs and afew other forum posts, but could do with some help. Alternatively if there are some good tutorials on this topic, please point me towards them.

I decided a reasonable usecase would be a localizable “LocLabel”. I have custom localization system working in uGUI, that for strings is very similar to I2 Loc from the asset store:

6973199--822161--upload_2021-3-25_14-45-24.png

So I decorate a Unity Text Component with a LocText and (optionally) LocTextStringParams. For setup in UXML, my thinking is that it makes sense to:

  • Derive a LocLabel from Label that holds a localization key and most of the logic

  • Derive a LocParam from VisualElement, such that each LocParam holds a single parameter and value

  • Multiple parameters would require multiple LocParams

  • LocParams would be VisualElements, but always set to Display : none

So XML layout for the above looks like:

    <RTM.Localization.LocLabel loc-key="CONTROLS/KEYBOARD/SELECT_ABILITY_NUM" name="SelectAbility">
        <RTM.Localization.LocParam name="NUM" value="50" />
        <RTM.Localization.LocParam name="OTHER_PARAM" value="TEST/OTHER_PARAM" localize="true" />
    </RTM.Localization.LocLabel>

First Question

  • is this even a sensible approach? Having hidden VisualElement children as an array of values feels slightly hacky, from a c# perspective, but feels very sensible from a UXML perspective, but maybe I’ve missed some features? Possibly something with the userData?

Assuming the answer to that is yes, here’s the implementation thus far (I’ve stripped out all the runtime translation parts, this is just the UIToolkit implementation:

    public class LocLabel : Label
    {
        public LocKey locKey { get; set; }

        public override string text
        {
            get => base.text;
            set => base.text=value;
        }

        public class UXMLFactory : UxmlFactory<LocLabel, UXMLTraits> { }

        public class UXMLTraits : Label.UxmlTraits
        {
            UxmlStringAttributeDescription locKey = new UxmlStringAttributeDescription{ name = "loc-key", defaultValue=""};
            UxmlStringAttributeDescription text = new UxmlStringAttributeDescription{ name = "text", defaultValue=""};

            public override IEnumerable<UxmlChildElementDescription> uxmlChildElementsDescription
            {
                get { yield return new UxmlChildElementDescription(typeof(LocParam)); }
            }

            public override void Init(VisualElement ve, IUxmlAttributes bag, CreationContext cc)
            {
                base.Init(ve, bag, cc);
            
                var locLabel = ve as LocLabel;
                string oldKey = locLabel.locKey;

                //locLabel.Clear();

                locLabel.locKey = locKey.GetValueFromBag(bag, cc);
                locLabel.text = text.GetValueFromBag(bag, cc);

                // TODO: need to set localized text value and ensure that it gets pushed to XML
            }
    public class LocParam : VisualElement
    {
        public class UXMLFactory : UxmlFactory<LocParam, UXMLTraits> { }

        public string value { get; private set; }
        public bool localize { get; private set; }

        public class UXMLTraits : VisualElement.UxmlTraits
        {
            UxmlStringAttributeDescription value = new UxmlStringAttributeDescription{ name = "value", defaultValue=""};
            UxmlBoolAttributeDescription localize = new UxmlBoolAttributeDescription{ name = "localize", defaultValue=false};

            public override IEnumerable<UxmlChildElementDescription> uxmlChildElementsDescription
            {
                get { yield break; }
            }

            public override void Init(VisualElement ve, IUxmlAttributes bag, CreationContext cc)
            {
                base.Init(ve, bag, cc);

                var locParam = ve as LocParam;
                locParam.Clear();
                locParam.value = value.GetValueFromBag(bag, cc);
                locParam.localize = localize.GetValueFromBag(bag, cc);

                locParam.style.display = DisplayStyle.None;
            }
        }
    }

This kinda works somewhat how I’d expect, but with a few issues / queries / roadblocks:

  • Calling Clear() (commented out on line 30) in LocText would make the children vanish in UI builder whenever I made a change in the UI. They were still in the XML and returned if I closed UI Builder and reopened the file. Why does clear do this and when / why should I be calling it?
  • Once children have been added to a label (even children with Display : None) it stops using the text to determine it’s rect and uses it’s children, so becomes zero height. Is it possible to return this behaviour to the Label behaviour or this a stumbling block of my approach?
  • I can’t see any way to push values down to the XML. Ideally when handling the localization key, I will update the Text field and then that will be serialized in the XML. There seems to be an assumption that in c# you only read from XML and writing to it is done manually via GUI or text editor
  • Is there any support for Customization of the UIBuilder Inspector yet? - in my Component-based version I have a dropdown to select keys but of course by default it’s just a string field in UIBuilder
  • Is there any way to get a per-update, or pre-repaint call to my VisualElement? In components when updating the key or parameters, I flag the LocText as dirty, then process the changes at the end of the LateUpdate (just prior to UI), so that I don’t process multiple changes in a single frame.
  • When can you gather references between VisualElements? In the UxmlTraits.Init() the parent is null and the childcount is 0, regardless of the heirarchy. Is there some other init step that I can use to find references

One approach that you could use would be to consolidate LocLabel and LocParam together and use UxmlStringAttributeDescription to contain comma-separated values. That would avoid adding children to your label (which, while it’s not currently enforced, should not contains children, per Unity - Manual: UXML elements reference).

Regarding the call to Clear:
The UI Builder may call Init multiple times (i.e. changing an attribute from the builder, undo/redo, etc.) Even though your call to Clear() removes the children in the builder, this is not propagated to the UXML file because it is not a change that the user is making from within the builder.

I want to challenge this idea here. You actually want localization to be resolved at runtime. If you could set a localization key and have it update all the related labels in the uxml asset itself, that would mean that anyone opening the asset in the builder with a different locale could end up making a change to the uxml asset, which is not desirable for collaboration.

One thing you could do is manually set the text to the default value for the localization and then rely on the Init call from the traits class to resolve the correct value.

No, not at the moment.

Do you intend to change the localization key or parameters at runtime?

2 Likes

Hi, thanks for the detailed response.

Yeah, this is one approach, but feels very fiddly for user input, especially parsing potential commas inside csv data (this gets painful very very fast) and basically require support for PropertyDrawers / EditorWindows equivalents for UIBuilder IMO.

I did see that the label wasn’t supposed to have children, I just hoped that if I override that in my subclass it would allow it. I guess I was expecting the behaviour of a flex-box <P> tag.It does seem odd that the layout breaks, even if all children are display:none. Because adding invisible tags in XML / UI builder feels like a much nicer way of decorating data than embedded CSVs. But I’m guessing this is well beyond scope to get “fixed”?

This is entirely down to workflow nuances, and what you describe is not actually that much of a problem. But I for that specific instance I can work around it. Though I think that there’s still an underlying question of why there is no automation for data entry? Because I can see that there is an argument against in a specific case, but not in a general case. Again, is this something that might be supported in a future implementation of PropertyDrawers or similar for UIBuilder? Is there a plan for that kind of feature?

Absolutely, like a lot - just think of any infobox or item display in any game - that’s one screen you update the text on, not a baked in separate asset for each item.

The key thing here is that I don’t want to update the output value as soon as a change is made. i.e: Set Key, Set Param, Set Param should not trigger 3 updates. Each one marks it dirty and then updates the text as late as possible in the frame. I’ve worked around this with some callbacks from a MonoBehaviour LateUpdate with very late execution order. Unfortunately there’s no way to track which locLabels are displayed or not so right now I’m pushing that to every label that exists in the game (which is very few right in my test cases but is going to be pretty inefficient later).

I opened another thread with feedback about some lacking events here and while I don’t fully agree with some of what I said in the OP, there’s definitely some value to be had in enhancing the event system by adding some events that are more valuable for Runtime UI than Editor UI

This is because the underlying layout engine that we use (Yoga) only calculates the layout from leaf elements, so when adding a child to the Label, it treats it as a container.

At some point, we’ll need to support localization in a more built-in way.

Extensibility of the UI Builder is something we are interested in, but there are a few missing pieces internally for us to get there.

Yeah, that makes sense. So I can see different use-cases:

  • I need to set all the visible localized texts when I start my game/app
  • I need to dynamically update all localized texts whenever a user changes the language in the settings.
  • I need to update localized text of some containers in a very dynamic fashion (the use-case you mentioned).

So I see 1. and 2. as being pretty much the same and they could be handled by having the localization manager keep a collection of all LocLabel currently being used. This can be done by doing something like this:

class LocLabel
{
    public LocLabel()
    {
        // Localization.Register could resolve (or not) the localization for the current settings and update the LocLabel,
        // so it's always resolved when it is added to the hierarchy.
        RegisterCallback<AttachToPanel>(evt => LocalizationManager.Register(this));
        RegisterCallback<DetachFromPanel>(evt => LocalizationManager.Unregister(this));
    }
}

By using the AttachToPanel and DetachFromPanel, you should be able to reduce the set of labels to update to what is currently in the UI hierarchy.

  1. could also be achieved by using the above if you set the localization key before adding it to the hierarchy. In other cases, if you must delay and bundle updates together, one way to achieve this would be to register a callback on changes to the localization key (and/or params) to add it to an update queue on the localization manager, which could be processed in your LateUpdate call.

so going back to pushing the text down to XML. Part of the reason why I want that I now remember is that I somehow don’t get calls to my UXMLTraits.Init() when I do a script change, but the preview of the UI is reloading the serialized label text from xml. Maybe built-in VisualElements are getting init()ed but derived ones are not?. Either way every script load trashes my preview and I have to make manual changes in UIBuilder to refresh. which is really not great for workflow. Going back to my original point - letting us push data to XML is pretty valuable. But on top of that, is this init() behaviour a bug?

As for Unity implementing loc - I implore you to put that right to the very back of the backlog. You have core systems that need work and loc is a really bad fit for general purpose engine features. “General Purpose” loc is a huge task and you’re extremely unlikely to produce nice workflows that work for all use cases - and it too will need lots of workarounds to massage it into shape. it’s better to just let people build their own that fits their workflows. And most people don’t need most of that stuff. Just stick to getting the core systems that everyone needs right before chasing “nice to haves”

I can also work around some of these other things - I already have the attach/detach handlers for example - but I have to say it’s very painful having to work around the limitations of UIToolkit right now. It’s extremely obvious that it’s been designed for editor UI in first principles and is now functional at runtime, without a proper consideration for what runtime UI needs. And I appreciate that it’s still early days, but I really hope you’re looking to take on board suggestions and improve usability over time. The foundation here is good, but at 1.0 it’s nowhere near ready as a replacement for uGUI yet, and needs a lot of improvement before it will be.

The Init call is done when cloning or instantiating the visual tree. It should get called every time you change one of the attributes (under the LocLabel foldout) as it does in the attached gif.

I’m not saying it’s a priority at the moment.
That being said, creating a localization system should not be done in the UI directly, it should be its own system and then the UI can leverage it.

And I appreciate that it's still early days, but I really hope you're looking to take on board suggestions and improve usability over time
That’s why we are taking the time to answer the forums. :slight_smile:

6991406--825683--init.gif