UI Toolkit Rant

Unity Engine
,
1

Hello Everyone
Before I start ranting, I would love to give this DISCLAIMER:

I love Unity and I love the hardworking team behind it, I do agree that Unity needed a major overhaul for their UI system, and the general approach seemed good to me.
Also I do understand the difference between Unity’s use case for the UI compared to other UI frameworks and the fact that Unity had to reinvent the wheel for so many components due to the fact that this is a game engine, not some UI framework running natively on one OS.
I don’t want this to sound like some whiny entitled post

Background
I have been using Unity for years now, I started working on unity Since Unity 4.0, and I have witnessed the ERA moving from IMGUI to uGUI, and it was a great and major update at the time.
Also it might be very important to know that I’m working in a team making a game that works with VERY HEAVY UI workflow, as a matter of fact, 90% of our game is just UI, so our case might be different from doing a simple HUD for a shooter game.

However, Unity UI system was always fine for these use cases, doing a simple HUD was a trivial task and required no overhaul for the system, the problems start to arise when a real use case for a UI intensive app is rapidly scaling.

We have been facing some very big issues for maintaining our UI in a huge project, we have been working before nested prefabs, and making components based architecture wasn’t trivial at all, but somehow we managed, but the app kept growing larger and larger, and soon we started to have some performance issues just to animate few elements and to use a recycler view for a very active chat thread and so on.

The app was getting really sluggish for an app with only menus and buttons and lists, compared to writing some UI mobile apps using native languages or even frameworks like flutter or react, it was extremely painful.

Of course I can’t blame all the performance issues on Unity, we had our fair share of mistakes and rushed code, but I have to say making UI performant in Unity is way more tedious task compared to those frameworks

When the new UI system got announced, a glimpse of hope sparked in our souls.

Rant Start

We kept waiting for months - or few years now - for the new UI to mature enough so it can be used in production and enough online resource can be found to solve problems, can’t say we are there yet, even when we were contacted by Unity’s success team they didn’t advise us to use it in production yet - this was around a year ago -

Today I decided to evaluate the system and see if we can adopt it soon, because the app really needs some heavy optimization, and we are considering alternatives to Unity if the new UI system wasn’t good for our needs.

So I started following this tutorial:

It seemed VERY long to me to make a very simple list, but I said I will keep my judgment till I’m done, maybe it’s just too detailed of a tutorial, Alas, my first hunch was actually correct.

Building a UI using the UI builder seemed good and intuitive enough, although it had some real problems like all the buttons show the same tooltip for example, and I don’t know what I’m clicking exactly if it weren’t for the tutorial. but At least generating a screen with one file asset that can be parsed somewhere seemed very good to me.

TLDR: you can start reading from here!

The real problems started when I reached the coding part, and this is where the core of this long post is, every decision made in this area by Unity seemed dead wrong to me!

Starting with how you get elements from the visual tree, I mean:

// Store a reference to the character list element
m_CharacterList = root.Q<ListView>("CharacterList");

Seriously? get element by a string ? This was hard to use when we used to code on Android gingerbread almost 10 years ago although the IDE offered some help back then to match the IDs

The fact that you can’t attach scripts to your UI elements (MonoBehaviour or not, I know they are not game objects), the fact that we can’t make a “prefab” that has access to some scriptable object, but rather the tutorial suggest we load the data this way:

m_AllCharacters.AddRange(Resources.LoadAll<CharacterData>("Characters"));

I mean this can simply kill our entire architecture, we should have some gigantic script as MonoBehaviour and then pass everything to the children though custom code, and then pass the data in a none typed object

newListEntry.userData = newListEntryLogic;

We also had to pass the item that will be dynamically created from the parent MonoBehaviour and then we kept reference to it so we can instantiated it, with assigning its height from code!

Assigning listeners and callbacks from code as well because the inspector can’t help you here since nothing is a game object

I can go on and on about this but this is just few mind blowing decisions I have seen while trying to explore the new system, and if I wanted to compile a list of what I think should be burned to the ground, this post would have been much longer

REALLY TLDR:
It seemed to me Unity is adopting the old web standard because everyone is familiar with it at the time that everyone is ditching it and evolving to something new, you can see what some frameworks are doing with WebAssembly instead of JS and jQuery stuff, but Unity decided that jQuery is a good part of the web and we should do uQuery ?
This seems to be the same way when Unity decided to support JavaScript to make scripts to draw the attention of the Armada of armature developers who are familiar with JavaScript, but years later they had to ditch it since it couldn’t keep up

Final words:
I really don’t think this is usable in any way for us, and I think coding a UI that is remotely serious and not a school project is a really painful task.

Is this really how it’s planned in the future ? is this how we are supposed to code our UI? or the system is still very early to use and some major scripting tools are coming?

24 Likes
2

The way you solve both these problems is by subclassing VisualElement. See the docs here:
https://docs.unity3d.com/Packages/com.unity.ui.builder@1.0/manual/uib-structuring-ui-custom-elements.html

My entire UI is built from custom elements and each custom element keeps pointers to its components. I never look up elements by string because the each custom element already knows where its children are. I don’t bother with XML at all–the top of each page is a custom element which builds the rest of the page. You can create custom elements using UI builder and XML–for more static pages that works. For a more dynamic UI I find it easier and more efficient to generate the entire DOM in code.

2 Likes
3

UI Elements is very flexible. Unlike @Arkenhammer I prefer to keep Visuals and Code seperated, I don’t have gigantic MonoBehaviours, instead I have a very simple MonoBehaviour creating small C# classes, which then run the logic.

This looks somewhat like this:

public class Main : MonoBehaviour{

[SerializeField] UIDocument uiDocument;

// not serialised (but could be), public for cross access when needed
public MenuView menuView;
public ContextView contextView;
public CrosshairView corsshairView;

void Start(){
   menuView =  new MenuView(this, uiDocument.Q<VisualElement>("MenuRoot"));
   contextView=  new ContextView(this, uiDocument.Q<VisualElement>("ContextRoot"));
   corsshairView=  new CrosshairView(this, uiDocument.Q<VisualElement>("CrosshairRoot"));
}

}

Inside the Constructor, it get’s all elements like so:

// not a MonoBehaviour
public class MenuView{

Button openButton, closeButton, cancelButton;
VisualElement root, entryParent;
public List<MenuEntries> entries;

public MenuView(Main main, VisualElement root){
   this.root = root;
   entryParent = root.Q("EntryParent");
   openButton = root.Q("OpenButton ");
   closeButton= root.Q("CloseButton");
   cancelButton= root.Q("CancelButton");
   Init();
}

void Init(){
   openButton.RegisterOnClickCallback(()=>{ /* Do stuff */});
   closeButton.RegisterOnClickCallback(()=>{ /* Do stuff */});
   // ... and so on
}

}

I don’t care how the UI is built, this is decided by the UI-Design-Guy in my team. I just write functionality and it works perfectly. I can structure my code however I want to, of course I can re-use things and I can also use MonoBehaviours (including Prefabs with set references, or even ScriptableObjects with references).
All they need is their “root” which is referenced by string (and all relevant sub-elements which are also referenced by string).

Also there are kind-of-prefabs in the UIElement itself … I think they are called “TemplateElements” and I use them whenever I build a List in the UI.

What I try to say is that it is very flexible. You can’t drag&drop reference things, yes, but everything else is possible. And there is definitely no need for gigantic MonoBehaviour Classes. Never ever. xD

3 Likes
4

We also have an UI heavy project. We’ve switched from UGUI about and year ago. What I can say is, that we’ll never come back to UGUI, with all its gameobjects and performance issues.

Yes, UIToolkit is still in development and have many issues or missing features (although in our case 90% of our needs are covered), but it’s much better (IMHO) than the old system.

When I’ve started using UIToolkit, I had same questions and it was kind of difficult to switch from Monobehaviour paradigm, projecting UGUI approach to UIToolkit didn’t worked for me, I had to step back and rethink it. Then it went very well. It’s kind of same as іf you’ve switched from Monobehaviour to DOTS (again kind of).

Give it one more try, maybe you’ll like it :slight_smile:

1 Like
5

Thanks everyone, I appreciate your feedback.

regarding to what Arkenhammer said, I understand that you can extend the UI elements, but I really don’t think this will scale well at all, extending every element just to make a minor change in the UI isn’t really good for large project, we would have thousands of components this way, Extending UI elements should be for special cases, and this won’t solve the binding issues.

The same can be said to Hannibal_Leo, although I like your approach better, but the last time I constructed UI with code by referencing some elements and then construct the visual tree by code is when I was learning java swing in college!

I’m not sure if I’m delivering the problem correctly here, but this will lead to thousands of lines of code. Visuals and code MUST be separated you are correct, but the code should be very minimal, the UI builder its self is nice and easy to use, but once I’m done, I should be able to add functionality with a line of code to it.

Take this as an example, I want some reusable status button that shows connectivity to our servers, the connection status its self is held in a scriptable object, actually every UI element that is interested in this status can just reference the scriptable object and serialization will take care of it, I won’t make an architecture illustration here, but this helps so much with reusability, testing when mocking, interfacing, and so on, I can’t stress how bad doing these stuff through code for the architecture.

6

It could totally be my ignorance, and if I gave it a bit longer I might get used to the new workflow, it’s not like uGUI was so good in the first place

That being said, I guess my rant is mainly because we are getting something new, a component that is in development in 2022, and we are still using a very old way to handle stuff, I just wanted to know if this is going to be the case even in the future or this is something temporary and more tools are coming.

Compare this to any UI framework, they can brag about how easy you can make a page with like 5 lines of code, auto mapping, compiled code combination between the view and the code classes, auto binding between properties and their fields.

If you have worked with any good modern UI system you would see how ridiculous and absurd that I have to get a reference for an input and then assign a listener to it to handle onChange events!

usually the “how to” tutorials in any framework make it look so easy - although problems start appearing later - , but the fact that Unity themselves are suggesting this approach to code the UI in their very simple one list view tutorial

I dunno man …

2 Likes
7

The way I handle this kind of situation is to stash a context object in the root element in my visual tree. That context object is a game object and can hold references to other game object and scriptable objects. Any visual element in the tree can reach up to the root to get at that data. In my game I hold a scriptable object which vends sprites, my string look up database (so my UI can be translated) and a bunch of more game-specific data.

As for how I handle custom elements, here’s some sample code. The key for organization is in my base class (creatively named BaseVisualElement) which defines an abstract property string UssClass. The base class methods assign the class to created sub element like so base-class__sub-class so, for instance, calling MakeElement("divider") returns a visual element with class “tab-bar__divider”. Building the UI this way both creates modular re-usable elements and enforces a fairly structured relationship with the stylesheet.

This particular example is a port from my old UI which was build with prefabs and attached MonoBehaviors. At least for my needs, the new version is both easier to work with and much more performant.

using System;
using System.Collections.Generic;
using UnityEngine.UIElements;

namespace Robants.UI.VisualElements
{
    public class TabBarElement : BaseVisualElement
    {
        public override string UssClass => "tab-bar";
        static readonly string[] _strings = new string[] { "tab-ids" };

        public new class UxmlFactory : UxmlFactory<TabBarElement, UxmlTraits> { }

        public new class UxmlTraits : VisualElement.UxmlTraits
        {
            readonly Dictionary<string, UxmlStringAttributeDescription> _stringDescriptions = VisualElementUtils.GetStringDescriptions(_strings);

            public override void Init(VisualElement ve, IUxmlAttributes bag, CreationContext cc)
            {
                base.Init(ve, bag, cc);
                TabBarElement mpe = ve as TabBarElement;
                mpe.AddToClassList(mpe.UssClass);
                mpe._stringValues = VisualElementUtils.GetStringsFromBag(_stringDescriptions, bag, cc);
                mpe.EnsureSubtree();
            }
        }

        Dictionary<string, string> _stringValues;
        string[] _tabIds;
        bool _initialized;
        Dictionary<string, SimpleButtonElement> _tabs = new Dictionary<string, SimpleButtonElement>();
        public string SelectedTabId { get; private set; }
        public Action<string> OnSelectTab;
        const string SelectedButtonClassName = "tab-bar__tab-selected";

        override public void EnsureSubtree()
        {
            if (!_initialized)
            {
                _initialized = true;
                AddToClassList(UssClass);
                if (_stringValues != null && _stringValues.TryGetValue("tab-ids", out string tabIds))
                    SetTabsIds(tabIds);
            }
        }

        public void SetTabsIds(string tabIds)
        {
            if (tabIds == null)
            {
                SetTabs(null);
                return;
            }
            SetTabs(tabIds.Split());
        }

        public void SetTabs(params string[] tabs)
        {
            EnsureSubtree();
            _tabIds = tabs;
            Clear();
            _tabs.Clear();
            if (tabs == null)
                return;
            int count = 0;
            for (int i = 0; i < tabs.Length; ++i)
                if (tabs[i] != null) ++count;
            if (count == 0)
                return;
            int tabNumber = 0;
            for(int i = 0; i < tabs.Length; ++i)
            {
                if (tabs[i] == null)
                    continue;
                ++tabNumber;
                SimpleButtonElement button = MakeSimpleButton("tab-button");
                Label label = MakeLabel("tab-label");
                string tabId = tabs[i];
                label.text = GetTabLabel(tabId);
                button.Add(label);
                button.OnClick = () => SelectTab(tabId);
                _tabs[tabId] = button;
                Add(button);
                if (tabNumber < count)
                {
                    VisualElement divider = MakeElement("divider");
                    Add(divider);
                }
            }
            UpdateSelectedTab();
        }

        public void SelectTab(string tabId)
        {
            if (tabId != SelectedTabId)
            {
                SelectedTabId = tabId;
                UpdateSelectedTab();
                OnSelectTab?.Invoke(tabId);
            }
        }

        void UpdateSelectedTab()
        {
            foreach (var pair in _tabs)
            {
                if (SelectedTabId == null)
                    SetTabSelection(pair.Value, false);
                else
                    SetTabSelection(pair.Value, pair.Key.Equals(SelectedTabId));
            }
        }

        void SetTabSelection(SimpleButtonElement element, bool selected)
        {
            bool isSelected = element.ClassListContains(SelectedButtonClassName);
            if (selected != isSelected)
            {
                if (selected)
                    element.AddToClassList(SelectedButtonClassName);
                else
                    element.RemoveFromClassList(SelectedButtonClassName);
            }
        }

        public string GetTabLabel(string tabId)
        {
            if (GetUIContext() == null)
                return tabId;
            return GetUIContext().UIStrings.LabelForTabId(tabId);
        }
    }
}
8

Not going to hijack this thread with advertisement for my own asset, but many of the points you mention here are handled in Interface.

In fact, Interface was made for many of the reasons you mention here. The new UI system is very good, but the workflow doesn’t suit Unity.

Interface allows you to add scripts directly to elements. You can reference scriptable objects. Component based, etc. etc.

So yeah, if you are interested I can tell you more on discord. Other than that, I agree with many of your points. The new UI system is better, and the technology is great. But as you said, it is adopting the old web standard instead of adjusting the workflow to fit Unity. Unity has an object based workflow, but the new UI system tries to force a more function based approach like on the web. Will be interesting to see where it will go from here.

9

Hello @HassanKhallouf ,

It feels like the main thing that bothers you at the moment is the need to reference objects by strings, which can be considered “fragile” depending on the workflow you want to adopt.

This is a very common bit of feedback. If you look at the UI Roadmap you’ll notice we are currently working on data binding.

This will unlock a workflow where you will be able to associate data properties, that can be strongly typed, within your UXML document. We are also looking at supporting event bindings this way.
This should remove the need to reference elements by class or name for the majority of use cases, e.g. binding data and logic to your UI layouts.

We understand that UI Toolkit isn’t fully ready to support game production for everyone, but for some users the added value is already real, hence our strategy to gradually release and improve the system.

That is why today we are presenting it as an alternative, while we are still working at the improving the workflow and add missing features.

4 Likes
10

This is exactly what I was trying to avoid, a context object in the root distrusting the data to children, this won’t scale well at all in a large project and won’t help with reusable components very much as I have explained before

I REALLY apricate your feedback though and for taking the time to bother with my post

11

I took a quick glance at your asset and it seems some really advanced stuff
we can talk more in details about it in private messages or discord, won’t create much noise for everybody in here

but this proves the point of the original idea, that a huge part is still missing and we need custom solutions for what should be out of the box

1 Like
12

Thanks Antonie, I really appreciate the response
I came across the roadmap yesterday and saw that you have databinding and events binding in the road map and I felt hope again :stuck_out_tongue:

I haven’t been following the development of UI toolkit closely until very recently, but I had to share this feedback although I knew it will be something common, as what I’m asking is the basic use case before I go any levels deeper into the components we are using right now

but I wasn’t able to find anything about how the new solution would look like, is there any development thread we can follow ?

I wouldn’t say this is the main thing but it’s a huge part of it
Databinding and Referencing would be my headline of missing features
but being able to script components and insert them without code is also a major feature for any architecture in my opinion

I will give you 2 quick examples to illustrate the problem

1- Assume I have a component that shows the player profile, the data is coming from a scriptable object, how does this SO get the data is irrelevant to the UI, I should be able to save this component with a reference to the SO (pretty much like how a prefab does) and just drop it in any screen or other components without the need to pass the SO from a parent, so even if the parent had some unity serialization, the sub components should be functioning on their own regardless if they were instantiated dynamically or dropped in the scene directly, I don’t know what the system you guys are working on but this involves a direct script always linked to this component, a lot of UI frameworks they compile both the view and the code in one class, but I doubt this is what is going on here

2- Supplying a type to be used in a layout, if I had a scrollable list of users, each entry is a component on it’s own, it’s very tedious and hard to manage when I have to script something to instantiate them by code one by one and to keep track of how they should behave! sometimes you need this at run time like a chat list view, other times I should be able to do something like


and that’s it!

I guess enabling custom script per component will solve most of the issues and at least will open the possibility of creating solutions even when they are not out of the box

combine that with databinding, and we are looking on some really good iteration of Unity UI systems

1 Like
13

@HassanKhallouf You’re frustration is understandable. Compared to the React framework, UI Toolkit feels like taking three giants steps backward. However, browser rendering engines have decades of development behind them, and React itself is almost a decade old with an enormous community and plenty of open source add-ons. Unity is starting from square one here and I see nothing but potential. Yes, data binding, life-cycle management, better svg support, and better element styling are still missing. I’d rather they roll out an unfinished product and source feedback from the community than keeping us in the dark for countless years.

2 Likes
14

Honestly it would be hard to disagree. I feel like complex VisualElements should have a UnityEvent/bindable property that can use scriptable objects to pass data from UI to game code and vice versa. Like buttons for example, even attributes could be used. I’m not sure why Scriptable Object and Unity Event compatibility wasn’t built in in the first place. Honestly all they need to do is have the unity event property drawer in the UI Builder for the canvas events. It should theoretically be possible to change the UI Builder and make a EventElement that builds UnityEvents as needed.

15

I wonder if I’m just old or stupid but why even use UXML DOM when you only ever keyhole it though root.Q queries. There should be full type safety and binding at edit time, there is no reason not to, this is .NET, not a JavaScript based compiler and software.

The DOM should be 99% code, not markup. Think Jetpack Compose, or for unity, whatever the final name of the evolution of UIForia is going to be (I’ve glimpsed its internals and it’s pretty glorious)

Or maybe I’m just old and think nothing was wrong with Qt4 and Borland VCL, and that Qxml was a step backwards.

React is extremely bad, don’t let anyone tell you otherwise. That’s why it changes its fundamentals every 2-3 years

5 Likes
16

TemplateElements. You create the LoadingBar or whatever as template element and wherever it’s used, you add the functionality by locating it (by name - this can be done for the whole UI, using the Querry system. You can get all instances of your template at the start of the game, but you can also add them when you create them at runtime) and combining them with the corresponding C# class.

You can also use a hierarchy of empty gameobjects, each referencing an Element in the UIDocument, which would provide pretty much the same functionality as the existing workflow, you would just need to write a base component that finds the UIElement you want by name. But the new workflow allows for a better structure.

You could also write a component that finds UIElements by a “path variable”.
→
(untested code below, just for demonstration purposes)

public string path = InGame/HealthUI/LowHealthWarningText;
public UIDocument uiDocument;
public string setText = "Health very low ";
public int surfixValue = 5;
Label myLabel;

void Start(){
    string[] subPaths = path.Split('/');
    VisualElement current = uiDocument.rootVisualElement;
    foreach(var subpath in subPaths)
        current = current.Q<VisualElement>(subpath);
    myLabel = current as Label;
}

void Update(){
    myLabel.text = setText + surfixValue;
}

So there is really only one thing you have to pay attention to: getting the Elements by String. Everything else is completely up to you. You can create any architecture you want to. You can write code into the elements themlselfs, you can write them in plain C#, you can write them in MonoBehaviours or ScriptableObjects, you can search everything at the Start of the game or construct the UI completely from code, out of templates, whatever fits your project best.

17

Thank you for your input!
I hope you understand that what you are suggesting is a very long hacky solutions for what we used to do by litterly dragging an element from the hierarchy to the inspector!

I want a circle element to be green if a certain value in a scriptable object is true, red otherwise, can you see how many hoops you had to go through to achieve this in the new UI system ?

Of course I can make my own query system, and I can get all instances (we have thousands btw) at start, you can also make your entire UI into a singleton class that passes references to whoever is interested!

you also suggested making a hierarchy of empty gameObjects to mimic the old workflow, why wouldn’t I just USE the old workflow?

That’s one of my main complaints! I don’t want to pay attention to strings, nobody should pay attention to strings! of course we can build a system to analyze and automate and bind etc etc, people have done this for decades now in the web realm, and they had to do this because of how ***** JS is! thy built jQuery, and then they built a whole eco system around jQuery, and years later it’s still horrible and outdated and makes you pull your hair while using it!

Mind you, all of this was done for UI scripting, here it’s a game engine and you have to mix it with tons of logic!
Can’t you see how unacceptable to get an element by string ID in the C# world? have you seen how wpf does elements binding more than 10 years ago now ?

everyone is now moving to strongly typed languages like TS or use webassembly to use your language inside the browser like blazor for example, and the main complaint is that unity is picking of where everyone has already finished

18

@HassanKhallouf

Thank you very much.

Your initiative generated a very interesting debate, making me go to the new interface

1 Like
19

As you stated in your very first post of this thread: performance. And also: formatting. I worked on a project with localization and we needed auto-resizing textboxes inside of auto-resizing List views (just a simple SMS-Chat-UI for ingame texts) and oh dear, was that a pain. ForceRebuildLayout madness at every single text-element, every list with elements that are dynamically loaded and can have varying size (images, attached to chat messages).

I built a pretty similar system with the new UI and it worked like expected. Things resized properly and affected their parent elements, lists grew and I could setup everything in any combination I wanted to without writing code to make it work the way it should.

Also: uniform style. You have one style-sheet and changing a button there will change all buttons with this style tag accordingly. With Prefabs this is a horrible experience. I had a prefab for the background of texts, another one for backgrounds of buttons, another one for … and so on, changing the shape of it (round corners for examle) meant I had to go to their prefab-variants (because background has to be the top most one and thanks to the whole resizing thing, I couldn’t just put it as child of an empty) like the actual button-prefab, resizable-button-prefab and so on, to change the outline accordingly.

Oh and don’t get me started with sorting … creating a canvas inside a canvas just to make things appear on top of other things despite their hierarchy position (hover text on buttons inside the button prefab for example) - no thanks. xD

All of that was a massive pain.
And especially in an UI-heavy project, I’d personally not even think about using the old UI-System.

Not really hacky, it’s just the possibility to build your own system. Of course initially it seems like more work, but as always, this pays off massively in the long term. If you write your system for your company, you can include any workflow you want. Also Unity is actively extending the whole UI-Toolkit System, we will see some pretty helpful features (especially in the UI-Builder) soon. See the new Roadmap 2022 Video they uploaded.

The old system won’t get anything else than some bugfixes as far as I can tell, so it will forever have all the issues mentioned above.

But I don’t want to push you in any direction here. All of this is just my personal experience and thoughts about the two UI-Systems. You should make the decision based on your personal needs or the needs of your company.

20

Do you have any more information about how you intend to implement this and how it will work?