Any good way to find out if a keyboard key is pressed with the mouse over a VisualElement?

I’ve got an editor window that shows a bunch of VisualElements in a grid. I want to know if the user presses “b” with the mouse over one of the elements, and if they do, I want to know which VisualElement it is.

I wanted to add a KeyDownEvent to the VisualElements themselves, but that only fires if they have “focus”, which isn’t applicable at all here. So my current hack is:

// in the specific visual elements:
RegisterCallback<MouseEnterEvent>(evt => window.mousedOver = this);
RegisterCallback<MouseLeaveEvent>(evt => window.mousedOver = null);

// in OnEnable() on the window itself:
if (rootVisualElement.parent == null) // this is null in OnEnable... sometimes
    EditorApplication.delayCall += SubscribeKeyDownEvent;
else
    SubscribeKeyDownEvent();

...

private void SubscribeKeyDownEvent() {
    var keyDownReceiver = rootVisualElement;
    while (keyDownReceiver.parent != null) {
        keyDownReceiver = keyDownReceiver.parent;
    }

    keyDownReceiver.RegisterCallback<KeyDownEvent>(KeyDown);
}

...

private void KeyDown(KeyDownEvent evt) {
    if (evt.keyCode == KeyCode.B && mousedOver != null) {
        HandleBPressedOver(mousedOver);
    }
}

This is just painful. It will also probably break if stuff starts having focus? idk. There’s gotta be a better way to handle this! In imgui I’d check if Event.Current was a keydown event with the b keyCode, and then I’d painstakingly figure out the mouse position vs. the element positions. The “what is the mouse over” thing has become a lot easier, but getting a notif when the key is down only being available for focused things seems like a very silly restriction.

Well, if you think this is painful here is something to feel better about yourself - how webdevs implement it in JS, haha:

Seriously, though, a simple check if mouse if over the element when key is pressed isnt that bad. And you can add condition if any input fields are focused… Okay, I see what you mean, this is becoming ugly.

Hi Baste,

You can use rootVisualElement.panel.visualTree.RegisterCallback<KeyDownEvent>(KeyDown, TrickleDown.TrickleDown); to subscribe to the event and make sure KeyDown gets called first no matter what child is or is not focused.

As for rootVisualElement.parent being null in OnEnable, if that is the case then you can do your SubscribeKeyDownEvent as a reaction to the AttachedToPanelEvent, instead of using a delayCall, which is much more consistent and also won’t prevent your project from building if your script is not in an “Editor” folder.

1 Like

This is an EditorWindow, so I’m not too worried about trying to make it compile in builds :stuck_out_tongue:

AttachToPanelEvent seems to be called every time I dock the window in a new spot. originPanel is always null and destinationPanel is always a new object. Does the previous panel always get destroyed?

It seems like the keyDownEvents doesn’t get registered if I dock the EditorWindow. If I have the window as a free floating element, they do work.

Also, the KeyDownEvent is always invoked twice - both times have the same phase (at target), but the second one is missing the keycode.

Here’s a test implementation. My real window uses stylesheets, but here I’ve just done everything inline:

using UnityEditor;
using UnityEngine;
using UnityEngine.UIElements;

public class MyEditorWindow : EditorWindow {

    [MenuItem("Test/Window")]
    public static void TestWindow() {
        GetWindow<MyEditorWindow>();
    }

    private void OnEnable() {
        if (rootVisualElement.panel == null) {
            rootVisualElement.RegisterCallback<AttachToPanelEvent>(SubscribeKeyDownEvent);
        }
        else
            SubscribeKeyDownEvent(null, rootVisualElement.panel);

        var root = new VisualElement();
        rootVisualElement.Add(root);

        for (int i = 0; i < 10; i++) {
            var row = new VisualElement();
            row.style.flexDirection = FlexDirection.Row;
            root.Add(row);

            for (int j = 0; j < 10; j++) {
                var cell = new VisualElement();
                cell.style.minHeight = 16;
                cell.style.minWidth = 16;
                cell.style.maxHeight = 16;
                cell.style.maxWidth = 16;

                cell.style.backgroundColor = Color.HSVToRGB(Random.value, 1f, 1f);
                row.Add(cell);
            }
        }
    }

    private void SubscribeKeyDownEvent(AttachToPanelEvent evt) {
        SubscribeKeyDownEvent(evt.originPanel, evt.destinationPanel);
    }

    private void SubscribeKeyDownEvent(IPanel oldPanel, IPanel newPanel) {
        oldPanel?.visualTree.UnregisterCallback<KeyDownEvent>(KeyDown, TrickleDown.TrickleDown);
        newPanel .visualTree.RegisterCallback  <KeyDownEvent>(KeyDown, TrickleDown.TrickleDown);
    }

    private void KeyDown(KeyDownEvent evt) {
        Debug.Log($"{evt.propagationPhase}/{evt.keyCode}/{evt.character}/{evt.target}/{(evt.imguiEvent != null ? evt.imguiEvent.type.ToString() : "NULL")}");

    }
}

So bug 1: Hit Test->Window, click a button. Note that you get two Debug.Log’s. The first one is WILD:

Why is it printing less? Turns out that evt.character is ‘\0’! Also turns out that that terminates the string! Is that a bug in Debug.Log? We’re logging C# strings, those are not null-terminated!

The actual bug is that there’s two invocations of the method, one with the data packed in one way and another one with it packed in another. That’s strange and unexpected.

Bug 2: Dock the window next to the inspector or any other window, give it focus and try to hit any keys. No respone. Undock it again, it gets a response.

Bug 1 is not a bug, it’s the way KeyDownEvents work. They are just a copy of GUI Events, which work exactly like that, if you try

void Update()
{
while(Event.PopEvent(e))
Debug.Log($"{e.typ}/{e.keyCode}/{e.character}");
}```
, you should see similar results. You can ignore the KeyDownEvent if keyCode==None, if you're not interested in the text content of the keyboard event, or ignore the one with character=='\0' if you're looking for text input.

Bug 2 seems more of a problem. Could you print `rootVisualElement.panel.focusController.focusedElement` to see if there's something that has the focus and that does something strange with it? For EditorWindows, there are a few IMGUIContainers that handle window-related events on the top of the visual tree hierarchy (you can see them in the UI Toolkit Debugger, usually there's 1), which can be receiving events before the rest and use them. If all keyboard events are systematically used by that element, then that's probably a real problem, and we might need to open a bug about it, indeed.

+1ing on this, I had to switch to using IMGUI events because of this exact problem, keystrokes were sometimes missing/probably consumed by something before UIE triggered them. This seemed especially true in cases where I wanted to react to say “cntrl+s” before the editor, and UIE seemed to never receive the input in time, but IMGUI can consume these events just fine and “snatch” them from the editor.

After I dock the element, focusedElement is null. Should I send a bug report?

Small correction to what I said earlier: you shouldn’t use panel.visualTree.RegisterCallback<KeyDownEvent>, but rather use rootVisualElement.RegisterCallback<KeyDownEvent>, because the panel.visualTree is shared by all windows docked in the same DockArea, that is, when you have multiple tabs grouped together in the editor, you don’t want some inactive tabs to be listening to events targeted at the other active tabs.

You can use rootVisualElement.focusable = true; rootVisualElement.pickingMode = PickingMode.Position; rootVisualElement.Focus();
to make sure that the window’s background is clickable and that doing that focuses the rootVisualElement.

You can also react to EditorWindow.OnFocus() to make sure rootVisualElement or one of its children reacquires the focus when your window tab is selected. Something like this:

using UnityEditor;
using UnityEngine;
using UnityEngine.UIElements;

public class TestKeyDownWindow : EditorWindow
{
    // Called immediately after OnEnable, once UIDocument's panel has been assigned
    public void CreateGUI()
    {
        rootVisualElement.Add(new Label("Some Label"));
        rootVisualElement.Add(new TextField());

        rootVisualElement.RegisterCallback<KeyDownEvent>(OnKeyDown, TrickleDown.TrickleDown);

        // Make sure rootVisualElement has the focus when we start.
        rootVisualElement.focusable = true;
        rootVisualElement.pickingMode = PickingMode.Position;
        rootVisualElement.Focus();
    }

    private void OnKeyDown(KeyDownEvent evt)
    {
        Debug.Log("UITK event: type=" + evt.GetType() + ", keyCode=" + evt.keyCode + ", character=" +
                  (evt.character == 0 ? "0" : "" + evt.character) + ", modifiers=" + evt.modifiers);
    }

    // Called when EditorWindow gets keyboard focus, for example when changing tab. Make sure we reacquire focus.
    public void OnFocus()
    {
        ReacquireFocusOnWindowRootOrChildren();
    }

    private void ReacquireFocusOnWindowRootOrChildren()
    {
        var focusedElement = rootVisualElement.focusController.focusedElement as VisualElement;
        if (focusedElement == null || !IsChildOfWindowRoot(focusedElement))
            rootVisualElement.Focus();
    }

    private bool IsChildOfWindowRoot(VisualElement ve)
    {
        return ve == rootVisualElement || ve != null && IsChildOfWindowRoot(ve.parent);
    }

    [MenuItem("Window/TestKeyDownWindow")]
    static void Open()
    {
        GetWindow<TestKeyDownWindow>().Show();
    }
}

That being said, there is still a situation where the DockArea of the EditorWindow will receive all keyboard events and stop them from going further down, that is, when you explicitly click on the window tab and nothing else. This is by design, as it’s the only way to tell the editor that you want to interact with the docking system itself and not the window.

That works!

Though, in general, it is a bit clunky! Especially since I had to go on the forums to understand what was going on.

So a follow-up question: this came up because I wanted my UI Toolkit based editor window have keyboard shortcuts. What’s the intended way to do that?

In this case, imagine you’re making an MS Paint clone in an editor window using UI Toolkit, and you want pressing “b” to select the bucket fill tool, when the window is focused.

Is the intended workflow for that to require setting the focusable and pickingMode fields to the correct values and calling Focus()? None of those are in any way intuitive prerequisites to have a callback do something.