UIElements on Android doesn't properly capture pointers

[perholmes]

Hi,

UPDATE: Please see further down. The issue is different than initially believed, and is about multiple pointer IDs being received for every tap, an apparent bug in UI Elements on Android.

ORIGINAL: I’m having some very different behavior of captured pointer events on Android.

On desktop and in the device simulator, capture is capture, nothing propagates, nothing leaks, the object owns all pointer events. But on Android, even when I capture pointer events in a child object, the parent object keeps receiving events.

I seems like a bug, because the behavior is markedly different, and breaks many UI assumptions. I’d be able to work around it, but I’d love for the UI Elements team to confirm whether or not this is a bug. And if it’s not a bug, then I’d love to hear the reasoning behind the difference in whether or not capturing stops bubble-up propagation.

Thanks,

Per

[perholmes]

I’m attaching a proof.

I’ve done the most thorough testing on the PointerMoveEvents of a child item and its parent. Both can capture the pointer. Here is the child item called MenuClickBase. Observe that it captures the mouse on pointer down, and logs mouse move events.

And here is the parent item. I have a mechanism where the child releases pointer capture, and the parent captures the mouse events instead.

On desktop, when I transfer from child to parent, the logging looks like this, with a clean transfer of capture:

[20:11:49.334] [INFO] Click base pointer move
[20:11:49.351] [INFO] Click base pointer move
[20:11:49.368] [INFO] Click base pointer move
[20:11:50.252] [INFO] Click base pointer down: 0
[20:11:50.634] [INFO] Click base pointer move
[20:11:50.652] [INFO] Click base pointer move
[20:11:50.668] [INFO] Content pointer move
[20:11:50.685] [INFO] Content pointer move
[20:11:50.701] [INFO] Content pointer move
[20:11:50.718] [INFO] Content pointer move
[20:11:50.734] [INFO] Content pointer move

But on Android, the capture isn’t working. Both on the original click, and the subsequent move, both child and parent receive pointer down and move events, regardless of capture:

09-03 20:07:33.517 18143 18168 I Unity   : [20:07:33.517] [INFO] Click base pointer down: 0
09-03 20:07:33.557 18143 18168 I Unity   : [20:07:33.557] [INFO] Content pointer down: 1
09-03 20:07:33.581 18143 18168 I Unity   : [20:07:33.581] [INFO] Click base pointer move
09-03 20:07:33.585 18143 18168 I Unity   : [20:07:33.584] [INFO] Click base pointer move
09-03 20:07:33.586 18143 18168 I Unity   : [20:07:33.586] [INFO] Content pointer move
09-03 20:07:33.609 18143 18168 I Unity   : [20:07:33.609] [INFO] Click base pointer move
09-03 20:07:33.844 18143 18168 I Unity   : [20:07:33.844] [INFO] Click base pointer move
09-03 20:07:33.847 18143 18168 I Unity   : [20:07:33.847] [INFO] Content pointer move
09-03 20:07:33.868 18143 18168 I Unity   : [20:07:33.868] [INFO] Click base pointer move
09-03 20:07:33.868 18143 18168 I Unity   : [20:07:33.868] [INFO] Content pointer move
09-03 20:07:33.884 18143 18168 I Unity   : [20:07:33.884] [INFO] Click base pointer move
09-03 20:07:33.885 18143 18168 I Unity   : [20:07:33.885] [INFO] Content pointer move
09-03 20:07:33.901 18143 18168 I Unity   : [20:07:33.901] [INFO] Click base pointer move
09-03 20:07:33.901 18143 18168 I Unity   : [20:07:33.901] [INFO] Content pointer move
09-03 20:07:33.918 18143 18168 I Unity   : [20:07:33.917] [INFO] Click base pointer move
09-03 20:07:33.918 18143 18168 I Unity   : [20:07:33.918] [INFO] Content pointer move
09-03 20:07:33.934 18143 18168 I Unity   : [20:07:33.934] [INFO] Click base pointer move
09-03 20:07:33.934 18143 18168 I Unity   : [20:07:33.934] [INFO] Content pointer move
09-03 20:07:33.952 18143 18168 I Unity   : [20:07:33.952] [INFO] Click base pointer move

This basically breaks UI Elements on Android.

[perholmes]

Also, I’ve noticed that I’m getting double touch events on Android. This is a Pixel 2 XL. This could somehow interact poorly with the capture mechanism. My code above filters it out because I only react to the first pointer id. But maybe the capture mechanism gets confused.

[perholmes]

The real issue seems to be that two pointer events are received for every tap. This may be hidden in other code, but since I work with captures, it means that I only capture the first tap, and the second tap propagates to the parent.

This still seems to be a bug, just a different one. Here I’m logging pointerId and position from the event. Observe that:

• Two events are received.
• They have unique pointerIds, so Unity is generating them on purpose.
• They have the same position.
• My capture only captures the first one, and the second bubbles to the parent, creating the appearance that capture doesn’t work.

09-03 22:56:51.237 15474 15496 I Unity   : [22:56:51.236] [INFO] Click base pointer down: 0, position (527.94, 239.88, 0.00)
09-03 22:56:51.274 15474 15496 I Unity   : [22:56:51.274] [INFO] Click base pointer down: 1, position (527.94, 239.88, 0.00)
09-03 22:56:51.276 15474 15496 I Unity   : [22:56:51.276] [INFO] Content pointer down: 1

On desktop and in the device simulator, I only receive one pointer for every click. I’ve verified that this is not somehow an issue with callbacks being instantiated twice. But this is fine in desktop. And multiple callbacks would still only get one pointerId.

So my best bet is that UIElements sends double pointer events on Android. This will break captures and any pinch-zoom or multi-finger gesture, because it becomes impossible to reason about the pointers.

[perholmes]

Being dead in the water, I’ve had to roll a temporary fix to de-duplicate pointer events on Android. This hack only works because all my controls are custom for this app. Maybe it’ll help someone. But this issue needs urgent attention, because all assumptions about event capture and propagation go out the window.

The following PointerFix static class registers pointers and detects if they happen in the same screen position. It then lets the caller know whether to process the event or not.

using UnityEngine;
using System.Collections.Generic;
using UnityEngine.UIElements;

// This is an EXTREMELY TEMPORARY filter to work around UIElements sending multiple identical
// pointers on Android with different pointer IDs, causing capture and propagation problems. This
// fix is incompatible with hovering, since we only process move events when a pointer is down.

public class PointerFix
{
    private static Dictionary<int, Vector3> pointers = new Dictionary<int, Vector3>();

    // Call on Down. Returns false if the event should not handled.
    public static bool Down(PointerDownEvent evt)
    {
        if (pointers.ContainsKey(evt.pointerId)) {
            Debug.Log($"Pointer {evt.pointerId} is already registered");
            evt.StopImmediatePropagation();
            return false;
        }

        foreach (var pointer in pointers) {
            if (Vector3.Distance(evt.position, pointer.Value) < 1f) {
                Debug.Log($"Pointer {evt.pointerId} rejected at {evt.position} because it's a duplicate");
                evt.StopImmediatePropagation();
                return false;
            }
        }

        Debug.Log($"Pointer {evt.pointerId} pressed at {evt.position}");
        pointers[evt.pointerId] = evt.position;
        return true;
    }

    // Call on Move. Returns false if the event should not handled.
    public static bool Move(PointerMoveEvent evt)
    {
        var active = pointers.ContainsKey(evt.pointerId);
        if (!active) {
            evt.StopImmediatePropagation();
        }
        return active;
    }

    // Call on Up. Returns false if the event should not handled.
    public static bool Up(PointerUpEvent evt)
    {
        var active = pointers.ContainsKey(evt.pointerId);
        Debug.Log($"Pointer {evt.pointerId} released");
        pointers.Remove(evt.pointerId);
        if (!active) {
            evt.StopImmediatePropagation();
        }
        return active;
    }

    // For my personal needs, since I do delicate hand-offs of pointer capture between children and
    // parents.
    public static void ForceUp(int pointerId)
    {
        var active = pointers.ContainsKey(pointerId);
        Debug.Log($"Pointer {pointerId} forcibly released");
        pointers.Remove(pointerId);
    }
}

Every PointerDown, Move and Up event should then be prefixed with a call to this. This is why it only works on custom controls:

It’s ugly, but I get to live another day.

[perholmes]

And to document the problem again, here is the output of PointerFix on desktop:

Pointer 0 pressed at (754.07, 404.71, 0.00)
Pointer 0 released
Pointer 0 pressed at (755.07, 139.71, 0.00)
Pointer 0 released
Pointer 0 pressed at (769.07, 397.71, 0.00)
Pointer 0 released
Pointer 0 pressed at (766.07, 156.71, 0.00)
Pointer 0 released

But when running on Android, every pointer is a duplicate, which is detected by PointerFix and discarded:

09:16:02 I/Unity    : Pointer 0 pressed at (530.73, 245.96, 0.00)
09:16:02 I/Unity    : Pointer 1 rejected at (530.73, 245.96, 0.00) because it's a duplicate
09:16:02 I/Unity    : Pointer 0 released
09:16:02 I/Unity    : Pointer 1 released
09:16:04 I/Unity    : Pointer 0 pressed at (551.01, 75.56, 0.00)
09:16:04 I/Unity    : Pointer 1 rejected at (551.01, 75.56, 0.00) because it's a duplicate
09:16:04 I/Unity    : Pointer 0 released
09:16:04 I/Unity    : Pointer 1 released
09:16:05 I/Unity    : Pointer 0 pressed at (509.68, 236.08, 0.00)
09:16:05 I/Unity    : Pointer 1 rejected at (509.68, 236.08, 0.00) because it's a duplicate
09:16:05 I/Unity    : Pointer 0 released
09:16:05 I/Unity    : Pointer 1 released
09:16:06 I/Unity    : Pointer 0 pressed at (558.36, 89.26, 0.00)
09:16:06 I/Unity    : Pointer 1 rejected at (558.36, 89.26, 0.00) because it's a duplicate
09:16:06 I/Unity    : Pointer 0 released
09:16:06 I/Unity    : Pointer 1 released

[AlexandreT-unity]

Hi @perholmes , could you submit a small repro project through the bug reporter so we can investigate? Go to Help > Report a bug… Thanks!

[perholmes]

I’ve created a reproduction project where it still happens, and submitted it. I was not given an ID. Here is the text of the report, to help you locate it:

UIElements sends double pointer events on Android, causing all assumptions about event propagation and pointer capture to break down. When running the reproduction project on desktop, pointer events are received in the expected quantity (log output):

PointerDownEvent: 0, Position: (300.64, 283.50, 0.00)
PointerUpEvent: 0, Position: (300.64, 283.50, 0.00)

However, when running on Android, two pointers are received for every pointer down and up.

07:34:33 I/Unity : PointerDownEvent: 0, Position: (297.60, 266.10, 0.00)
07:34:33 I/Unity : PointerDownEvent: 1, Position: (297.60, 266.10, 0.00)
07:34:33 I/Unity : PointerUpEvent: 0, Position: (297.60, 266.10, 0.00)
07:34:33 I/Unity : PointerUpEvent: 1, Position: (297.60, 266.10, 0.00)

This results in both CapturePointer and StopImmediatePropagation stopping working in cases where a child object is tracking the pointer down/up cycle, causing pointers that are believed to be captured to actually go to the parent, breaking UIs completely.

[perholmes]

Case 1364340