Automated testing webgl mouse events not working

Questions & Answers
1

I’m working on end-to-end automated tests with a webgl build using the Cypress framework.

The purpose is to test from the perspective of an end user (BDD testing), so the tests need to simulate interaction at the browser level.

It works great for a number of things, but it is failing to simulate mouse events properly.

The mouse events are sent to the canvas, but they do not register with the unity client.

One of the major differences between my simulated clicks and a real click is the event property ‘isTrusted’, which is true for a real event, but is designed specifically to be false for simulated events. Is this the source of my headache?

simulated click:

MouseEvent {isTrusted: false, stopPropagation: ƒ, screenX: 0, screenY: 0, clientX: 70, …}
altKey: false
bubbles: true
button: 0
buttons: 0
cancelBubble: false
cancelable: true
clientX: 70
clientY: 20
composed: false
ctrlKey: false
currentTarget: null
defaultPrevented: false
detail: 1
eventPhase: 0
fromElement: null
isTrusted: false
layerX: 20
layerY: 20
metaKey: false
movementX: 0
movementY: 0
offsetX: 20
offsetY: 20
pageX: 70
pageY: 20
path: (7) [canvas##canvas, div#gameContainer, div.webgl-content, body, html, document, Window]
relatedTarget: null
returnValue: true
screenX: 0
screenY: 0
shiftKey: false
sourceCapabilities: null
srcElement: canvas##canvas
stopPropagation: ƒ ()
target: canvas##canvas
timeStamp: 2080510.8000000035
toElement: canvas##canvas
type: "click"
view: Window {postMessage: ƒ, blur: ƒ, focus: ƒ, close: ƒ, parent: Window, …}
which: 1
x: 70
y: 20
__proto__: MouseEvent

real click:

MouseEvent {isTrusted: true, screenX: 3355, screenY: 162, clientX: 75, clientY: 31, …}
altKey: false
bubbles: true
button: 0
buttons: 0
cancelBubble: false
cancelable: true
clientX: 75
clientY: 31
composed: true
ctrlKey: false
currentTarget: null
defaultPrevented: false
detail: 1
eventPhase: 0
fromElement: null
isTrusted: true
layerX: 25
layerY: 31
metaKey: false
movementX: 0
movementY: 0
offsetX: 26
offsetY: 32
pageX: 75
pageY: 31
path: (7) [canvas##canvas, div#gameContainer, div.webgl-content, body, html, document, Window]
relatedTarget: null
returnValue: true
screenX: 3355
screenY: 162
shiftKey: false
sourceCapabilities: InputDeviceCapabilities {firesTouchEvents: false}
srcElement: canvas##canvas
target: canvas##canvas
timeStamp: 25545.880000005127
toElement: canvas##canvas
type: "click"
view: Window {postMessage: ƒ, blur: ƒ, focus: ƒ, close: ƒ, parent: Window, …}
which: 1
x: 75
y: 31
__proto__: MouseEvent
2

Because Cypress doesn’t actually move the mouse, the simulated clicks do not happen at the position you require. I managed to find out a workaround for your specific case, which is to do a slight modification to StandaloneInputModule.cs. I’ve tested this only in WebGL and might not work or might require modifications for different platforms.

You can create a copy of StandaloneInputModule and modify the ProcessMousePress function to do an explicit raycast with the new click coordinates, instead of relying on the ones from the mouse position.

if (Event.current != null)
{
	// Y axis is inverted in the Event coordinates
	pointerEvent.position = new Vector2(Event.current.mousePosition.x, Screen.height - Event.current.mousePosition.y);

	// Raycast the received click position
	eventSystem.RaycastAll(pointerEvent, m_RaycastResultCache);
	var raycast = FindFirstRaycast(m_RaycastResultCache);

	// Save the raycast result and replace the "clicked" gameObject with the new result
	pointerEvent.pointerCurrentRaycast = raycast;
	currentOverGo = pointerEvent.pointerCurrentRaycast.gameObject;
}

You can then remove the StandaloneInputModule script from the EventSystem object in your scene and replace it with you new CustomStandaloneInputModule. An example of how this might be done is attached [136990-customstandaloneinputmodule.zip|136990]

The way the UI input system works is to detect GameObjects underneath the mouse pointer and whenever a click event happens, it propagates the event to the corresponding GameObjects.

You can browse through how all this works in the public repository.