Running UI Toolkit with unity in batch mode for visual test in the CI

Unity Engine
,
1

Hi there! We are using UI Toolkit for our development and we wanted to add some visual test to ensure that nothing is changed by accident.

Basically, we have a baseline image that we compare pixel by pixel with the current result of the image that we render in the test

We have a CI with CircleCI that will run the test with unity in batchmode.

We are looking at how to implement the visual testing but we are getting some problem. It works well in normal mode but when we try with batchmode

We are tested the following approaches:

  • Try to render the UI Document directly as normal ( after the cameras). The problem is that Screencapture is not supported well in batchmode and the result of the image is not including the UI Document. However the normal canvas with an image (iGUID) is renderer correctly
  • Since IGUID is rendered correctly we tried to render the UI Document in a renderer texture that then will be renderer in a canvas with a RawImage. But it seems that the renderer image is not renderering the UI Document either
  • We added a camera and changed the canvas to be ScreenSpace camera to use the camera to take snapshots. The problem persists, no UI Document is rendered
  • We changed the canvas to be WorldSpace and settled them in positions to be visible. The problem persists

Is any way to accomplish this in batchmode?

2

Can you confirm that nothing is rendered when using batchmode even a local development machine, not just CircleCI ?
Can you also share the Editor.log file as well as any relevant information about the machine (GPU, OS)

Thanks

3

Thanks for the reply! Yes, I am opening the project locally with batch mode from the console. I am not in the machine, I will give you the log tomorrow

I have tried Unity 2021 and Unity 2019 without success

I have a GTX 1080 TI with Windows 10

4

We believe this should work as we have similar tests internally who do something similar.

Although we are using utilities from the Graphics Tests Framework (experimental) for Image Comparison.

Can you share the command line as well ?

It feels like a bug report would be the best way to proceed, to understand your setup a little bit more.

5

Sure!

This is the command line that we use to run them

xvfb-run --auto-servernum --server-args=‘-screen 0 640x480x24’ $UNITY_PATH/Editor/Unity
-batchmode
-projectPath “$PROJECT_PATH”
-logFile “$PROJECT_PATH/playmode-logs.txt”
-runTests
-testPlatform PlayMode
-testResults “$PROJECT_PATH/playmode-results.xml”
-enableCodeCoverage
-coverageResultsPath “$PROJECT_PATH/CodeCoverage”
-coverageOptions

Since it may be related to a bug we decide to change the approach and delay the visual test for the future. Thanks for your help!