PlayMode Tests marked with [ValueSource] attribute fail to appear in Unity Cloud Build "Tests" tab.

Unity Services
,
1

Description of issue:
PlayMode Tests marked with [ValueSource] attribute fail to appear in Unity Cloud Build “Tests” tab. Upon downloading the “Unit Tests (Play Mode)” file it appears they are in fact run, they just don’t display under the “PlayMode Tests” section.

Specifically in my case the tests that are not being displayed have the following signatures.

[UnityTest]
public IEnumerator LevelDeterminismFreshLoadTest([ValueSource(nameof(LevelsToTest))] LevelData levelData,
        [ValueSource(nameof(DefaultFrameTestBounds))] Vector2Int frameTestBounds,
        [ValueSource(nameof(DefaultTimeToTests))] int timeToTest,
        [ValueSource(nameof(PostLoadLevelWaitTimes))] float postLoadLevelWaitTimes)

[UnityTest]
public IEnumerator LevelDeterminismRestartTest([ValueSource(nameof(LevelsToTest))] LevelData levelData,
    [ValueSource(nameof(DefaultFrameTestBounds))] Vector2Int frameTestBounds,
    [ValueSource(nameof(DefaultTimeToTests))] int timeToTest,
    [ValueSource(nameof(PostLoadLevelWaitTimes))] float postLoadLevelWaitTimes)


[UnityTest]
public IEnumerator LevelConfigurationTest([ValueSource(nameof(LevelsToTest))] LevelData levelData)

Unity version:
2020.3.12f1

XCode version:
Xcode 13.3.1

Source control:
git

Local build platform:
Windows? However the issue is specific to Unity Cloud Build dashboard.

Build target:
Android & iOS

Relevant plugins being used (i.e. Google Play Game Services, other plugins that may perform automatic post-build behaviors)
None

Project name:
Command Center Earth

Build log (relevant chunk):
Image below and a test-case from the play mode test file provided. I have excluded including test case file specifically out of caution for any sensitive data that could be contained in it.

<test-case id="1071" name="LevelDeterminismFreshLoadTest(Simon (LevelData),(5, 60),4,0.0f)" fullname="LevelDeterminismPlayModeTests.LevelDeterminismFreshLoadTest(Simon (LevelData),(5, 60),4,0.0f)" methodname="LevelDeterminismFreshLoadTest" classname="LevelDeterminismPlayModeTests" runstate="Runnable" seed="1714533172" result="Passed" start-time="2023-03-15 00:49:19Z" end-time="2023-03-15 00:49:42Z" duration="23.296535" asserts="0">
            <properties />
            <output><![CDATA[<b>(Variable Log)</b>: Dynamic Parameter Variable '_levelStackCc' Encountered...
<b>(Variable Log)</b>: Parameter '_levelStackCc' (of type 'StackCanvasController') promoted to a Variable in Blackboard 'Graph'.
<b>(Variable Log)</b>: Dynamic Parameter Variable '_isPostLevelCCIntroTransitionAnimated' Encountered...
<b>(Variable Log)</b>: Parameter '_isPostLevelCCIntroTransitionAnimated' (of type 'Boolean') promoted to a Variable in Blackboard 'Graph'.
<b>(Variable Log)</b>: Dynamic Parameter Variable '_isPlayerWinner' Encountered...
<b>(Variable Log)</b>: Parameter '_isPlayerWinner' (of type 'Boolean') promoted to a Variable in Blackboard 'Graph'.
<b>(Variable Log)</b>: Dynamic Parameter Variable '_levelStackCc' Encountered...
<b>(Variable Log)</b>: Parameter '_levelStackCc' (of type 'StackCanvasController') promoted to a Variable in Blackboard 'Graph'.
<b>(Variable Log)</b>: Dynamic Parameter Variable '_isPostLevelCCIntroTransitionAnimated' Encountered...
<b>(Variable Log)</b>: Parameter '_isPostLevelCCIntroTransitionAnimated' (of type 'Boolean') promoted to a Variable in Blackboard 'Graph'.
<b>(Variable Log)</b>: Dynamic Parameter Variable '_isPlayerWinner' Encountered...
<b>(Variable Log)</b>: Parameter '_isPlayerWinner' (of type 'Boolean') promoted to a Variable in Blackboard 'Graph'.
]]></output>
          </test-case>

Does it build locally from a fresh checkout? (Instructions below):
Yes

8879091--1212906--Unity Cloud Build Backend.png
8879091--1212906--Unity Cloud Build Backend.png1920×890 144 KB

2

I took a look at your project and was able to see the issue. I’ve assigned a bug to one our web developers to resolve the issue with rendering the test results.

1 Like
3

Thanks a ton @Benjamin-Gooding ! Looking forward to this getting fixed.

4

Hey guys, I’m still seeing this is an issue on Cloud Build and it’s kind of a big pain for us. Just curious if it’s gained any traction.

Thanks in advance!

5

Hi @Meatloaf4 ,

Thanks for reaching out about this issue. We really appreciate your patience and for taking the time to bring it to our attention. We know it’s been a pain for you, and we’re sorry about that.

The good news is that we’ve looked into the issue and fixed it. You should be able to see all your tests on the “Tests” tab on the Build History Details page now.

Once again, thanks for letting us know about this. Your feedback is super important to us, and it helps us make our products better. If you have any other concerns, don’t hesitate to give us a shout.

Thanks,
Melinda

1 Like
6

Ah this is great news! Thank you so much for fixing that issue.

I also went ahead and checked that the fix works retroactively, and it does! Thanks so much!