Are ValueSource tests supposed to run in PlayMode test runner?

Unity Engine
1

This test runs fine in edit mode:

    public class LevelValidationTests
    {
        [UnityTest]
        public IEnumerator LevelIsValid([ValueSource("LevelTestCases")] string levelName)
        {
             yield return null;

            Assert.IsTrue(true);
        } 

        public static IEnumerable<string> LevelTestCases
        {
            get
            {
                var themeManager = Resources.Load(ObjectNames.ThemeManager) as ThemeManager;
                Debug.Assert(themeManager != null, "themeManager != null");
                var levels = themeManager.Themes.SelectMany(t => t.Levels).Select(l => l.SceneName);

                return levels;
            }
        }
    }

In play mode at first I see all test cases, but when I try to run them I am getting

According to documentation I would say that ValueSource should be supported:

Or not?

2

Well it looks like the tests are actually done, it is just buggy how the results are displayed.

Before the tests are run (when I switch back and forth to EditMode tab):

3300833--256031--before.png
3300833--256031--before.png696×524 14.3 KB

After the tests are run:

3300833--256030--after.png
3300833--256030--after.png690×521 12.9 KB

EDIT: Maybe it is something with ScriptableObject loading from resource during play mode test, because if I hardcode level names everything is ok.

3

I solved it by moving resource loading to class variable like this:

        static readonly IEnumerable<string> LevelNames = Resources.Load<ThemeManager>(ObjectNames.ThemeManager).Themes.SelectMany(t => t.Levels).Select(l => l.SceneName).ToList();
       
        public static IEnumerable<string> LevelTestCases
        {
            get
            {
                return LevelNames;
            }
        }
4

Well, for the record, my problems had nothing to do with ValueSource attribute. Levels in my ThemeManager (ScriptableObject) was initialized inside Awake method. And it sometimes was called, sometimes wasn’t. I moved initialization to property getter and now everything works fine.

For the next time I should remember that Awake != constructor.