I use Jenkins (formerly Hudson) CI Server with that line as a build step (windows batch):
%unity_dir%\Editor\Unity.exe -batchmode -nographics -quit -createProject "%WORKSPACE%\unity_test" -projectPath "%WORKSPACE%\unity_test" -assetServerUpdate 192.168.0.2 unit_testing username password -executeMethod CIScript.start
%unity_dir% is an environment variable I set up, %workspace% is from Jenkins.
The “CISript” is the entry point for all my automated building and testing. (EDIT: to be precise, the static method “start” of the class “CIScript” is the exact entry point)
SharpUnit features a XML-reporter, I beefed that up a little to resemble JUnit enough to be directly usable from Jenkins - release pending.
I have some Tests as editor scripts, they set up a new scene and add GameObjects, to add MonoBehaviour-derived classes as components, respectively. What is currently not possible is to run a simulation (i.e. EditorApplication.isPlaying = true) while in batchmode, because you can’t enter play mode, run a few or many frames and leave it graciously. The leaving part is the problem (ref: Concurrency and editor scripts and the play mode
and Programmatically playing a scene for a single frame)
For automated smoke/application tests you will have to do the following things:
- run the batch mode as prescribed above but don’t use -quit
- in the static method do all of the following:
- setup a scene, use throwaway ones with EditorApplication.NewScene() or load some prepared ones.
- set EditorApplication.isPlaying to true
- add one essential script to the scene, see below, that will enable you to end the simulation
- add a static method to EditorApplication.playmodeStateChanged delegate, the respective method will be run twice, so make sure that the game actually ran before you do anything within that method.
Oh yeah, that’s the part where you can do assertions enclosed in try blocks. You should do assertions through the OneFrameSignaller script before the play mode is ended. You can also use the code on this delegate to set up another scene and repeat the test cycle on the next scene.
Be aware the the static method (i.e. entry point of the batch mode execution) must terminate before the play mode can actually start, due to all unity api stuff running on the same system thread aparently. Thus don’t use the -quit option for the batch mode.
The key script below is an editor script. You can’t manually add that to a GameObject in the actual unity editor by drag&drop. Programmatically this works never the less, don’t stop and wonder why, embrace the fact - it makes things so much easier…
using UnityEngine;
using UnityEditor;
using System;
using System.Collections;
using System.Threading;
public class OneFrameSignaller : MonoBehaviour
{
public static bool atLeastOneFrameRan = false;
public static DateTime startTime;
public static TimeSpan testDuration;
void Update()
{
atLeastOneFrameRan = true;
StartCoroutine(signalFrameEnd()); // use this to quit after this frame, you could use a counter instead
if (DateTime.Now - OneFrameSignaller.startTime > OneFrameSignaller.testDuration) // use this if you want a timed end of the test
{
StartCoroutine(signalFrameEnd());
}
// Thread.Sleep(20); //this is somewhat evil, us this to extend the execution time
// of the frames if you feel they run to fast for useful assertions
}
void Start() {
OneFrameSignaller.startTime = DateTime.Now;
OneFrameSignaller.testDuration = TimeSpan.FromSeconds(10);
Application.runInBackground = true;
}
IEnumerator signalFrameEnd()
{
yield return new WaitForEndOfFrame();
EditorApplication.isPlaying = false;
}
}
So what happens is this:
The static method exits, through Unity’s internal workings the play mode starts as EditorApplication.isPlaying was set to true. The above script is executed like a normal MonoBehaviour script. If the exit-condition is met, at the end of the frame EditorApplication.isPlaying is set to false. Any assertions about the state of the existing scene should be made before isPlaying is set to false. For assertions I use Assert from SharpUnit. Be sure to enclose this in try-blocks because test setup is very costly here. So defy the principle of using one test-setup/teardown per test, just make sure that assertions don’t interfere with each other.
Setting isPlaying to false triggers the execution of the delegate EditorApplication.playmodeStateChanged. From there you can either quit now with EditorApplication.Exit(0) or another non-zero number to indicate test failures to the continuous integration server during build - or you go ahead and setup a new scene and repeat the cycle. Just be sure to call Exit on some point or your build will never finish.
If all Tests pass I build a new version of the game using the BuildPipeline.