Unity Test Tools

Unity Engine
,
1

Hi all,

We just released Unity Test Tools - a set of tools for testing Unity content. It includes unit tests runner, integration tests runner and assertion component. Try it out yourself!

Read more about it on the blog post: http://blogs.unity3d.com/2013/12/18/unity-test-tools-released/
Get the tools from the Asset Store: Unity Asset Store - The Best Assets for Game Making

Happy testing!
Tomek and the Toolsmiths Team @ Unity

4 Likes
2

Very interesting!

Thanks a lot

3

Very exciting news!

4

Thanks Tomek, this is very useful. It looks like the example unit tests only deal with manipulating basic primitives. Are there any example unit tests that create and check game objects (and their properties) in a scene? By this I mean unit tests written in C# and run through NUnit, not integration tests.

5

The unit test framework allows you to instantiate a GameObject from the code. Simply call var go = new GameObject(). From there you can manipulate it and test it. Unfortunately, no frame-dependant operations will work since all tests are executed in the same frame.

6

Hi,

First of all, thanks for sharing this amazing tool! I am really excited about this after seeing the screenshots. Are there any step-by-step instructions on how to run the tests in the example scene? I opened the AngryBots TestScene.unity, but I am not sure how I can bring up the TestRunner. Even more importantly, how would I add this to an existing application and run tests every now and then?

Ideally, I would like to include this into my continuous build system as well. Is there a way to include it in the build?

Thanks,
Ingo

7

Hi Ingo!

You will find answers in the docs provided with Unity Test Tools :slight_smile:

To save you some trouble: you open the Integration Test Runner under Unity Test Tools menu bar (or press ctrl+alt+shift+t).

Tomek

8

Hi Tomek,

Thank you very much! I can now execute the integration tests that come with the AngryBots example. I’ve also found the UnityTestTools.pdf file. Looks like to run the tests on a continuous integration server, I’d run them in headless mode as described on page 13/14.

Unity.exe PATH_TO_YOUR_PROJECT -batchmode -executeMethod BatchTestRunner.RunAllTests -testscene=IntegrationTestsExample

This generates an XML file which I can parse to determine if they passed/failed. Do you recommend any specific plugins for running Unity tests on CI systems? For Jenkins there seems to be at least Unity3dBuilder.

Do tests still have to be stripped from the compiled game? Or will they already not be part of the resulting application? I just want to make sure that none of the test dependencies increase the size of the released game.

Thanks,
Ingo

9

There is a little mistake there. The PATH_TO_YOUR_PROJECT should be succeeded with -projectPath. You will find some more information about command lines you can use with Unity and how to run it in a batch mode here: http://docs.unity3d.com/Documentation/Manual/CommandLineArguments.html

I think you don’t need any plugins for just running tests. Simply run Unity in batch mode as described in the docs and the read and parse result file. Nevertheless, I never used any plugins for CI.

Unfortunately, not at the moment. You need to manually (or automatically in you build pipeline) strip them before building your production code. Unit tests, however, will not be included as long as you keep them in Editor folder.

10

Very nice tool and more stable than TestStar…

Unfortunately it seems that the XML output can’t be validate by the NUnit XSD
So my JenKins NUnit plugin don’t want to display the test result…
The test-suite element is missing.

11

Thanks for pointing it out! We’ll give it a look :slight_smile:

Tomek

12

Well i have added this

  private void AddDefaultTestSuite(ResultSummarizer summaryResults)
        {
            xmlWriter.WriteStartElement("test-suite");
            xmlWriter.WriteAttributeString("type",
                                            "TestFixture");
            xmlWriter.WriteAttributeString("name",
                                            "ALLTests");
            xmlWriter.WriteAttributeString("description",
                                            "ALLTests");
            xmlWriter.WriteAttributeString("executed",
                                            "True");
            xmlWriter.WriteAttributeString("result",
                summaryResults.Failures >0 ?  "Failure" : "Success");

            xmlWriter.WriteAttributeString("success",
               summaryResults.Failures > 0 ? "Failure" : "Success");

            xmlWriter.WriteAttributeString("time", "0.0");

            xmlWriter.WriteAttributeString("asserts", "0");

            xmlWriter.WriteStartElement("results");
        }

On InitializeXmlFile and i patched TerminateXmlFile()

private void TerminateXmlFile()
		{
            //Add by XL
				xmlWriter.WriteEndElement(); // test-suite
				xmlWriter.WriteEndElement(); // result
                
            //
				xmlWriter.WriteEndElement(); // test-results
				xmlWriter.WriteEndDocument();
				xmlWriter.Flush();

				xmlWriter.Close();
		}

And it’s working now…

But adding the test-suite notion to organize the integration tests could be nice.

13

Hi xlarrode,

are you saying (1) you customized Unity Test Tools to fix the XML output and (2) the Jenkins plugin can now detect if the tests passed/failed?

Thanks,
Ingo

14

Yes exactly…

15

There is a problem when trying to build a WinPhone8 project because the integration test framework relies on System.Xml.XmlTextWriter, which is not present on this target platform.

Could perhaps switch to System.Linq.Xml but that would break the web target.

16

Hi,

We’ve rewritten the result writer not to depend on the xml library. We also made sure the xml is compatible with appropriate xsd. This should solve some of the problems with platforms and result parsing. It will come with next release.

Tomek

17

That’s great!

I think the dependency should also be fine with XmlTextWriter being included in 4.3.3 now for WP8.

On a sidenote, we actually changed our codebase to use the Linq XML package so I wish Linq.Xml was added to WebPlayer for 4.3.3 instead of the missing System.Xml classes added to WP8 :smile:

18

Any advice for testing custom editor code in isolation? For example with a custom PropertyDrawer, field info and attribute info are only available as read-only instance properties, which can’t be set through the constructor of PropertyDrawer. And event data is given through the global current event property. So you can’t easily mock that data either.

I guess I could try to create the testing class as a subclass of the class I’m testing, and test the protected methods of the class instead of the public ones. But is that a good idea? Or would you recommend a different approach?

19

Hi RC-1290,

Unfortunately, the global event state makes some things hard to test. Maybe if you shared some your code you want to test, we could find a different approach.

Tomek

20

Thanks for the offer Tomek. I guess it was a bit silly of me to ask for advice without posting any example code. But I’m a bit reluctant to post the code with which I’m having trouble automating the testing, because it’s at the core of a product of which I’m not yet sure if I want to release it for free or not.

I’ll try to refactor the code for testing the protected methods (passing the required data as parameters, in stead of having the method accessing global state directly).

If it turns out that I can’t manage to figure it out on my own, I guess I might as well post the code here, because it won’t be a really reliable product if I can’t easily test it with future versions of Unity.