Automating builds with Jenkins

News & General Discussion
1

UPDATE (2017/08/31): we have moved away from using the Unity3d plugin in Jenkins. We wanted to have the ability to replicate more of the commands locally. So we have created the u3d tool to download/install/run unity from the command line. See u3d an open source tool to Download, Install, Run Unity from the command line. - Industries - News & General Discussion - Unity Discussions for more information.

The original Jenkins plugin is still there of course. It is maintained by the community of Jenkins users and developers.

Automating builds is a great way to be more productive and more professional.

At WeWantToKnow, we target 8 different platforms. Doing this by hand would be very time consuming. So we have lots of automated jobs. Currently over 100 jobs, with over 60 Unity3d ones.

Hereā€™s a example of the jobs for one of our products, DragonBox Elements.

2070886--135116--Screen Shot 2015-04-16 at 11.07.01.png
2070886--135116--Screen Shot 2015-04-16 at 11.07.01.png1205Ɨ607 122 KB

There are multiple ways to achieve this with Unity3d, for example using the Cloud Build service. Others might prefer to use a custom / personnal build server like Jenkins. For example if you cannot use Cloud solutions, or need more control of the build processes.

A few years ago, I developed the Unity3d Jenkins plugin (https://wiki.jenkins-ci.org/display/JENKINS/Unity3dBuilder Plugin), and this thread is open here for people to give their feedback, share their use cases or just network a bit.

Also, if you want to help improving the plugin or just working on tools related to build automation, feel free to take contact. I am open for hacking sessions, etc.

4 Likes
2

hey jerome,

first of all thanks for developing the plugin.

i ran into some trouble, maybe you can help? disclaimer: iā€™m a first time jenkins user and do usually work under windows.

after installing the whole thing and setting up java (what fun that is under osx) i happily installed your plugin, set up my job and it ran up until the point where it tried to copy files (it was the xcode project file i think). so i guess it had something to do with the access rights. so i had a look where all that stuff got placed.

i was surprised to see the jenkins workspace under /Users/dev/.jenkins instead of /Users/Shared/Jenkins. but if it is allowed to create all those files under /Users/dev it has to be the dev user executing the build process which in turn wouldnt make sense because then the file copying should work as the target folder was under /Users/dev too.

so i did what i learned from windows: i restarted.

when i logged into the jenkins web interface after that there were no more jobs and all my settings were gone. when i added a new job i saw why: now the workspace is /Users/Shared/Jenkins. well, ok, i thought and added the plugin and set up the job again. now it fails because it cannot acquire the graphics resources as you stated on the plugin site (The values of CFBundleTypeRole entries must be ā€˜Editorā€™, ā€˜Viewerā€™, ā€˜Noneā€™, or ā€˜Shellā€™). so i guess that is because i am logged in as dev, but jenkins runs as the (somewhat hidden) jenkins user?

soā€¦ maybe youā€™re a superhero and understood all that mess iā€™ve been talking :slight_smile:

cheers

EDIT:
ok, iā€™m logged in as the jenkins user now. iā€™m not really comfortable with that, but i really want to see a successful build :slight_smile:
problem stays the same, it cannot copy:
Error building Player: IOException: Failed to Copy File / Directory from ā€˜Temp/StagingArea/Trampoline/Unity-iPhone.xcodeproj/project.pbxprojā€™ to ā€˜/Users/Shared/Jenkins/Target/x/Unity-iPhone.xcodeproj/project.pbxprojā€™.

iā€™ve tried copying the file via terminal (same user) and that worksā€¦ :confused:

EDIT2:
iā€™ve manually created the target folder for the project.pbxproj file, so it gets copied now, yay. next problem is:
Error building Player: DirectoryNotFoundException: Directory ā€˜/Users/Shared/Jenkins/Target/mellerud/Dataā€™ not found.

so it seems like unity or your plugin (or whoever tries to copy the files) is not able to create the folders. any idea?

EDIT3:
it was BuildOptions.AcceptExternalModificationsToPlayer which only applies to ā€œappendā€-builds and does not work work for the first one.

3

Hello cp_sn, a bit hard to troubleshoot things from here, but Iā€™ll try.

WRT to troubleshooting, I recommend that you look at Unity3d . Try the various steps from there.

WRT to your target directory, my target directories are always related to my jobs. This to avoid issues in the long run. Do you have the same setup ? It doesnā€™t look from your examples.

As for appending, I tend to make build clean projects (remove data before hand). Appending has caused me problems in the past when removing files. The speedup wasnā€™t worth it for me.

4

Hey. I just stumbled upon this plugin. Looks really nice! Is it still actively used and developed? Happy to contribute if I find any bugs/gaps over the next week.

5

Hei strich. The plugin is used and developed. I am considering rewriting it from scratch using a set of ruby wrappers but I donā€™t know how usable that would be in Windows land.

Otherwise if you want to tackle issues, feel free to look the wiki page. Maybe Loading...

Feel free to take contact.

6

You shouldnā€™t rewrite if itā€™s going to break cross-platform. Iā€™ll probably touch base again in a week or so once I have a good feel for the plugin and Jenkins in general.

7

Hi, Jerome! Iā€™ve been using your plugin for a long time and itā€™s doing great job so thank you! Recently Iā€™ve had a question, is there a possibility to have 2 different versions of Unity on build machine with Jenkins and configure which version to use for particular build? For example, to make some updates for old projects I want to use pre-5.3 Unity version, and for my new projects I want use newest Unity version.

8

any suggestions on which plugin to use to upload to the new testflight within apple?

9

Hi Jerome,

this is not an issue with your plugin, but as you are using Jenkins with Unity3D you might have some idea on this issue ā€¦
Iā€™ve set up Jenkins to run a build with the exact same arguments that we used when running the builds through manually started batch scripts. Running the batch manually on the build server works fine, when Jenkins starts a build though (no matter if through a Windows Batch build step or your plugin) it always fails at updating a sound file with a hard crash. Even the same stack trace each time and there doesnā€™t seem to be any record of that issue on the net so far.
End of the log can be seen here: Unity3D build from Jenkins - Pastebin.com

Wonder if you had any idea what might cause this. The only difference I can see at all between manually invoking the build and having Jenkins do it is that Jenkins runs as a service. But I suppose it does so for everyone so if this was the problem it should affect anyone that does Unity builds through Jenkins?!

Iā€™m really at a loss here currently :frowning:

Regards,
Chris

10

Ok, seems to be solved. The important hint was at the start of the Unity log:

FMOD failed to get driver capabilities ... Error initializing output device.

I donā€™t know why it worked with the manually run build but not the one from Jenkins, but starting the Windows service ā€œWindows Audioā€ fixed the Jenkins one.

11

Kind of a necro post, but since there hasnā€™t been any new versions in some time Iā€™m assuming this thread is still semi active. After getting a local git server running, installing Jenkins, and setting up your plugin, I seen to be unable to build a project through Jenkins. If I run my script from Unity there are no problems, but from Jenkins not a single one of my assets gets copied to the build folder. Library and project settings move just fine so I donā€™t believe it has anything to do with permissions. The code works fine from outside of Jenkins however, which points towards permission issues. Help? Unity 5.3.2f1, Windows 7 x64, Unity, git, and Jenkins are all local. Jenkins is still the system user as per default install options, the plugin is definitely running, but I donā€™t get any assets moved over to the build workspace, and no build is actually output. Stranger still is the fact that everything completes in the build step and moves on to the next step without error. Please let me know if you need further information.

12

Hello @jaspercayne

I havenā€™t monitored this thread properly, hence the lack of responses :slight_smile:

Have you tried the plugin troubleshooting section, in particular check the section about the -logFile argument, running the command in a Windows batch command, as a jenkins Batch script or in a CMD,

If those 2 works, then the problem could be in the plugin. Otherwise itā€™s most probably external.

13

Thanks @Alloc , that problem is apparently affecting several games running 5.x. The internal logging mechanism in unity makes it sometimes hard to view these issues in the large log files.

14

@Nihil688 I am using fastlane for all our automation needs. Not really a jenkins plugin though, but integrates well as long as you have it installed on your server.

15

@ruslan.k it is possible to install many versions of Unity on the same machine. The plugin documentation points to some ways of doing it

16

Does this work with Jenkins 2.0?

17

@longroadhwy AFAIK Jenkins 2.0 is backwards compatible with 1.0. The version was bumped for marketing/roadmap reasons :slight_smile:

18

Hey thanks for that @jerome-lacoste , Iā€™ve been trying to integrate that since your post but Iā€™ve been having issues, any guides out there?

I have found this: https://labs.kunstmaan.be/blog/ios-continuous-delivery-with-jenkins-and-fastlane but itā€™s a bit fiddly with unity builds and I canā€™t seem to get it working

19

Hello Jerome, Iā€™ve been using your great unity plugin for quite some time now in combination with jenkins, and it works like a charm, thank you for that! But I have to admit I have a problem with the new Jenkins pipeline project type and getting it to run properly. I am aware of the ā€œtoolā€ command that retrieves the current location of Unity in combination with potential automatic install. Problem is, I havenā€™t yet figured out a way to use your plugin to actually execute the build process inside the pipeline. The reason is of course to get ā€œliveā€ access to what is happening inside the build process. I could of course dump the editor.log file right after the build has run, but in the past I have run into endless loops in the build process which I wouldnā€™t be able to detect except for taking a look at the editor.log on the according node - or using your plugin :slight_smile:

Right now, my pipeline code looks something like this:

node('unity') {
   // Mark the code checkout 'stage'....
   stage 'Checkout'
   git env.GITREPOLOCATION

   def WORKSPACE = pwd()
   def unityHome = tool name: 'Unity current', type: 'org.jenkinsci.plugins.unity3d.Unity3dInstallation'

   stage 'Build Maingame'
   bat "\"${unityHome}\\Editor\\Unity.exe\" -projectPath \"${WORKSPACE}\\myproject\" -quit -batchmode -executeMethod BuildCraftyProject.PerformWindowsDevBuild \"-targetdir ${WORKSPACE}/build/\""

   stage 'Build Bundles'
   bat "\"${unityHome}\\Editor\\Unity.exe\" -projectPath \"${WORKSPACE}\\myproject\" -quit -batchmode -executeMethod BuildCraftyProject.PerformBundlesBuildWindows \"-targetdir ${WORKSPACE}/corebuild/\""
}

Sorry if the answer is obvious, Iā€™m a coder but not yet familiar with what groovy can actually do or not do. And thanks for any help on this! :slight_smile:

20

Hi Guys,
we start having Unity Pro license problems with Jenkins recently. We are not using Unity Plug -in, just running unity command line from batch files and DOS commands. We do not have a way to reproduce this yet, seems random right now. It maybe linked to number of concurrent jobs on the same machine, before it was only two and now we started adding a third one. After doing a couple of builds Unity forgets the license and logs this:

BatchMode: Unity has not been activated with a valid License. Could be a new activation or renewalā€¦
(Filename: C:/buildslave/unity/build/Editor/Platform/Windows/WinEditorMain.cpp Line: 795)
DisplayProgressbar: Unity license
Canceling DisplayDialog: Failed to activate/update license. Missing or bad username and password. Please try again using valid credentials or contact support@unity3d.com
(Filename: C:/buildslave/unity/build/Editor/Platform/Windows/EditorUtility.cpp Line: 1148)

after manual reset it seem to run for a bit and then the same thing happens.

Any thoughts would be appreciated.

Yury