My disappointing experience using Unity in a CI pipeline

[JesseSTG_1]

This past week I’ve been trying to set up a CI pipeline for my game. I’m currently using the Personal edition of Unity 2019.2.2, but will upgrade to Unity Plus if I need to. I want my pipeline to automatically do these three things, so I don’t have to do them by hand:

• To ensure that my tests pass when I make changes to my game

• To ensure that my game builds on all the platforms I support

• To deploy to all platforms and storefronts I support

I looked into Unity Cloud Build, but unfortunately that’s not an option for me, for these reasons:

• My game uses Burst, which cannot be cross-compiled, and therefore cannot be built on Unity Cloud Build

• Unity Cloud Build does not support building for consoles, which I intend to do

• I’m using Trimmer to build multiple variations of my game, and Unity Cloud Build doesn’t support some of the features that Trimmer uses

Instead, I decided to try out the GitHub Actions beta for these reasons:

• I already have a paid GitHub plan, so I get 3000 minutes included per month

• It supports Windows, macOS, and Linux, so I can build for any supported Unity platform on the same pipeline

• If I want to add some build step external to Unity (like importing files or generating code), I can do that at my leisure

GitHub Actions has been a great experience for me; it’s doing everything I want it to and doesn’t get in my way with technical limits. Seriously, give it a try…with a project unrelated to Unity. Here’s why.

Unity recently published some improvements to their test framework , and has expressed a renewed interest in CI . These new efforts led me to believe that I could set something up that would at least fit my requirements, even if there were some warts along the way. I was mistaken.

Unity’s licensing mechanism is 100% getting in the way of me building the CI pipeline that I want. Currently, a Unity license must be obtained manually, either in the editor or through the process described here and here. For manual use (i.e. when actually designing a game), this is fine; I had no issue installing and licensing Unity for my development machine.

The problem is that you still have to do this when installing Unity on a CI runner. Most CI services don’t make promises about a runner’s identifying details (network address, hostname, serial numbers, other OS identifiers). As a result, a Unity license for a CI runner could become useless at any time without warning. In fact, that’s exactly what’s been happening with my own pipeline. This undermines every effort the Unity developers make to offer a smooth automation experience. What’s the point of automated builds, tests, or deployments if I have to manually license Unity every time I run my CI pipeline?

I don’t have a problem with downloading and installing Unity on the CI runners, nor do I with actually invoking it on the command line. It’s rougher around the edges than I’d prefer, but at least it fits my needs. All I want is to be able to use Unity for automated builds, test, and deployments in a CI runner without any manual intervention; the best way I can see that is to remove or totally automate the licensing process for CI pipelines.

Or, if your legal team would get nervous, perhaps consider special CI-only builds of Unity that don’t need to be licensed, but can only be run headless? Such a build couldn’t be used for designing content, and thus would not be worth pirating.

I love Unity and it’s generally a bliss to work with, but in this case I’m incredibly disappointed. I know the dev team can do better.

[superpig]

What was the actual problem you ran into? Are you trying to preinstall the license inside a Docker container, or something?

We are working on a new licensing system that will make this sort of situation much smoother, but it should still be possible to get something working today, albeit with a bit of a bumpy ride.

[JesseSTG_1]

Hi, Richard. Thank you for your reply, especially on a Saturday night.

@superpig

What was the actual problem you ran into? Are you trying to preinstall the license inside a Docker container, or something?

No. This is what I did for each operating system;

• Spin up a runner

• Install Unity (or, in the case of Linux runners, pull this Docker image) as part of the script

• Request a license as described here (section “Create activation file and import license by command”), and save the .alf file

• Add the generated .alf file as an artifact, then manually get a license as described here (The runner is finished by this point)

• Manually add the contents of the .ulf license file as a secret to my GitHub configuration (not the repository, to a config item on the website)

• Spin up a new runner of the same OS (I can’t continue using the first runner because of the manual intervention needed)

• Install Unity again

• Activate the license as described here (section “Create activation file and import license by command”)

• Run the tests or build the game (depending on the workflow that’s running)

• If applicable, deploy the game

If I already have a license then steps 1-5 are ignored. They’re still run, but I haven’t figured out how to conditionally disable them yet.

The problem is:

• This works well enough on Github Actions’ Linux runners, I guess because of the Docker image I’m using. But I have no idea when the license will be made useless under my feet (because of changes to the runner’s environment)
• I believe that at a minimum I have to do this every time I upgrade Unity (which I do frequently, even for patch releases)
• This doesn’t work at all on the macOS or Windows runners, because whatever information Unity uses to request a license doesn’t seem to be constant as I expected. As a result, I have not managed to make a single successful macOS or Windows build on GitHub Actions. When I try to activate Unity on these runners I will get errors that look like this:

/Applications/Unity/Unity.app/Contents/MacOS/Unity -quit -batchmode -username <redacted> -password <redacted> -manualLicenseFile /Users/runner/.local/share/unity3d/Unity/Unity_lic.ulf -nographics -silent-crashes -disable-assembly-updater -logFile -

Initiating legacy licensing module
DisplayProgressbar: Unity license

LICENSE SYSTEM [2019831 16:49:13] Next license update check is after 2019-09-01T16:20:51


LICENSE SYSTEM [2019831 16:49:13] VMuD6VlWKP6I != VMPkeFQqLH/F

[UnityConnectServicesConfig] config is NOT valid, switching to default
User <redacted> logged in successfully
Cancelling DisplayDialog: Failed to activate/update license. Timeout occured while trying to update license. Please try again later or contact support@unity3d.com
This should not be called in batch mode.
(Filename: /Users/builduser/buildslave/unity/build/Editor/Platform/OSX/EditorUtility.mm Line: 288)

It looks about the same on the Windows runners, but I can post more detailed information or logs if you’d like.

And, of course, these extra steps eat up my runner minutes.

@superpig

We are working on a new licensing system that will make this sort of situation much smoother, but it should still be possible to get something working today, albeit with a bit of a bumpy ride.

Color me intrigued. If it’s at all possible, can you tell me more about how it’ll work, and what kind of timeline you have in mind?

[superpig]

@JesseSTG_1

The problem is:

• This works well enough on Github Actions’ Linux runners, I guess because of the Docker image I’m using. But I have no idea when the license will be made useless under my feet (because of changes to the runner)
• At a minimum, I have to do this every time I upgrade Unity (which I do frequently, even for patch releases)
• This doesn’t work at all on the macOS or Windows runners, because whatever information Unity uses to request a license doesn’t seem to be constant as I expected. As a result, I have not managed to make a single successful macOS or Windows build.

Right, the .alf file contains machine binding information that you should not be relying on from run to run.

Ohhhhh, you’re on Personal. Right, so I think usually when people are doing this they’re on Plus or Pro, which means they have a serial number and can just pass it to the Editor with -serial (and then -returnlicense when done), as per the documentation, in a kind of ghetto license-leasing setup. If you want you could contact your regional salesperson and ask for a trial license for Plus, in order to test it out. Unfortunately we don’t have a good solution for Personal right now AFAIK

Color me intrigued. If it’s at all possible, can you tell me more about how it’ll work, and what kind of timeline you have in mind?

Well, there are technical and business decisions that haven’t been made yet, so don’t take this as any kind of guarantee or promise… but I think we’re headed towards a future where you can just sign into your Unity account, and then an appropriate license is acquired for you automatically, without the license being bound to your specific machine. We’ve been beta-testing it with a handful of enterprise customers for the past six months or so, and that’s gone well, but there’s a lot of work still to do before a public release - so no timeline to share just yet.

[JesseSTG_1]

@superpig

Ohhhhh, you’re on Personal. Right, so I think usually when people are doing this they’re on Plus or Pro, which means they have a serial number and can just pass it to the Editor with -serial (and then -returnlicense when done), as per the documentation, in a kind of ghetto license-leasing setup. If you want you could contact your regional salesperson and ask for a trial license for Plus, in order to test it out. Unfortunately we don’t have a good solution for Personal right now AFAIK

Huh. Wasn’t aware of this. So Plus/Pro don’t use the .alf/.ulf system?

Forgot to mention; another big plus of GitHub Actions is that I can run many jobs at the same time. As of right now, my attempt at a pipeline runs ten jobs concurrently; four to activate Unity (one for each build target), four to build and deploy the game (one for each build target), and two to run the tests (play mode and edit mode).

I could try this out, but the FAQ tells me that I can only use two machines per license. That’s a problem, because I have Unity Personal installed on three computers.* Even if I uninstalled Unity on two of them (I can’t because I use them for development), I wouldn’t be able to run more than one job at a time.

*Two unique machines, one of which has two operating systems installed.

@superpig

Well, there are technical and business decisions that haven’t been made yet, so don’t take this as any kind of guarantee or promise… but I think we’re headed towards a future where you can just sign into your Unity account, and then an appropriate license is acquired for you automatically, without the license being bound to your specific machine. We’ve been beta-testing it with a handful of enterprise customers for the past six months or so, and that’s gone well, but there’s a lot of work still to do before a public release - so no timeline to share just yet.

Fair enough. Thank you for the honesty; if this is already in the oven then I suppose all you can do is keep cooking it. Do you at least have an idea of the order of magnitude you’re working with? (Weeks? Months? Years?)

[Yacuzo]

+1 for this issue in general.
Also +1 for a completely headless unity executable and headless installer. This would make it possible to run on more efficient headless OSes and make it possible to automate the installation of new unity versions on the runners. The last one is important if you update often.

Edit: It would be a dream if it could be run on Windows Nano. Windows Server is so heavy.

[superpig]

@JesseSTG_1

Huh. Wasn’t aware of this. So Plus/Pro don’t use the .alf/.ulf system?

All versions of Unity use .ulf licensing (for now), but usually - like when you activate Unity interactively - the file is automatically generated, uploaded, and the .ulf is downloaded and installed. The problem is that we have no way to trigger that mechanism from the command line without providing a serial number, which you don’t have for Personal (it’s generated automatically during activation).

Forgot to mention; another big plus of GitHub Actions is that I can run many jobs at the same time. As of right now, my attempt at a pipeline runs ten jobs concurrently; four to activate Unity (one for each build target), four to build and deploy the game (one for each build target), and two to run the tests (play mode and edit mode).

I could try this out, but the FAQ tells me that I can only use two machines per license. That’s a problem, because I have Unity Personal installed on three computers.* Even if I uninstalled Unity on two of them (I can’t because I use them for development), I wouldn’t be able to run more than one job at a time.

*Two unique machines, one of which has two operating systems installed.

Indeed, if you want to run Unity on four build nodes, you would need at least two licenses. For Personal that’s not a problem because you can just generate as many as you want, but not so for Plus/Pro…

Fair enough. Thank you for the honesty; if this is already in the oven then I suppose all you can do is keep cooking it. Do you at least have an idea of the order of magnitude you’re working with? (Weeks? Months? Years?)

I think we’re talking 1-2 years to roll out the complete solution. You may see individual pieces start to appear much earlier than that though - the client-side pieces of new system are already included in 2019.3, for example.

[JesseSTG_1]

@superpig

All versions of Unity use .ulf licensing (for now), but usually - like when you activate Unity interactively - the file is automatically generated, uploaded, and the .ulf is downloaded and installed. The problem is that we have no way to trigger that mechanism from the command line without providing a serial number, which you don’t have for Personal (it’s generated automatically during activation).

Is this inherent to the licensing process? There’s no way you could offer serials for Personal?

@superpig

Indeed, if you want to run Unity on four build nodes, you would need at least two licenses. For Personal that’s not a problem because you can just generate as many as you want, but not so for Plus/Pro…

My budget is best described as “shoestring”. I can’t justify three (builders + test runners) Plus licenses right now.

@superpig

I think we’re talking 1-2 years to roll out the complete solution. You may see individual pieces start to appear much earlier than that though - the client-side pieces of new system are already included in 2019.3, for example.

Can you tell us more about these client-side pieces that made it to 2019.3?

[DylanYasen]

omg I’ve been banging my head against the wall trying to get github action working. following this thread

[Awarisu]

@superpig

we have no way to trigger that mechanism from the command line without providing a serial number

As a quick fix, could you not just add something like a -personal command line switch? These licenses are handed out for free anyway.

[liortal]

@superpig i’ve tried the -serial option a long time ago, it did not work for us.
See my other (orphan) thread that received 0 replies here: https://forum.unity.com/threads/automated-unity-license-activation.770069/

[goldbug]

I have a similar problem with the current license:

I have a github repository with a github action that downloads and install unity personal in a docker container with a manual activated license. https://github.com/MirrorNG/MirrorNG/runs/440714979?check_suite_focus=true

It was a royal pain to set up, but alas, it is done.

The challenge I have is that the license is set up as a secret, which means it is not available for pull requests from forks. Note this restriction applies to any CI/CD tool, not just github actions. So I cannot run my CI/CD pipeline for pull requests. I cannot tell if a pull requests passes the tests or even compile without a lot of manual work.

I would really like for a way to have unity command line run without having to provide secrets.

[hmusr15]

@JesseSTG_1 So building the healdess server in Docker is your problem. Not releasing the headless build in a docker container to a server to be played on? Am I correct in my assumption of this?

[JesseSTG_1]

@hmusr15

@JesseSTG_1 So building the healdess server in Docker is your problem. Not releasing the headless build in a docker container to a server to be played on? Am I correct in my assumption of this?

My game doesn’t run as a server, it’s offline. I’m not sure I understand the question, can you clarify?

[RDeluxe]

Digging this up. After 1 year, where are we on this ?

I’m currently setting up Github actions and it seems to run “okayish”, but I still wish the process was simpler.

[Kiran-Bsw]

@JesseSTG_1

Is this inherent to the licensing process? There’s no way you could offer serials for Personal?

My budget is best described as “shoestring”. I can’t justify three (builders + test runners) Plus licenses right now.

Can you tell us more about these client-side pieces that made it to 2019.3?

Hey there! Unity CICD is royal pain. Currently struggling to setup github actions on unity game. Do you have any working reference code then it would really help me.

Thank you.

[JesseSTG_1]

@Kiran-Bsw

Hey there! Unity CICD is royal pain. Currently struggling to setup github actions on unity game. Do you have any working reference code then it would really help me.

Thank you.

Nope, sorry. I’m setting up a solution on AWS instead; Unity just takes too long to install and set up for it to be viable on GitHub Actions. When it’s ready I’ll be sure to share some code.

[jcs090218]

I have managed to automate the update license task, see the link JCSUnity/.github/workflows/license.yml at master · jcs090218/JCSUnity · GitHub.

The license is updated every time in schedule (everyday in my workflow file), so you don’t have to worry about license expiration anymore!