u3d an open source tool to Download, Install, Run Unity from the command line.

jerome-lacoste · August 31, 2017, 2:17am

Hello,

the DragonBox team is happy to release u3d, an open source project to interact with Unity from the command line.

It currently allows to:

download and/or install unity
uninstall unity
list the installed versions and their packages
run unity
prettify the output
display your activated licenses
integrates with FastLane (comes with its own action), and is Jenkins friendly

It knows about your project and can make decisions based on its configurations (which version you use)

It works on Mac, Linux, and Windows.

The latest version is v1.0.21, released 2018-04-27. Changelog

Some screenshots:

For more information about the features or question, check the project page on GitHub - DragonBox/u3d: U3d is a cross-platform set of tools to interact with Unity3D from command line.

We would love your feedback!

Thanks,

Jerome on behalf of the DragonBox team

ippdev · September 3, 2017, 6:18pm

Could you use this to run Occlusion Culling, Lightmapping and Editor plugins on a server…in Editor Mode obviously?

r0bur · September 5, 2017, 6:45pm

Hey! I work with Jerome on u3d!

While I’m not sure what you mean by Occlusion Culling, I have answer for Lightmapping and Editor plugins: u3d has been designed to work for a build server, so it will definitly work for some sort of baking server. The idea nevertheless is that it is meant to make working with Unity easier, not improve his feature set.

While I don’t know exactly the project that you have, here is what I would do if I wanted to bake lightmaps on a server using our tool; with no graphical interface, so only from command line:

Install our tool: gem install u3d
Install Unity with the version that you want: u3d install
Clone, download… a project with some prepared scenes to bake and a script to run Bake. Say MyBakerClass.MyBakingMethod().
NOTE: To create such a script (useful to bake without going through GUI) you will probably want to have a look at https://docs.unity3d.com/ScriptReference/LightmapEditorSettings.html and https://docs.unity3d.com/ScriptReference/Lightmapping.html.
Go to your project folder, and launch your script: u3d – -executeMethod MyBakerClass.MyBakingMethod() -batchmode -quit
This command would run your baking method in batchmode and exit when completing the task. More on that: https://docs.unity3d.com/Manual/CommandLineArguments.html

Similar steps could be taken to run Editor Plugins easily, and while I don’t know much about Occlusion Culling, I guess that if that involves Unity’s GUI you sure can create a script to bake that, and do the same.

I hope I answered your questions!

ippdev · September 5, 2017, 8:28pm

r0bur:

Hey! I work with Jerome on u3d!

While I’m not sure what you mean by Occlusion Culling, I have answer for Lightmapping and Editor plugins: u3d has been designed to work for a build server, so it will definitly work for some sort of baking server. The idea nevertheless is that it is meant to make working with Unity easier, not improve his feature set.

While I don’t know exactly the project that you have, here is what I would do if I wanted to bake lightmaps on a server using our tool; with no graphical interface, so only from command line:

Install our tool: gem install u3d

Install Unity with the version that you want: u3d install

Clone, download… a project with some prepared scenes to bake and a script to run Bake. Say MyBakerClass.MyBakingMethod().
NOTE: To create such a script (useful to bake without going through GUI) you will probably want to have a look at https://docs.unity3d.com/ScriptReference/LightmapEditorSettings.html and https://docs.unity3d.com/ScriptReference/Lightmapping.html.

Go to your project folder, and launch your script: u3d – -executeMethod MyBakerClass.MyBakingMethod() -batchmode -quit
This command would run your baking method in batchmode and exit when completing the task. More on that: https://docs.unity3d.com/Manual/CommandLineArguments.html

Similar steps could be taken to run Editor Plugins easily, and while I don’t know much about Occlusion Culling, I guess that if that involves Unity’s GUI you sure can create a script to bake that, and do the same.

I hope I answered your questions!

OC is built in to the engine. So…yeah…it is good to go. Neat toolset.

dmontesdeocaSMK · October 24, 2017, 7:30am

Hey!

I’m trying to use this with Jenkins, but I’m not sure how it’s integration should be. I’m running it as a windows command (on jenkins), but it throws me an error, saying that it couldn’t recognize ‘u3d’ as a command. When I launch it throught cmd it works correctly.

Any help here?

Thanks!!

jerome-lacoste · October 24, 2017, 8:29am

Hello @dmontesdeocaSMK . We use u3d with Jenkins ourselves, so maybe we can help!

Note: our servers are Mac, Linux so far.

What you need is to install ruby on your Jenkins server, and run u3d within a Ruby environment. I recommend using RVM to install ruby on your server and use the ruby RVM plugin (rvm) to integrate it with jenkins. Then enable the “Run the build in a RVM-managed environment”. After that you can use a shell plugin to run your u3d builds.

For example, here’s the code I use to install unity across servers.

if [[ ! `which u3d` ]]; then
  gem install u3d
else
  gem update u3d
fi

echo "${U3D_INSTALL_ARGS}"

PASS_KEY=U3D_PASSWORD_${NODE_NAME}
echo "PASS KEY: ${PASS_KEY}"
export U3D_PASSWORD=${!PASS_KEY}
u3d install --trace --verbose $U3D_VERSION $U3D_INSTALL_ARGS
u3d list

I also set up some environment variables such as

U3D_PASSWORD_name_of_my_slave → points to the root password for the slave named “name_of_my_slave”

You can use the Build Secret plugin to store your passwords and configure your job to pass them to your build script. Or you use the keychain integration if on mac.

and then pass your u3d configuration like this:

U3D_INSTALL_ARGS=-p Unity,Android,iOS,Linux,Windows,WebGL
U3D_VERSION=2017.1.2f1

(these could be set using the Jenkins Parameter plugin).

As for running builds, we are in fact using fastlane to drive our builds, and u3d is used in one of our lane to build the app before we let other fastlane actions (supply, deliver, hockey …) distribute the builds to Google, Apple, etc.

I plan on sharing our Fastfile (fastlane configuration file) in the near future.

A pure u3d Ruby Jenkins (GitHub - jenkinsci/jenkins.rb: Deprecated, see https://www.jenkins.io/jep/7) could have been possible, but I don’t intend to work on that as we use u3d within fastlane ourselves. Also as I’ve never seen the pure ruby Jenkins plugins solution take off, I am uncertain as this would be useful.

dmontesdeocaSMK · October 24, 2017, 9:17am

@jerome-lacoste Thank you very much!! Let’s do it. I’ll keep you updated.

jerome-lacoste · November 11, 2017, 10:17am

@dmontesdeocaSMK Did you manage to get something working?

jerome-lacoste · January 10, 2018, 8:08am

u3d 1.0.13 is released with a few interesting features

u3d/install: allow to specify packages download directory
u3d/available: introduce a central cache
u3d/available: allow to match using regular expression

This allows to do things like:

List the available versions matching 2017.3 and get the result almost instantaneously:

Full release changelog

jerome-lacoste · January 16, 2018, 11:54am

u3d 1.0.15 released

support downloading packages for another platform (already in 1.0.14)
support installing the 2018.1 beta, including the new VisualStudio and Facebook modules.

Full release changelog

bdovaz · January 16, 2018, 9:27pm

Thanks @jerome-lacoste great work! We are using it and it’s a lifesaver.

bcjordan · January 17, 2018, 8:59pm

Thanks for sharing @jerome-lacoste , this is super cool!

Are most others using Windows and OS X build machines? Do they require VNC-based GUI activation of licenses?

Would be really smooth if this could be used in Travis CI / Circle CI / GitLab CI to get build-and-upload setups going with the ease of cloud build. Linux instances tend to be quite a bit cheaper (.5x the cost of OS X / Windows VMs), and lots of great CI/CD tools are coming out that only support Docker/Linux-based containerization.

From what I can tell there is a breaking issue with headless license activation among the last year or two’s worth of Unity Linux builds (even using the -serial $UNITY_SERIAL -username “$UNITY_USERNAME” -password “$UNITY_PASSWORD” options, it will complain in various forms)

jerome-lacoste · January 17, 2018, 9:18pm

@bcjordan we primarily use macs at our office because we target IOS heavily, but I’ve started experimenting with Linux editor builds on the side. If the licensing wasn’t so in the way I would have done that a long time ago. Also we have not only to pay for a license per build server, but to also create a Unity account for each of them. And I also get various weird issues where the servers lose their licenses once in a while. This isn’t very satisfying.

This is one project where I played with circleci: GitHub - lacostej/u3d_ci_example: Toying around CI (circleci) and u3d

Given that it takes quite a bit to set things up, I wanted to play with docker instances (and generate all the configs) but didn’t get to do it. Happy to discuss this further anytime.

I have seen the editor CLI registration issue reported earlier online, but I can’t remember if I had the problem or not.

bdovaz · January 24, 2018, 4:48pm

@jerome-lacoste have you seen this?

https://blogs.unity3d.com/es/2018/01/24/streamline-your-workflow-introducing-unity-hub-beta/

It would be great if Unity integrates u3d like tool.

My only concern is that they will do like always, have a team for a short period of time where they finish this project and then move on to another thing and have this Unity hub abandoned.

rgjones · January 26, 2018, 4:40pm

Have you folks been able to deploy Unity apps in docker containers?
NVM… That would require GPU support which is only in alpha so far on Linux and not planned for windows.

jerome-lacoste · February 4, 2018, 5:55pm

@bdovaz we added support to the latest package installer for Linux. Sometimes in the future, I will look into the hub to see what it provides.

Evolht · March 8, 2018, 8:18pm

Hi Jerome! I don’t suppose you’d be willing to help me?

I currently have U3D setup and working which is awesome and I have it telling unity to make an xcode project in the builds folder. I can’t seem to figure out how to make u3d get fastlane to then recognise that new xcode folder and run my lane functions there.

Do I need a whole separate fastlane folder in my builds folder?

jerome-lacoste · March 8, 2018, 8:56pm

@Evolht

I should probably share our own fastlane setup. Here’s a small extract of our fastfile.

def u3d_run(args: nil, unity_version: nil)
  Dir.chdir('..') do
    path = Dir.pwd
    if(File.file? "#{path}/u3d_rules.json")
        ENV['U3D_RULES_PATH'] = "#{path}/u3d_rules.json"
    else
        UI.message 'Using default u3d logging rules'
    end
    u3d(
      run_args: "-logFile '#{path}/u3d.log' -projectpath \"#{path}\" #{args}",
      unity_version: unity_version
    )
  end
end

def run_simplebuild(method)
  # taken from https://docs.unity3d.com/Manual/CommandLineArguments.html
  valid_targets = %w[win32 win64 osx linux linux64 ios android web webstreamed webgl xboxone ps4 psp2 wsaplayer tizen samsungtv]
  # taken from SimpleBuild, list of execute methods that match Perform#{method}Build
  valid_methods = %w[Android AndroidSplit IOS Linux32 Linux64 LinuxUniversal MacOSX MacOSX64 MacOSXUniversal WebGL Win32 Win64]

  if !valid_methods.include? method
    UI.important "run_simplebuild: method '#{method}'' not found in list of known SimpleBuild methods '#{valid_methods.join(',')}'. Build might fail. Check your parameter"
  end
  active_target = method.downcase

  if !valid_targets.include? active_target
    if active_target.index('mac')
      active_target = 'osx'
    elsif active_target.index('android')
      active_target = 'android'
    elsif active_target.index('linux')
      if active_target == 'linux32'
        active_target = 'linux'
      else
        active_target = 'linux64' # doc unclear about whether LinuxUniversal should use activeTarget linux64
      end
    else
      raise ControlledAbortError.new("run_simplebuild: can not find a valid buildTarget for method '#{method}'")
    end
  end
  UI.message "Mapped method '#{method}' to active target '#{active_target}'"
  u3d_run(args: "-quit -batchmode -buildTarget #{active_target} -executeMethod WWTK.SimpleBuild.Perform#{method}Build")
end
desc "Slack something if slack is enabled (SLACK_URL defined). Options:\n"\
   "* @:message: the message\n"\
   "* @:success: true by default\n"\
   "* @:payload: defaults to BUILD_URL information"
lane :do_slack do |options|
  if ENV['SLACK_URL'].to_s.strip.length == 0
    UI.message("Missing SLACK_URL. No slack")
  else
    slack(
      message: options[:message],
      success: !options.include?(:success) || options[:success],
      payload: options[:payload] || {
        "Build URL" => ENV['BUILD_URL'],
        "Product info" => "BUILD_PRODUCT_ID: #{ENV['BUILD_PRODUCT_ID']} V:#{release_version_number}-#{ENV['BUILD_NUMBER']} --env=#{Actions.lane_context[Actions::SharedValues::ENVIRONMENT]}"
      }
    )
  end
end

platform :ios do
  before_all do
    check_env(keys: ['BUILD_PRODUCT_ID', 'BUILD_BUNDLE_ID',
                     'XCODE_VERSION', 'HOCKEY_API_TOKEN',
                     'FL_UNLOCK_KEYCHAIN_PASSWORD', 'MATCH_PASSWORD'])
    xcversion(version: ENV['XCODE_VERSION'])
  end

  desc "Build and deploy IOS binary to HockeyApp. Also slack."
  lane :all_Hockey do
    unity_ios
    xcode_dev
    hockey_ipa
    do_slack(message: "IOS build sent to HockeyApp")
  end

  desc "Build Unity for IOS"
  lane :unity_ios do
    simplebuild(target: "IOS")
  end

  desc "Build IOS for dev"
  # works from xcode 8 but only tested from xcode 9
  lane :xcode_dev do
    xcode9(profile_type: "adhoc")
  end

  # for development, prov profiles are of type adhoc as per Apple's rules
  lane :xcode9 do |options|
    profile_type = options[:profile_type]

    # password passed using standard fastlane environment variable
    # Note: by default no options passed to set-keychain-settings, meaning it is unlocked without timeout!
    unlock_keychain(path: "fastlane")

    match(readonly: true,
          keychain_name: "fastlane.keychain",
          shallow_clone: true,
          type: profile_type)

    # we don't use all of these, but good for documentation
    sigh_key="sigh_#{ENV['BUILD_BUNDLE_ID']}_#{profile_type}"
    env_pp_uuid="#{sigh_key}"
    env_pp_name="#{sigh_key}_profile-name"
    env_pp_path="#{sigh_key}_profile-path"
    env_pp_team_id="#{sigh_key}_team-id"

    pp_path=ENV[env_pp_path]

    raise "Couldn't find provisioning profile at #{pp_path}" unless File.file? pp_path
    UI.message "Using provisioning profile #{pp_path}"

    xcodeproject_path = "target/#{ENV['BUILD_PRODUCT_ID']}_ios/Unity-iPhone.xcodeproj"

    update_project_team(
      path: xcodeproject_path,
      teamid: ENV[env_pp_team_id]
    )

    update_project_provisioning(
      xcodeproj: xcodeproject_path,
      profile: pp_path,
    )

    gym(project: xcodeproject_path,
        scheme: "Unity-iPhone",
        archive_path: "target/#{ENV['BUILD_PRODUCT_ID']}_ios/build/Unity-iPhone.build",
        xcargs: "CODE_SIGN_IDENTITY='iPhone Distribution'",
        #export_options: {
        #  provisioningProfiles: {
        #    ENV['BUILD_BUNDLE_ID'] => ENV[env_pp_name]
        #  }
        #},
        buildlog_path: "#{project_dir}/target/xcode_build_logs",
        output_directory: "target/",
        output_name: "#{ENV['BUILD_PRODUCT_ID']}.ipa")
  end

  desc "Deploy IPA to Hockey"
  lane :hockey_ipa do
    markdown_type = 1
    hockey(
      ipa: "target/#{ENV['BUILD_PRODUCT_ID']}.ipa",
      api_token: ENV['HOCKEY_API_TOKEN'],
      notes: jenkins_dev_change_log,
      tags: 'qa',
      #notify: '2', # Notify all testers
      #status: '2', # Make available for download
      #release_type: '2' # 'alpha' release type
    )
  end

There are several things I would do differently, if I had to start today, in particular our SimpleBuild integration (our own Editor Build class). We haven’t had to touch it much in years, so I don’t bother. Let me know if that helps, or not!

jerome-lacoste · March 8, 2018, 8:58pm

As to the answer to your specific question, I specify the xcodeproject_path to gym in the above code extract

    xcodeproject_path = "target/#{ENV['BUILD_PRODUCT_ID']}_ios/Unity-iPhone.xcodeproj"

Evolht · March 12, 2018, 10:02pm

This is excellent! I’ve managed to get it working and now my team thinks I’m some kind of god! Thank you so much

Topic		Replies	Views
Automating builds with Jenkins News & General Discussion	49	34584	May 4, 2019
Occlusion culling Bake on Start() ?? Unity Engine Scripting	2	1869	February 21, 2015
[Released] Batch Processor Community Showcases Asset-Store-Assets	5	2192	June 18, 2014
No way to bake occlusion culling from script? Unity Engine Scripting	9	8568	October 4, 2021
Occlusion Culling For Unity Free Users Community Showcases Asset-Store-Assets	21	5802	June 23, 2012

u3d an open source tool to Download, Install, Run Unity from the command line.

Related topics