Would it be possible to run steamcmd to upload a build to Steam by using a Post Build Script?
I’m curious if anyone has tried it. The roadblocks may be:
Access to the Steam upload tools. Could be solved by adding to own repo and accessing in build script using relative path?
Making sure the Steam upload tools work. I believe this means using the mac version, as all cloud builds run on macs.
Get access to the uncompressed build for upload. I am not sure if we can do that, or if there would need to be a manual unzip. Is there some scratch directory we could use?
So the idea is to have the Post Build script automatically upload the build to Steam. It could use environment variables for things like the steam branch name.
Getting closer with this. I’m able to access the uncompressed build in $WORKSPACE/.build/last/ from a post build script.
I have been able to successfully launch steamcmd.sh, but it fails due to needing a steam code. steamcmd supports a “set_steam_guard_code” parameter, but I don’t think is going to work with Unity Cloud Build spinning up a new virtual machine for each build.
I think @victorw will know more about whether it’s possible but I’m not sure it would be easy, especially if you need to run custom upload tools as I doubt Unity would allow that. The only reason you can upload iOS builds is because the tools are part of xCode and are already installed.
If it helps I used to upload to TestFlight using an external server. You can setup cloud build to notify you when a build is complete. I used that to call a script on my server that then used the cloud build API to get a list of the latest cloud builds, download the uncompress build using the data provided and upload it to TestFlight. Maybe you could do something similar for Steam?
p.s. Here’s a guide I wrote for creating a ‘build bot’ for Microsoft Teams. The PHP script might be handy if you take the server approach, it allows you to download all the latest cloud build info including download URLs etc… Guide: Creating a cloud build bot in Microsoft Teams
I have a system that uses a node app on EC2 to listen for build notifications, which then downloads the build, unzips it, and then uploads to steam. However I am looking for a zero cost solution, and this feels like it is close (I’ve already verified that steamcmd launches fine). The only known roadblock left is the steam guard code.
You can upload directly to TestFlight using a Post Build Script in UCB, which was the inspiration for doing the same for Steam.
Heh, I’m in the forums today to see if anyone’s solved this. I think I’ve actually gotten further than most, though, so hopefully I can point a direction. Ultimately, at least for me, I don’t think post-build scripts will work for Steam without some changes from Unity, though.
My environment: Mac build of a Mac target.
The Steam docs mention that, if you’re using a machine that gets reimaged regularly, like a VM, then you just need to make sure to repopulate the <steam>/config/config.vdf and <steam>/config/ssfn* files. The latter is a “Sentry File” (its exact filename is given on a “SentryFile” line in config.vdf), and contains the “proof” that the machine has previously passed Steamguard.
But there’s more.
The additional difficulties and my workarounds:
[edit] All of this is being orchestrated by a post-build.sh script that I put into my git tree.[/edit]
Steam isn’t installed on the build servers (…is it?). I added the ContentBuilder directory to my git tree, at mytree.git/tools/steam/, so for instance, mytree.git/tools/steam/builder_osx/steamcmd.sh is the path to steamcmd.
$HOME on the build server is /BUILD_PATH, so Steam’s directory would normally be at /BUILD_PATH/Library/Application Support/Steam. Unfortunately, something in that path isn’t writable, so you can’t put Steam’s files there.
I worked around this by adding a stub Steam directory within a stub home directory to my git repo. In other words, drop your config.vdf at mytree.git/tools/steam/home/Library/Application Support/Steam/config/config.vdf, and when you call steamcmd, make sure to export HOME=pwd/tools/steam/home first. The pwd will substitute the current directory, which ought to be your build root.
My config.vdf lists its SentryFile as home/Library/Application Support/Steam/ssfn<somenumber> (that is, uses a relative pathname), and when I run steamcmd, I do so from the mytree.git/tools/steam directory. My goal was to not commit absolute pathnames, in case things moved.
Similar for my mytree.git/tools/steam/config/*.vdf files: relative pathnames. Overall, pathnames will be tricky, since there’re lots of opportunities for things to be different here.
In my case, I’m trying not to commit the Sentry file (or team username/password) to git, since they’re sensitive. My post-build script looks for environmental variables set in the build environment and relies on those.
Doing all that got it to upload the build to steam! Hooray!
…except that post-build runs before notarization happens, so the app that it uploads to Steam isn’t notarized. This unfortunately seems to be a deal-breaker. I think we need a post-notarization script to do the upload for mac targets.
I haven’t tried Windows targets at all. That was going to be Next for me.
Thanks for the detailed steps – I am have been trying the same approach. I haven’t got it to work quite yet for Windows (getting stopped on Steamguard), which may be due to me generating the ssfn file on Windows.
The notarization step issue for Mac could be a show-stopper though, as if it can’t work for Macs then this path is a dead-end for developers who support Mac (like me).
Could Unity please offer some insight on whether we can get the post-build script to run after notarization?
Hey, I managed to get this working without webhooks, only using UCB.
I put Steamwork’s “ContendBuilder” at the root of my projects so it gets added to my source control.
If you’re running your builds locally as well add these to your ignores
I then used this post build method
[PostProcessBuild]
private static void PostBuild(BuildTarget target, string pathToBuiltProject)
{
if (target != BuildTarget.StandaloneWindows64)
{
Debug.LogError($"Target not supported: {target}");
return;
}
var dir = pathToBuiltProject.Replace($"{Application.productName}.exe", "");
Debug.Log($"Root path: {Environment.CurrentDirectory}");
Debug.Log($"PathToBuiltProject: {pathToBuiltProject}");
Debug.Log($"Post build path: {dir}");
// remove things
Directory.Delete($"{dir}\\{Application.productName}_BurstDebugInformation_DoNotShip", true);
// copy files into steam builder content
var moveToDir = $"{Environment.CurrentDirectory}\\ContentBuilder\\content\\";
Directory.Delete(moveToDir, true);
// need copy all files and not use Directory.Move() as it will empty the build directory resulting in a failed build on UCB
CopyFilesRecursively(dir, moveToDir);
Debug.Log($"Moved: {dir} to {moveToDir}");
// start the steam upload
var StartInfo = new ProcessStartInfo
{
FileName = $"{Environment.CurrentDirectory}\\ContentBuilder\\builder\\steamcmd.exe",
CreateNoWindow = true,
RedirectStandardOutput = true,
RedirectStandardError = true,
UseShellExecute = false,
Arguments = $"+login {STEAM_USERNAME} {STEAM_PASSWORD} " +
$"+run_app_build \"{Environment.CurrentDirectory}\\ContentBuilder\\scripts\\app_build.vdf\" " +
"+quit"
};
using var steamBuildScript = Process.Start(StartInfo);
if (steamBuildScript == null)
{
Debug.LogError($"Process failed to start. {StartInfo.FileName}");
return;
}
Debug.Log($"Steam builder started: {steamBuildScript.StartInfo.FileName}");
steamBuildScript.WaitForExit();
var output = steamBuildScript.StandardOutput.ReadToEnd();
var error = steamBuildScript.StandardError.ReadToEnd();
if (!string.IsNullOrEmpty(error))
Debug.LogError($"Steam builder error: {error}");
else
Debug.Log($"Steam builder completed. {output}");
}
/// <summary>
/// Src: https://stackoverflow.com/questions/58744/copy-the-entire-contents-of-a-directory-in-c-sharp
/// </summary>
/// <param name="sourcePath"></param>
/// <param name="targetPath"></param>
private static void CopyFilesRecursively(string sourcePath, string targetPath)
{
// Now Create all of the directories
foreach (string dirPath in Directory.GetDirectories(sourcePath, "*", SearchOption.AllDirectories))
Directory.CreateDirectory(dirPath.Replace(sourcePath, targetPath));
// Copy all the files & Replaces any files with the same name
foreach (string newPath in Directory.GetFiles(sourcePath, "*.*",SearchOption.AllDirectories))
File.Copy(newPath, newPath.Replace(sourcePath, targetPath), true);
}
And it uploads to steam no worries.
I only have it set up for windows builds and I haven’t tested on UCB without running on a local machine first. There are probably some files that are cached to set up the environment.
I like this technique, but I feel like it’s a bad idea to have your Steam login info so easily accessible from your repository. If it’s a small or solo project, it might not be that big of a deal. If you could conceal that info somehow, that would be useful.
This is super cool though because it sounds like you don’t need a Jenkins server or AWS
You would put that info the in the UCB environment variables.
I’ve moved away from this approach now anyway because UCB is too slow and expensive I miss the old UCB
Sam Jones indicates that its possible to run a bash script. Okay, great. I added the example script into my GitHub repo, in a folder that makes sense.
I’ve confirmed this is the correct location in my windows path. However,
[2023-03-01T11:24:01.513Z] - 7.3.0.2.4.2 - INFO: Compile ran in '478.6752099' seconds
[warning] [2023-03-01T11:24:21.273Z] - 7.3.0.2.4.2 - WARN: ! Script configured, but not found at quantum_unity/ContentBuilder/bash-build.sh. Skipping.
… and I have no way of debugging what the state of the environment is in which the shell is executing… and the iteration times here are 30 minutes long. So I can’t just guess and check till it works in a reasonable time frame. Can some additional insight/documentation be added to Sam Jones post… such as two example paths for a win / mac url?
EDIT: This happened because the script path is relative to the unity project root, not the repository root.
But the result is:
[warning] [2023-03-01T21:39:25.396Z] - 7.3.0.2.4.2 - WARN: ! Script configured, but not found at bash-build.sh. Skipping.
its right there. in the root of the git repo.
EDIT: It cannot be in the root of the repo for this to work it must be in the folder above the Assets folder. This is not ideal, so you need to use relative paths to find the correct folder.
../tools_steam/post-build.sh ../some-script.sh
for the shell script to be found.
I also highly recommend people experimenting with build scripts to upload a completely empty unity project with no dependencies and put their post-build script in the pre-build script location to test your script until you know it roughly works.
"BUILD_PATH/p/quantum_unity/../tools_steam/bash-build.sh" "BUILD_PATH/p/quantum_unity/temp20230301-1367-13d1v22" "BUILD_PATH/p/.build/last/steamuploadertest" standalonewindows64
START
1| BUILD_PATH/p/quantum_unity/temp20230301-1367-13d1v22
2| BUILD_PATH/p/.build/last/steamuploadertest
3| standalonewindows64
Unity Exe: /cygdrive/c/Program Files/Unity/Editor/Unity.exe
Output Directory: BUILD_PATH/p/.build/last/steamuploadertest
Project Directory: BUILD_PATH/p/quantum_unity
Workspace: BUILD_PATH/p
BUILD_PATH/p\.build\last\steamuploadertest\SteamUploaderTest.exe
BUILD_PATH/p/quantum_unity/../tools_steam/bash-build.sh: line 26: =cp -rf BUILD_PATH/p/.build/last/steamuploadertest BUILD_PATH/p/tools_steam/ContentBuilder/content/content_windows: No such file or directory
BUILD_PATH/p/quantum_unity/../tools_steam/bash-build.sh: line 29: =BUILD_PATH/p/quantum_unity/../tools_steam/ContentBuilder/builder/steamcmd.exe +login USERNAME_ENV PASS_ENV +run_app_build BUILD_PATH/p/quantum_unity/../tools_steam/ContentBuilder/scripts/simple_app_build.vdf +quit: No such file or directory
Logs confirm the ContentBuilder is in the relative path:
But the empty folder for content doesn’t exist, even tho there was a .file in the repro to ensure it was created. Now modifying the bash script to create the appropriate folder… and hopefully try again.
Edit:
Now running the paths through cygpath to try to get it executing both the copy and the upload commands.
#!/bin/bash
#This is a sample that will dump the variables you need for steam builds
echo "START"
echo "1| $1" # unknown
echo "2| $2" # output folder
echo "3| $3" # buildtype
echo "Unity Exe: $UNITY_EXE"
echo "Output Directory: $OUTPUT_DIRECTORY"
echo "Project Directory: $PROJECT_DIRECTORY"
echo "Workspace: $WORKSPACE"
#PLAYER_PATH=$UNITY_PLAYER_PATH
#IF we are using a Windows Builder and using the path to pass it to a native Windows application we need to properly convert the cygwin player path to windows format
#if [[ "$BUILDER_OS" == "WINDOWS" ]]; then
#PLAYER_PATH=$(cygpath -wa "$UNITY_PLAYER_PATH")
#fi
#echo "$PLAYER_PATH"
if [[ "$BUILDER_OS" == "WINDOWS" ]]; then
outputPath="$WORKSPACE/tools_steam/ContentBuilder/content/content_windows"
outputPathCyg=$(cygpath -wa "$outputPath")
mkdir -p $outputPathCyg
echo "COPY Artifacts to $outputPathCyg"
cmd="cp -rf $OUTPUT_DIRECTORY $outputPathCyg"
echo "COPY Command: $cmd"
eval $cmd
pathToConfig="$WORKSPACE/tools_steam/ContentBuilder/scripts/simple_app_build.vdf"
pathToConfigCyg=$(cygpath -wa "$pathToConfig")
cmd2="$WORKSPACE/tools_steam/ContentBuilder/builder/steamcmd.exe +login $BUILD_USERNAME $BUILD_PASSWORD +run_app_build $pathToConfigCyg +quit"
echo "CMD2: $cmd2"
eval $cmd2
else
echo "You're a mac, mate."
fi
I miss the old UCB
