Build failed to compile but completed with success

I have just upgraded this project to Unity 6 after having the build machine spit out a build it fails to run with the log:

Mono path[0] = 'W:/SteamLibrary/steamapps/common/Pasture The Livestock Simulator/Pasture The Livestock Simulator_Data/Managed'
Mono config path = 'W:/SteamLibrary/steamapps/common/Pasture The Livestock Simulator/MonoBleedingEdge/etc'
The assembly mscorlib.dll was not found or could not be loaded.
It should have been installed in the `W:\SteamLibrary\steamapps\common\Pasture The Livestock Simulator\Pasture The Livestock Simulator_Data\Managed\mono\4.5\mscorlib.dll' directory.

Looking through the editor log for the build I found this:

Fatal error in Unity CIL Linker
Mono.Cecil.AssemblyResolutionException: Failed to resolve assembly: 'nunit.framework, Version=3.5.0.0, Culture=neutral, PublicKeyToken=2638cd05610744eb'

I found another discussion here about this error, however it doesn’t explain why the build didn’t fail. I have removed the references that namespace in my main assembly which doesn’t actually reference the assembly and that has fixed the error.

Note in 2022.3 everything works as expected so it looks like there are two issues here:

  • something changed in 6 with the assembly resolution
  • something changed in 6 with how builds fail

I have added this code the end of my build code which seems to fix the issue with the build not failing:

if ((success != BuildResult.Succeeded) && Application.isBatchMode)
{
    Debug.Log($"Build Failed");
    EditorApplication.Exit(1);
}

So either BuildPipeline.BuildPlayer isn’t setting the exit code, or something else it stomping it (nothing I can see in source).