Neverending IL2CPP build

I’m building on 2019.3 for Windows with IL2CPP. Everything seems to be working right when building, but it never finishes. I’ve left it overnight and still nothing. There are no errors in the editor. And looking at the Task Manager in Windows, it shows a Microsoft C-C++ Compiler Driver still running at around 14% CPU.

For comparison, the Mac IL2CPP builds successfully in about 10-20 minutes max.

What’s going on here? Is there a way to debug it?

Can you check which executable is taking that long? Is it cl.exe or link.exe? If it’s cl.exe, can you check the command line invocation to figure out which source file it’s compiling?

It’s cl.exe.

Here’s the last segment from the Editor.log:
Editor.log

Invoking UnityLinker with arguments: -out=“C:/Users/Brett Johnson/Documents/Stealth%20Adventure/Temp/StagingArea/Data/Managed/tempStrip” -x=“C:/Users/BRETTJ~1/AppData/Local/Temp/tmp184b0b39.tmp” -x=“C:/Users/Brett Johnson/Documents/Stealth%20Adventure/Temp/StagingArea/Data/Managed/TypesInScenes.xml” -d=“C:/Users/Brett Johnson/Documents/Stealth%20Adventure/Temp/StagingArea/Data/Managed” --include-unity-root-assembly=“C:/Users/Brett Johnson/Documents/Stealth%20Adventure/Temp/StagingArea/Data/Managed/Assembly-CSharp-firstpass.dll” --include-unity-root-assembly=“C:/Users/Brett Johnson/Documents/Stealth%20Adventure/Temp/StagingArea/Data/Managed/Assembly-CSharp.dll” --include-unity-root-assembly=“C:/Users/Brett Johnson/Documents/Stealth%20Adventure/Temp/StagingArea/Data/Managed/Unity.Polybrush.dll” --include-unity-root-assembly=“C:/Users/Brett Johnson/Documents/Stealth%20Adventure/Temp/StagingArea/Data/Managed/AwesomeTechnologies.TouchReactSystemPro.Runtime.dll” --include-unity-root-assembly=“C:/Users/Brett Johnson/Documents/Stealth%20Adventure/Temp/StagingArea/Data/Managed/UnityEngine.UI.dll” --include-unity-root-assembly=“C:/Users/Brett Johnson/Documents/Stealth%20Adventure/Temp/StagingArea/Data/Managed/NavMeshComponents.dll” --include-unity-root-assembly=“C:/Users/Brett Johnson/Documents/Stealth%20Adventure/Temp/StagingArea/Data/Managed/AwesomeTechnologies.VegetationStudioPro.Runtime.dll” --include-unity-root-assembly=“C:/Users/Brett Johnson/Documents/Stealth%20Adventure/Temp/StagingArea/Data/Managed/Unity.Postprocessing.Runtime.dll” --include-unity-root-assembly=“C:/Users/Brett Johnson/Documents/Stealth%20Adventure/Temp/StagingArea/Data/Managed/Unity.TextMeshPro.dll” --include-unity-root-assembly=“C:/Users/Brett Johnson/Documents/Stealth%20Adventure/Temp/StagingArea/Data/Managed/Rewired_Core.dll” --dotnetruntime=il2cpp --dotnetprofile=unityaot --use-editor-options --include-directory=“C:/Users/Brett Johnson/Documents/Stealth%20Adventure/Temp/StagingArea/Data/Managed” --rule-set=Conservative --editor-data-file=“C:/Users/Brett Johnson/Documents/Stealth%20Adventure/Temp/StagingArea/Data/Managed/EditorToUnityLinkerData.json” --platform=WindowsDesktop --engine-modules-asset-file=“C:/Program Files/Unity/Hub/Editor/2019.3.7f1/Editor/Data/PlaybackEngines/WindowsStandaloneSupport/Whitelists/…/modules.asset”
C:\Program Files\Unity\Hub\Editor\2019.3.7f1\Editor\Data\il2cpp\build/deploy/net471/UnityLinker.exe exited after 6740 ms.
Invoking il2cpp with arguments: --convert-to-cpp --emit-null-checks --enable-array-bounds-check --dotnetprofile=“unityaot” --compile-cpp --libil2cpp-static --platform=“WindowsDesktop” --architecture=“x64” --configuration=“Release” --outputpath=“C:\Users\Brett Johnson\Documents\Stealth%20Adventure\Temp/StagingArea/Data\Native\GameAssembly.dll” --cachedirectory=“C:\Users\Brett Johnson\Documents\Stealth%20Adventure\Assets..\Library/il2cpp_cache” --incremental-g-c-time-slice=3 --profiler-report --map-file-parser=“C:/Program Files/Unity/Hub/Editor/2019.3.7f1/Editor/Data/Tools/MapFileParser/MapFileParser.exe” --directory=“C:/Users/Brett Johnson/Documents/Stealth%20Adventure/Temp/StagingArea/Data/Managed” --generatedcppdir=“C:/Users/Brett Johnson/Documents/Stealth%20Adventure/Temp/StagingArea/Data/il2cppOutput”

I meant the command line invocation for cl.exe. You can find it via various tools like process explorer or task manager.

Oh gotcha. Here’s the command line invocation for cl.exe:

“C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.25.28610\bin\HostX64\x64\cl.exe” “C:\Users\Brett Johnson\Documents\Stealth%20Adventure\Temp\StagingArea\Data\il2cppOutput\Rewired_Core10.cpp” /nologo /c /bigobj /W3 /Zi /EHs /GR- /Gy /utf-8 /wd4102 /wd4800 /wd4056 /wd4190 /wd4723 /wd4467 /wd4503 /wd4996 /wd4200 /Ox /Oi /Oy- /GS- /Gw /GF /Zo /MT /DNET_4_0 /DUNITY_AOT /DIL2CPP_MONO_DEBUGGER_DISABLED /DGC_NOT_DLL /DRUNTIME_IL2CPP /DIL2CPP_ENABLE_WRITE_BARRIERS=1 /DIL2CPP_INCREMENTAL_TIME_SLICE=3 /D_WIN32 /DWIN32 /DWIN32_THREADS /D_WINDOWS /DWINDOWS /D_UNICODE /DUNICODE /D_CRT_SECURE_NO_WARNINGS /D_SCL_SECURE_NO_WARNINGS /D_WINSOCK_DEPRECATED_NO_WARNINGS /DNOMINMAX /D_NDEBUG /DNDEBUG /DWINDOWS_SDK_BUILD_VERSION=18362 /DWINAPI_FAMILY=WINAPI_FAMILY_DESKTOP_APP /I"C:\Program Files\Unity\Hub\Editor\2019.3.7f1\Editor\Data\il2cpp\libil2cpp" /I"C:\Program Files\Unity\Hub\Editor\2019.3.7f1\Editor\Data\il2cpp\libil2cpp" /I"C:\Program Files\Unity\Hub\Editor\2019.3.7f1\Editor\Data\il2cpp\external\bdwgc\include" /I"C:\Program Files\Unity\Hub\Editor\2019.3.7f1\Editor\Data\il2cpp\external\xxHash" /I"C:\Users\Brett Johnson\Documents\Stealth%20Adventure\Temp\StagingArea\Data\il2cppOutput" /I"C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.25.28610\include" /I"C:\Program Files (x86)\Windows Kits\10\Include\10.0.18362.0\shared" /I"C:\Program Files (x86)\Windows Kits\10\Include\10.0.18362.0\um" /I"C:\Program Files (x86)\Windows Kits\10\Include\10.0.18362.0\winrt" /I"C:\Program Files (x86)\Windows Kits\10\Include\10.0.18362.0\ucrt" /Fo"C:\Users\Brett Johnson\Documents\Stealth%20Adventure\Library\il2cpp_cache\8F6F44A13D25792CF4CCC2CB93E9B876.obj" /Fd"C:\Users\Brett Johnson\Documents\Stealth%20Adventure\Library\il2cpp_cache\8F6F44A13D25792CF4CCC2CB93E9B876.pdb"

1 Like

@guavaman I’m noticing it could be related to Rewired (“Rewired_Core10.cpp”). Any thoughts on whether it’s a bug or something I might be missing?

1 Like

Rewired version please?

The obfuscation is known to cause IL2CPP errors on Windows and Android builds. I manually test every single version of Rewired for each version of Unity (release and trial) with IL2CPP on Windows and Android by building before publishing because of this issue. But you could certainly be using some old version that wasn’t tested before publishing.

Incidentally, this issue has been traced to compiler bugs in both Visual Studio and Android Studio. Apparently the bug that causes it was fixed in VS 2019 in some service pack/update as I recall. I’ve also not seen the bug in recent versions of Android Studio.

1 Like

I’ve been running into the same issues since moving to a new computer last week. It will build on other machines just fine, but not my new home machine. I’ll get something like:
il2cpp.exe didn’t catch exception: Unity.IL2CPP.Building.BuilderFailedException: Rewired_Core10.cpp

So originally I thought it was an issue with the version of Rewired. When I updated the Asset to the latest version - then I get a build that just hangs forever. So far I’ve let it run for 4 hours with no luck.

I’ve tried repairing Visual Studio, and uninstalling / reinstalling Unity a few times.

Also - I should mention that Rewired does seem to work in an empty project with IL2CPP (though it still takes about 30+ mins to compile which seems odd for a completely empty project - its about a 40 second build without the asset). However, when I upgrade the actual project our team is working on, that’s when it hangs.

I’m using Unity 2019.3.7 and Rewired 1.1.30.3. I installed Visual Studio through the Unity Hub.

Like @NateReese77 , I moved the project from another computer (Mac), and I’ve also tried reinstalling everything.

@NateReese77 It definitely should not take 30 minutes to build Rewired in an empty project.
@brettj I just tested compiling to IL2CPP in 2019.3.7 with Rewired 1.1.30.3. I am using Visual Studio 2017 as I do not have VS 2019 installed. It is definitely not stalling on build. It only takes a couple of minutes to build, therefore it should not be related to the obfuscation issue.

Unless there is some kind of new bug in VS2019 that was not there in VS2017, I can’t see how either of you could be having this problem. Am I going to have to start doing test builds in every version of Unity and in every version of Visual Studio now? These compiler bugs with IL2CPP are really a huge problem and have exploded the amount of time it takes me to do an update.

1 Like

Yep. Installing VS 2019 16.5.1 and doing a build results in eternal build time. New VS compiler bug. I cannot possibly test all versions of Visual Studio. One person may have 16.5.1 installed while another has 16.1.0 installed.

Apparently IL2CPP chooses the latest version of VS to build to it finds installed, so I won’t be able to do tests with multiple versions of Visual Studio on the same machine. Even if I have it create a VS project, when I build that project from VS 2017, it invokes IL2CPP which in turn invokes cl.exe from VS 2019.

I have no choice but to set up a bunch of VMs, each with one version of Visual Studio 2015, 2017, and 2019 and Unity 2017, 2018, and 2019 so I can control what version I’m testing. I can only possibly test the latest version of VS of each branch. What a complete mess.

1 Like

The problem is even worse than I imagined. After extensive testing, even when IL2CPP is able to build properly with VS 2019 (different Rewired builds that don’t have the rare condition that causes the bug), the executable VS 2019 creates cannot run. Unity just freezes after the splash screen. This is regardless of the version of Unity I build with and regardless of the version or build of Rewired I build with.

I tried building various different versions of Rewired with 2018.4.20, 2019.1.0f2, and 2019.3.7f1. All 3 work flawlessly when building with Visual Studio 2017 (15.9.21 and 15.9.19). All 3 fail in the exact same way (freeze after splash screen) when built with Visual Studio 2019 (16.5.1). All 3 also work flawlessly when built with Mono. All I have to do is install VS 2019 16.5.1 on the system so it becomes the preferred compiler by IL2CPP and every build of any version of Rewired made with any version of Unity fails to run.

There’s a serious issue here with Visual Studio 2019, and I suspect it’s something in a recent update because both I and Unity (bug case) have tested Rewired IL2CPP on Visual Studio 2019 before without any issues. Now I can’t get it to run at all when built with VS 2019.

If the compiler is compiling the code without failing or going into an infinite loop, it should be executing when the app is launched, but it’s not, instead appearing to go into an infinite loop in the application. I’m going to have to ask Unity to get involved here to help me figure out what is going on.

1 Like

@guavaman Thank you for the quick and fast support! Sorry this is such a frustrating bug. I’ve gone ahead and uninstalled Visual Studio 2019 for the time being - however I actually ran into a few issues downgrading. The Windows SDK in the registry was set to an incorrect version - I followed this post and was just able to do a build without issues: Cannot build with ILC2PP in 2019.1.8f1

Going to do a few tests again with 2019 using this SDK and will let you if I have any success.

You shouldn’t have to be forced to use VS 2017. I’m going to keep poking at this from every angle I can think of to see if I can find some kind of workaround.

As for SDK problems after uninstalling VS 2019, the exact same thing happened to me. I was getting errors from IL2CPP indicating the Windows SDK was also uninstalled. I had to reinstall the SDK again separately from VS and I was able to get things building again.

2 Likes

Best I can guess upon tracing the disassembly is that compiler optimizations are throwing it into an infinite loop. Worst case scenario.

1 Like

Compiler bugs are so nasty to deal with :(. We can certainly dig into it and report the bug to Microsoft once you send us a repro.

1 Like

Thanks! Much appreciated.

This is going to be very time consuming and complicated. I’m going to have to do 3 separate bug reports. There are 3 cases of failure, each different:

  1. On some builds, the cl.exe build will hang forever.
  2. On some builds, cl.exe will finish, but the resulting code will hang forever on launch.
  3. New scenario – I just simplified the obfuscation code I believe is triggering this and that only lead to a completely different fail scenario. Now with this change, the il2cpp.exe build is failing with il2cpp.exe didn’t catch exception: Unity.IL2CPP.Building.BuilderFailedException.
2 Likes

Wow.

Thanks @guavaman for the quick and helpful support as always. And thanks @Tautvydas-Zilys for pushing on it from your side.

While we all wait for compiler fixes, should I downgrade to Visual Studio 2017?

I think that would be the quickest path to unblock yourself.