"compiling cpp for libil2cpp" takes much longer suddenly.

Hey,

I’ve been working on my project for a few years, using IL2CPP just fine. Build times for Android have always taken about 30secs on average. Though now suddenly all builds take up to 20mins, and 99% of the time is spend on “compiling cpp for libil2cpp”.

And my CPU goes through the roof when it goes, bogs up my computer!

What I’ve tried based on other threads with roughly the same topics:

  • Disabled BitDefender
  • Deleted my project library
  • Reinstalled Unity
  • Reinstalled my IDE (Rider).

It does seem to be random though, very occasionally the build time will be 30 secs. Then the rest of the time about 15-20mins. And I’m not doing anything different, I’ve done some experiments to check before building each time, like editing some code in various scripts, editing prefabs, scenes etc… Though it just seems to be quite random as to when it needs to do the “compiling cpp for libil2cpp” for 20mins.

In my editor logs, all the files being compiled seem to be the same between my 30 sec builds and 20mins builds. The difference is the time spent on certain files. Basically all the files in “C_Android_arm64 Library/Bee/artifacts/Android/d8kzr” (like 50 files or so) take about 0-2secs each in the 30 secs build.
But in the 20mins build they average from 10secs to 180 secs!

Some of the threads about IL2CPP are years old, wondering if there’s any clear solutions since then to solve builds taking so long? Especially that mine seems to be random, and out of the blue. Like… sometimes it uses the cache and othertimes it doesn’t?

And to clarify again, this has never been an issue over the years. No matter if I make new scenes, make new scripts, import assets etc… The build time would barely take a few minutes the first time maybe, but then 30 secs most of the time.

Thanks!

9245976--1292547--Unity_zXnzQpWDRr.png

1 Like

Which version of Unity are you using?

2022.3.7f1

I had a look at the editor log in realtime for once. It really does the usual for the first 30secs, then just gets stuck on all the “C_Android_arm64 Library/Bee/artifacts/Android/d8kzr/” files for the next 20mins, before getting to the build gradle part which takes about 10secs. Then it’s complete.

It doesn’t freeze or anything, it just goes through a huge amount of these .o files while pretty much using 100% of my CPU (i9-10850k), I can barely do anything else on my computer:

example:

1647/1673 18s] C_Android_arm64 Library/Bee/artifacts/Android/d8kzr/vm8y7zb3ae6s.o
[1648/1673 3s] C_Android_arm64 Library/Bee/artifacts/Android/d8kzr/1jqxnvpgc7em.o
[1649/1673 17s] C_Android_arm64 Library/Bee/artifacts/Android/d8kzr/a3bskqtllyxi.o
[1650/1673 28s] C_Android_arm64 Library/Bee/artifacts/Android/d8kzr/urefy0vhg78i.o
[1651/1673 26s] C_Android_arm64 Library/Bee/artifacts/Android/d8kzr/qv10oc9wnpz5.o
[1652/1673 15s] C_Android_arm64 Library/Bee/artifacts/Android/d8kzr/a97orr79qv03.o
[1653/1673 19s] C_Android_arm64 Library/Bee/artifacts/Android/d8kzr/nitceoz6wb6z.o
[1654/1673 22s] C_Android_arm64 Library/Bee/artifacts/Android/d8kzr/odnwnnfsri39.o
[1655/1673 15s] C_Android_arm64 Library/Bee/artifacts/Android/d8kzr/a74asqx57pu9.o
[1656/1673 18s] C_Android_arm64 Library/Bee/artifacts/Android/d8kzr/47dwone1gz66.o
[1657/1673 14s] C_Android_arm64 Library/Bee/artifacts/Android/d8kzr/8nhx2jxqbfo4.o
[1658/1673 17s] C_Android_arm64 Library/Bee/artifacts/Android/d8kzr/y0hwkm1uahhv.o
[1659/1673 30s] C_Android_arm64 Library/Bee/artifacts/Android/d8kzr/yh5nbv70lt9v.o
[1660/1673 20s] C_Android_arm64 Library/Bee/artifacts/Android/d8kzr/1nsqeziuzatc.o
[1661/1673 37s] C_Android_arm64 Library/Bee/artifacts/Android/d8kzr/zgevjw988vkf.o
[1662/1673 16s] C_Android_arm64 Library/Bee/artifacts/Android/d8kzr/fslseevlmxb4.o
[1663/1673 28s] C_Android_arm64 Library/Bee/artifacts/Android/d8kzr/p7zzlbdkr3o1.o
[1664/1673 1s] Link_Android_arm64 Library/Bee/artifacts/Android/d8kzr/libil2cpp.so

Hope that helps a bit! Thanks for the assist.

I’m using the same version and it takes a lot of time to do ‘compiling cpp for libil2cpp’. Hope you can look into this. Thank you in advance!

Thanks for the information! In Unity 2022.3.7, the Unity player build system will emit a file named buildreport.json in the Library/Bee directory of the project. This is a Chrome tracing file (see Chrome Tracing as Profiler Frontend · Aras' website for details).

If you can find this file, load it up in Chrome - it should break down the timing of all parts of the build process, and help you to narrow down what the specific cause is.

My guess is that the linker from the Android NDK is the bottleneck now, but I might be wrong. Let me know what you find in this file, and we can go from there.

1 Like

Awesome! That’s great, never saw that before.

So I loaded up a build report that took 10mins to complete. It actually shows exactly how I described it before:
The usual process happens in less than a minute, then spends the rest of the time of something fishy. Then gradle build quickly before ended.

I’ve attached screenshots of the report, can see the majority of time is spent in “UpdateFromStreamAsync”, and then inside that there’s a huge amount of “ReadAsync”. Not entirely sure what any of this means though, so any guidance here would be appreciated.

If your guess is correct though, is there a more stable version of Android NDK I should be using if not the one that comes with Unity?

Thanks!

9251418--1293597--chrome_V6MbX9hknE.png
9251418--1293600--chrome_Rb516IBSay.png

Sharing this part of the log as well. As this seems to be what we see as well in the build log. The " C_Android_arm64 " which I meantioned previous seems to be hogging up all the power for 10mins.

Though based onlooking at the log, and the loading bar together I can see there seems to be some kind of repetition somewhere that’s causing this? For example, when I make bigger changes to the project, it’ll take 3mins or less to build previously.

Here we can see, it gets through a first phase of compiling at about 3mins, which is normal. And would usually finish then with a build time of 3mins. But instead, even the loading bar returns from almost finished back to the start. And starts all over again. But this time round for the next 7 mins throttles my CPU and then finishes the build after that.

This is reflected in the loading bar, the build log and the build report.

It’s not like the entire process is just overall longer, it looks like it’s trying to do something after it should be complete. Though very occasionally it doesn’t. For example, this morning I logged in, pulled changes from git, and the project built in under a minute! Hurrah! But then… I tried to build a second time after a tiny bit of work, and it takes 10mins. And now all builds take 10-20mins, even when I don’t do anything.

@Hyperactive : Can you send me the full buildreport.json file from a long build (maybe via direct message)? I’d love to have a look and see if anything jumps out at me.

Done! Thanks.

Thanks, I’ll follow up on the private message, then report back here to provide any public details for others.

What’s the issue? @JoshPeterson any updates? thank you :smile:

Sorry, I forgot to follow up! It looks like the build cache is unexpectedly being cleared, but we cannot tell why.

2 Likes

hi, exact issue on my side as well. @JoshPeterson did you and the team find any solutions to this? can this be caused by some build pipeline settings? (maybe 2 options that shouldn’t be used together are set to be used together?)

Unfortunately, we did not track down the cause of this issue. It happens because the build cache was cleared, leading to a full rebuild when we should need only an incremental build. But as to why that build cache was cleared - we are unsure.

If you can provide any other information or insight about why this is happening, that would be appreciated!

2 Likes

Hi @JoshPeterson

may this be related to UCVS? im not sure excatly but might synchronising work cause the build cache to be reset or modifed in a way forcing unity to full build rather than incremental?

if that maybe the problem, Is it a bug? or is there any good practice that can be done in UCVS project to prevent this?

I’m not familiar with UCVS, so I can’t offer specific recommendations. However, we don’t want a version control system to be involved with an build cache artifacts. Those can be generated from source code that is under version control, but the Unity build system will assume that build cache artifacts are not managed by some external actor.

I think we’re starting to see this after an upgrade to Unity 2022.3.19f1 from Unity 2021. @JoshPeterson

Has Unity found any cause for this yet? I have a coworker who had their long build times fixed by using a fresh Windows 11 install partition, but it seems to have reappeared after a day or two in that partition.

Where are these build caches located?