I’m trying to convert a very large project (100gb project folder, 250+ scenes) to use the latest Asset Bundle system, in hopes of saving build time for builds with small changes between them.
Currently we use the old asset bundle system for a few types of assets, but the scenes are all built with the standard Unity build process. I am trying to convert these scenes (and their required assets) to asset bundles. I’ve ended up with a process that I think is very similar to how Unity builds scenes by default, with each scene in it’s own file (now asset bundle), and shared assets files to go with each scene (separate asset bundle from the scene.)
Now, building these all the first time seems to take 4-5 hours (a regular build for us is about 2.5 hours.) An incremental asset bundle rebuild with no, or very small changes, goes quickly (minutes.) However, during the initial build, and any other build with a decent number of changes, I’ve noticed while it is building the bundles, it seems to repeat a “Compiling Scripts…” step for every bundle (or at least every scene bundle.) This step in our project takes about 40-60 seconds unfortunately, and because we have so many scenes, that really adds up. The other parts of the asset bundle build steps seem to go very quickly, but then it get stucks on “Compiling Scripts…” again, etc. So in effect, it looks as if the build process is compiling the scripts ~250+ times when we do a full asset bundle rebuild. Is this really necessary? Is it actually compiling scripts in this step, or doing something else? I don’t see why they would need to be recompiled for every asset bundle.
We are on 5.3.3p3. I have tried various asset bundle build options (such as DisableWriteTypeTree and IgnoreTypeTreeChanges) and they all seem to behave the same in this respect.
Has anyone else noticed this, and found a way to avoid it, or is there perhaps something Unity could do to improve this?
I double checked, and it is indeed only asset bundles with scenes inside them that cause this “Compiling Scripts…” step. So, I can speed things up a bit by putting more scenes per asset bundle, but for dependency and other reasons, I would much prefer to have one scene per asset bundle.
I’ve also confirmed that this happens in a sample project, such as the Asset Bundle Manager sample from Unity. Just add extra scenes, and put them in their own bundle, and build asset bundles. You can see the Compiling Scripts step flash by… it goes quick in this case because script compilation is fast, but in a project like ours with 40+ second compile times… it is a real pain.
Thank you for the report. I was actually just made aware of this issue by an other customer 2 days ago. It is a bug script compilation should of course only happen once.
I don’t know if it’s a separate issue, but not only does it recompile all the scripts for each of the bundled scenes, it also rebuilds all the associated assets and that takes a huge amount of time. Any minor change to one asset, and it will rebuild all of the scenes (in our case it takes about 20 mins). It even does so for empty scenes.
It should be a separate issue, seems asset bundle incremental build totally breaks for you. Could you please open a bug for it with your project? Then we can have a look.
Yeah, you’re absolutely right. After further investigation, we have found out that it happens somewhat randomly and is not limited to scenes. It won’t rebuild the bundles when run immediately after the previous build, but sometimes just 15 minutes later Unity decides a bunch of bundles need to be rebuilt, often even empty scenes and small single object bundles that reference only a model, texture, material and a shader. Looking at Editor.log we can see the list of files included in each bundle and we’re sure none of them changed in any way (git also conifrms this).
I would file a bug, but our assets folder is currently 24 Gb, so I think attaching it is going to be problematic. We’ve tested on a clean project and everything worked fine, so the problem seems to be with our code. Now, is there a way to debug how Unity decides if the assets were changed? Even if there may not be an API for this, maybe we could reimplement the check ourselves (calculating hashes?) to try to catch the moment when this happens and for which files to further detect what causes this.
Has anyone found a workaround for this? Incremental building for asset bundles seemed to be working for me in 5.3.3p2 but after upgrading to 5.3.4p6 (and then 5.3.5) Unity is re-building untouched asset bundles. I can’t seem to find any rhyme or reason to it.
Any chance you are using the unity cache server? In some recent patches, I have noticed that having the cache server enabled during the build process causes Unity to reimport many things that it doesn’t need to. If you are using it, try disabling it.
The specific issue this thread is about is when a scene does actually need updating, and it needs to build a bundle with a scene in it, it re-compiles scripts for each such bundle.
Seems to be something I’m doing on my end. I wanted to detect asset bundle changes (when rebuilding) and then automatically increment an asset bundle version number. The approach I’m taking:
Compare the main platform manifest (in this case, iOS.manifest) checksum before and after calling BuildPipeline.BuildAssetBundles.
If the checksum changes, increment the version number that’s stored in an xml file (that’s also part of an “xml” bundle) and then call BuildPipeline.BuildAssetBundles again.
This seems to work well enough at first. If I rebuild the asset bundles again Unity reports no changes, as expected. However, if I switch to another app and then back to Unity (I’m running OSX) and then rebuild bundles, it rebuilds the main iOS bundle and the CRC value in iOS.manifest changes. I’m guessing that some sort of asset refresh occurs when the app is brought back into the foreground.
I’ve tried forcing a refresh of the two bundles/manifests (iOS, xml) before second call to BuildPipeline.BuildAssetBundles, but that doesn’t seem to help. I’m at a loss and honestly a bit frustrated that we can’t just easily make use of the seemingly unused “ManifestFileVersion” value. I’m sure there’s another/easier way to go about this so if anyone has suggestions I’m all ears.
The fix is unfortunately not a simple fix. Large refactoring of our player building had to be done.
But it getting closer to completion, but is probably still going to take some time to get ready for shipping.
