Diagnosing Very Slow Asset Database Refresh Issues (In 2020.1)

Unity Engine
,
1

Hello Unity Community!

I am trying to get some information on how I could solve an ongoing issue with my project.

Whenever I change a line of code in my project, unity takes just under 3 minutes to rebuild, and another minute and 30 or so to enter the game. This is obviously a serious issue for production and efficiency.

The reason, I suspect, is that I am working on a complex virtual world simulation, and as such, I have a large project folder.
My build size is 12 GB, which seems fine, but my project folder (Including the library, shaders, etc) is about 70 GBs. This is obviously huge, but I can’t really reduce this too much. Most of that data is needed. I could maybe shave off a few GB’s by removing old art assets or unused asset store assets, etc, but it’s still going to be a fairly large project.

I am currently using 2020.1, but I have upgraded this project from previous versions several times. The issue that I am having may be related to this:

So, I am wondering what I can do to reduce Asset Database refresh times?

I have tried adding ASMDEF’s to my project wherever I can, but it seems that most of the issues are not with compiling code, but with the actual Database refresh, which would seem to indicate that unity is refreshing my art assets, is that correct? Why would unity be refreshing art every time a code file is changed, the art is static?

About a year ago, when I first noticed this problem, I moved my unity project to an SSD (Which I bought specially for that purpose), and this did seem to help. I can’t remember how much, but it did make the editor refresh faster.

Could this issue be hardware related?

My computer is quite old, I am running a 6700K and a 980ti, with 32 GB’s RAM, and my main drive is a spinner. Unity, as I said, has its own SSD, but most of my dev programs, etc, are on the HDD.

I am currently buying parts for a new build, and I am getting an AMD 5950x, nVidia 3090, and the fastest 2TB M.2 SSD that I can get (Sabrent Rocket or Gigabyte Aorus, probably). I intend to run my OS, all of my dev programs, and Unity, from the same drive. Would this make a substantial difference?

Could there also be software issues? I have added the unity drive as an exception to my antivirus software, and I have verified that the issue occurs even when a backup is not running, although antivirus and backup programs are still running in the background, could they be conflicting with unity in some way?

When I restart my system completely, Unity often seems to rebuild a lot faster for the first few code changes, then goes back to taking 3 mins again. This seems to point to a software issue?

Is there anything else that I can do to diagnose or improve this issue, or, and this is my real worry, is the size of my project going to guarantee long refresh times regardless of the hardware or software that I am using?

I have tried profiling the editor, but it just confirms that the “Assetdatabase.refresh” is causing the issue. Is there a way to pin this down to a particular asset, of a particular file, that could be corrupt or badly written, that could be responsible?

Thanks for any advice!

2

Use the Editor Iteration Profiler to figure out what specifically is slow:

If it’s related to asset database refresh as you suspect, check if it makes a difference if you use Asset Database v2 (backup your project before activating it):

Enter Play Mode can get really slow as the number of objects in your scene grows ( see this ). Use Unity’s new Configurable Enter Play Mode options to turn off domain and scene reloads. However, this often requires code changes to work properly:

If it’s related to changing code, then it’s very likely the slow-down is caused by domain reloads. Various asset store packages have been identified to poorly implemented some “refresh” related things. The Editor Iteration Profiler should give you some clues in this case.

3

Check if you have Collaborate enabled and get rid of it: it’s known to make everything much slower.

If you created the project in 2020, asset database v2 is the default. It’s intentions are noble but currently there are several reports of it massively slowing down iteration times. Try using v1 and compare.

But ultimately, Unity is notorious for not scaling well when projects become large. It excels at prototyping and small projects, because those are the use cases Unity actually tests internally.

For anything beyond that, Unity relies entirely on highly detailed bug reports with attached repro projects to try and figure out how their engine is supposed to work on actual published games with thousands of assets.

1 Like
4

Thank you, those are excellent suggestions, it seems there is some hope for resolving this!

I will spend some time working on those, and post back with my results.

I do have a backup project that I can use.

I do suspect, as Peter77 said, that it is to do with changing code, so hopefully the editor iteration profiler should point me to a “smoking gun” asset store package that I can remove.

I didn’t create the project in 2020, it was an older project that I ported, so I don’t know which asset DB version I am using, I can definitely check that too.

5

I installed and ran the EIP, thanks for that suggestion!

It looks like I caught some interesting data.

The script compilation event took 96649 ms, which is about 90 seconds.

Most of the time was spent in “reloadassemblies” and “AssetDatabase.V2.CompileScripts”, as well as “AssemblyReload”, where, again, compilescripts took a lot of time.

It seems that a lot of time was spend in unetweaver::weave(), which is bizarre. I am building a multiplayer game, but why would so much time be spent in that function?

If I was to replace unet (Which I will need to do anyway), I could possibly save some time there?

These are some screenshots I took, showing the above data:

Imgur
Imgur
Imgur
Imgur
Imgur

6

Looks like the UNET package has some kind custom tasks performed on the post-compilation callback (looks like it’s using cecil to patch compiled binaries or something), so yeah, removing it should cut down 13 or so seconds.

7

That’s promising at least, but it would be problematic to remove UNET now.

8

I have been wondering about something else.

It seems that the AssetDatabase.Refresh runs twice, is this normal?

This image shows what I mean:

1368Ă—989 207 KB

In Assetimport>mainthread>editorloop>application.message

There is an “assetdatabase.refresh” which takes 37942 ms, or 37 seconds, a huge amount of time.

Then, in Assemblyreload>mainthread>editorloop>application.message

There is another assetdatabase.refresh, which lasts the exact same amount of time, to the millisecond (37269).

Why would unity be calling the assetdatabase refresh twice, once in assetimport, and once in assemblyreload? Is this expected behaviour?