Inconsistent realtime GI bake times with cache and in standalone builds

Lost-in-the-Garden · June 3, 2016, 7:44am

Hi everyone!

We are using the realtime GI system and we have a shared network drive that houses our GI cache. Its an ssd that we connect through a NAS on gigabit ethernet. If it works, it works quite swiftly…

However, realtime GI bake times are drastically inconsistent and we are struggling to understand why. Here comes the long list of issues/question

1.) Even If I have a GI solution in the editor, sometimes it is completely rebuilt, or at least it takes a very long time to build it, when creating a standalone build. Subsequent builds without changes to the scene, might be faster though, but it still fells inconsistent.

2.) We often have the issue, that one of us has a working GI solution, but we are unable to retrieve it from the cache, or at least, it again, takes a very long time to autobake it in the editor.

3.) The most consistent issue is that the cache seems incompatible between win and mac. One of us is working on a mac, and so far it seems that we could never use his cache entries. To isolate the problem, we set his cache to local, so the above cases, are all pure Windows 10 with unity 5.3.4p6.

4.) Sometimes changes that we would deem insignificant, i.e. not changing any of the meshes or meshes’ flags in the scene causes the cache to fail and prompts a complete rebake.

All these issues are very hard to isolate for us since its so many factors changing between each issue we have. We’d be curious to learn more about the GI and caching system, what causes an invalidation of the cache, how the hashes are computed, and what data is included in the hashes. This could help us improve our workflow to better control bake times.

KEngelstoft · June 8, 2016, 11:55am

If the cache doesn’t contain all the files needed (perhaps the cache size is too small), some steps of the baking process will have to be redone. It is hard to say if this is expected behavior without more any reproduction steps.
If the scene doesn’t have any changes at all and you are all on the same editor version, it should work. If it doesn’t something is wrong. Likely with how we hash the inputs (or a cache file is missing, e.g. antivirus ate some of the GI Cache files).
I have forwarded this to our QA.
Changes in lights or materials will affect the input to the baking process too. What kind of changes are you doing that shouldn’t cause a rebake?

I will update this section Unity - Manual: GI cache with some information about the hashing.

Lost-in-the-Garden · June 15, 2016, 9:54am

Thanks for the reply!

we set the cache limit to 200gig on all machines, currently cache utilisation is at about 100gig (unity is calculating right now…) so this should not be a problem.
We always work on the same editor version (currently 5.3.4p6). We could never really pinpoint what caused a rebake. It some times worked, and sometimes it didn’t. We never managed to isolate either case. Let me know if there is anything that we could try to replicate either state.
We have our own importer pipeline where we can go back and forth between Unity and Maya. It sometime seemed (again very fuzzy, maybe something else changed as well) that just doing a reimport of the level through our pipeline caused a re-bake, even though nothing changed since the last import.

I will try to provide some more info if we manage to reduce the case a bit.

a little addendum:
It’s also hard to judge weather data is coming from the cache or not or if it is recomputed locally. For example, I have a working GI solution for a small scene with blockout geometry in the editor, but still doing a build takes quite long and it seems that data is recomputed (i.e. “7/11 Lighttransport | 17 jobs”). In the performance view of the taskmanager I can see that data is pulled from the network eventually… In another scene, the computation takes pretty much no time, and the network is also not used. neither disk nor network…