Unity Cache Server re-crunching textures on each machine connected to it

Unity Engine
1

Hi,

We have setup a local network cache server on which all our computers can successfully connect to.

The problem is that for the same project, synched with perforce on all machines, the whole project has to be imported again. I also think that multiple instances of the same textures are now stored on the cache server…

Is there anything that can cause this? Should we delete the library folder beforehand? I’ve read the docs and we are inline with the requirements to have it work, but it just does not.

2

Bump. This is really an annoying issue.

Further tests confirm the behavior: Each machine will crunch the texture again even if it exists on the Cache Server.

Is there anything I’m missing here?

3

Re-Bump.

This is a major issue costing my company several man-hours each week just for re-crunching things.

4

How often is this happening? Does it happen immediately with individual files? Or some weekly occurrence?

Here are some things to try:

  • Sanity check that each computer can access the server with the “test connection button” in the cache server setup dialog

  • Make sure everyone has the exact same settings on the “Build Settings” window. All of that appears to affect the hash used to look for a specific reference. windows X86 vs windows 64 are not the same and will re-import. That happens a lot esp with fresh installs which will default back to x86 for windows.

5

Well what’s the point of using a Cache Server if we can’t switch from one platform to another like in a non-cache-server environment?

Before moving to a Unity 5.6 setup with Cache Server, we never had to recrunch our textures when changing platforms. Windows x86, Windows x86_64, Mac and Linux did not trigger a recrunching of all the textures… Now, with the Cache server setup, it seems like Unity checks on the Cache server for the available texture, and if they are not present, it reimports, recrunches and reupload them all. It might be a cool workflow for a one-platform game, but it’s totally useless in a multi-platform environment.

We thought that the Cache Server would be kind of a network library folder, but it simply is not. There is no advantage in using a network cache when the local library folder offers a quicker way to switch platforms. I guess we’ll have to trash the whole thing for the moment unless there is some magic trick I don’t know about.

6

Unless I missed it, I didn’t read anywhere in your posts that this was due to platform switching, so I assumed that it just wasn’t working for you, that why the ensure same settings was suggested.

What version of unity were you using before 5.6?
What version is your cache server install?

I’m assuming since you’ve had it working for a while so you’ve set it up to have enough space. We’ve had frustrations with low cache expiration times, even just after a couple of days. I wish there was some more atomic control of how it works.

7

Since my initial post, we found out that:

  • Sometimes, when updating Unity (for instance from 5.6.0p1 to 5.6.0p4) this breaks the Cache compatibility, meaning that opening up a project on a new machine would trigger a full reimport / recrunching instead of downloading the assets from the Cache.
  • Furthermore, switching platforms on one machine does the same; it reimports all of the assets if no other devs had that platform setup on his machine. Meaning that switching from x86 to x64 will recrunch all the textures and take space on our server.

This is worse than a non-cache workflow, in every single possible way.

For our gold master, we were using 5.5 and shipped the game on Windows, Mac, Linux, Xbox One and PS4 using one build machine that switched from one platform to another in between each build.

It worked well except that each time Unity pushes a major update, for instance when we moved to 5.6, it required all the machines to rechrunch the textures, which is a 12 hours long process, on each machine.

We wanted to avoid that, so we though of using our build machine to import the data once on the new update, and then have all the other machines grab the data from the server, therefore saving a truck load of time.

However, the reality is that while we avoid the basic x64 reimport on all the machines, switching platform will cause a recrunching of all the textures. With the 5 platforms we support, this is plain stupid. If we push that work on the build machine, it would require 60 hours of reimport on each Unity version, including mere patches.