Content Directory output size

Asset & Content Management Content Directories
,
1

As most, I read about the new content directories and after reading the post about all the benefits it should have, I was excited to try it out!

Currently we make heavy use of assetbundles to distribute our content. We have a custom hardware solution for our customers, and we provide new content (games) through our own distribution solution. Essentially it boils down to the following:

We make a game inside the unity editor and build all the behaviour into a .dll file. All the content for that particular game is then build into an assetbundle. Through our distribution solution we update the .dll on the machine for our customers and we push the new content (assetbundle) on their machine. Our platform knows how, when and what to load to play the game.

We distribute new content and updates over the air to our customers (with a data limit) so reducing size would be very beneficial.

So I figured that the overall size and performance benefits would be a good thing to explore. As a test, I used one of our latest games. We always have a “one point of entry”, which is a scriptable object that serves as the main point of entry for the game, which would be perfect for content directories.

When I build the asset bundle, the size is about 110MB, but when I build the content directory, I end up with 990MB. I figured the assetbundles are probably compressed and content directories aren’t. Initially I built the content directory into an archive, but that seemingly doesn’t compress, just packs it up.

When I zip the content directory myself, it’s about 120MB, so that is getting closer.

I inspected the build report and next to some stray assets, it seems to be generally what I expect. I did see some .cs files and the .dll file I mentioned earlier, which I didn’t expect to be there.

The folder containing all the (raw) content is about 230MB in size, so I am a bit puzzled on why the content directory ends up so much bigger.

I currently tested it out as a direct replacement, so I didn’t use any of the loadable logic yet. Would that make any difference? Am I wrong in assuming that the file size would at least be comparable to that of an assetbundle without any changes?

2

It seems to all boil down to compression. Assetbundles are compressed by default, while content directories aren’t, which makes sense I guess!

I added compression to the BuildContetDirectoryParameters and now it produces a compressed archive. It’s slightly bigger, but that might be due to some additional files/info I guess.

3

Hey @JoeySpringlab , I’m glad you figured out the compression option already.

I did see some .cs files and the .dll file I mentioned earlier, which I didn’t expect to be there.

It is normal that .cs files would show up in the build report, that just means that there are references to objects coming out of those .cs files (but the actual code is not included in the build). I haven’t personally seen a .dll showing up, but on consideration that might be expected if you have MonoBehaviours or ScriptableObjects that are in a pre-built assembly rather than being build from source. Again the .dll itself won’t be part of the content directory build.

If you want to pinpoint more details of the different between your AssetBundle and ContentDirectory build then UnityDataTools can show you the archive compression ratio, the content of the compressed content directory, the presence of typetrees, and lots of other information related to build size and build content. If you have an AI agent you could install the tool and ask the agent to compare the builds and explain the size difference. From my experience it should be able to do a good job pinpointing exactly what is going on. Otherwise even without an agent it can be quite educational to learn more about the build output and differences between the build output if you use it directly.

4

Actually i was thinking that a quite simple explaination for a somewhat larger content directory build could be the choice of compression algorithm. AssetBundles default to LZMA, which is larger compression but not efficient for loading. Content Directories use chunk based compression LZ4/LZ4HC, which will tend to be larger overall but allows us to selectively load content without decompressing the entire archive into memory. UnityDataTool has an archive command that can be used to see which compression algorithm is being used.

5

Thanks @AndrewSkow!

The tool looks very handy, will gave it a try.

Ah right, so even when compressed, it would mean it would still be more efficient in loading? Compared to non-compressed content directories that is. I imagine some compression would cause some overhead in loading right?

Additional question when it comes to distribution. Am I right to assume that if we have a system in place to deliver remote content it would work? So in our system we are doing the following with assetbundles, but would replace it with content directories:

  1. Client downloads new game (asset bundle + some meta data)
  2. Client copies downloaded data to streaming assets
  3. When the user starts that game, we load that specific asset bundle and run the game
  4. When the game is finished, unload the assetbundle

I assume this would also work with content directories, providing we adjust the code to load content directories.

I do wonder if this will still provide any benefit over assetbundles when it comes to loading speed and memory usage, but that is something I could measure.

6

The compression explanation clears up the size difference. AssetBundles using LZMA while Content Directories use chunk-based LZ4/LZ4HC is an important distinction, especially when the goal is selective loading rather than simply minimizing the archive size.

I’d also use UnityDataTools to compare the two builds directly, particularly the compression ratio and archive contents. That should make it much easier to identify whether the remaining size difference comes from compression, typetrees, or additional build metadata.

7

I agree with @jhonwalker000091muvcamy5 's reply. And to get into a bit of the specific questions:

I imagine some compression would cause some overhead in loading right?

LZ4/LZ4HC can be decompressed very efficiently, so while there is some cost I don’t think it will be a concern for modern hardware. Compressed data is smaller on disk so there is less IO cost to read it. So there are some trade offs but both uncompressed and LZ4 content directories should work well. When using uncompressed it is still a good idea to put them inside an archive file if the build is large, to reduce the number of individual files.

For the distribution question - yes what you describe should work. So long as you download the entire content directory and locate it into StreamingAssets then you can register it.

8

Thanks for the insights @AndrewSkow and @jhonwalker000091muvcamy5, very helpful!

I will have to run some tests and analyze that to see if/how much of the benefits will be for our use case, but at the very least I assume we will be more future ready.