New Unity 6.6 ContentDirectory GOOD and BAD Q&A

Unity 6.6 announced new recommended way to work with assets in runtime is ContentDIrectory

I have tested them and this is what i found Good and Bad about current design and how it actually works.

What is Good

  1. Very simple API to use. 100 times simpler than Addressables! Just Build → Register → Load → Unload that’s it.
  2. Uncompressed form is really just Directory with builded assets and manifest you can check any dependency graph and any other assumption after actual build not before the one
  3. Deduplication is built-in no need to do anything
  4. Incremental rebuilds rebuild only what changed. Fast iterations
  5. Easy to migrate from any good OnDemand Loading solution (not bloated one). For my Flexy.AssetRefs it is just one config change in project and everything work from ContentDirectories
  6. Easy to see what assets become part of build and size of each. Easy to analyze build size
  7. Out of The box Ready for Different AppStores delta patch updates like steam, PlayStore, AppStore etc
  8. With simple trick it is easy to deliver simple individual assets of dictionary on demand. replace asset with empty file with same name and deliver it when you are ready just replace it in directory and load again. It works with asset dependencies too.
  9. Sources for build is not Entire files on disk but individual Assets in them and every asset result in separate file in ContentDirectory. It is amazing :slight_smile:
  10. Unloading asset Just Works without any hoops. Load prefab with material and heavy texture on it. After some time unload prefab and underlying material and texture will be unloaded too. Unlike Addressables and bundles that actually unload only if entire bundle will be unloaded! Amazing :slight_smile:
  11. If some texture is not in DIrectory loading Prefab with reference to texture fail only fail on texture all other will be loaded as expected. An unloading prefab than replacing texture with correct one and loading again will load prefab with texture. Loading Just Works :slight_smile: I really like this new system :slight_smile:

What is Bad

  1. To deliver assets on demand we need to replace them with empty files. While it is easy to do, it is weird step that must not be there. System must init based on files in manifest. Not on actual files in directory.
  2. Asset file hold only partial info about asset not all of it. Some metadata live elsewhere and replacing texture with same one but imported with another srgb mode do nothing. New build see new texture version but old one see old one even after replacing. It is looks like a BUG
  3. You can replace asset inside Directory with another one but only if it have exactly same size in bytes for texture it is mean same dimensions and compression and may be everything but some pixel changes. This is very restrictive and fell as bad design that forbid small Patches and Game Mods Support in the first place. Size of asset and other actual metadata must be part of ContentFile not stored elsewhere. So we can tweak or replace asset of a game by just changing files with new ones. Think about HD Mod that replaces all textures with bigger and nicer ones.

Currently Unknown

  1. How to build few content archives from single ContentDirectory (need deep dive into addresables to see how they do it) and test how it works overall
  2. How to create new assets into new ContentDirectory that will replace old ones in build by loading second content directory. Say new Hat styles for Hallowing without chaging original Hats. Like build new hat prefab and make system to load it instead of old asset on GUID path. There is almost no information for this.

Do anyone know or tested something more or have another questions/usecases to ask/test about ConentDirectories?

7 Likes

Hey, as someone who builds web stuff i had heck of a job with addressables, so as a result, ive not tried this if you can give a quick example of how you “build, register, load, unload” id love you for it

I have use my package Flexy.AssetRefs for managing on demand assets instead of addressables because they was bad from the beginning. That was actually the reason I have created Flexy.Assetrefs.

They collect all Refs for ondemand build and than other tasks convert them to something that can be loaded in runtime.

By default they prepare and load through Resources
So the only thing I have changed is build ContentDirectory instead of refs from resources and replace loader to ContentDirectory API instead of Resources API

so this was not ground up work. Just few tweaks to the system to support ContentDirectory

I almost have done with it and tomorrow will push new version to Github with ContentDirectory support

main pieces is:

  • Create ScriptableObject with array of references in my case map AssetRefs to Loadable
[SerializeField] AssetRef[]       _assetRefs       = Array.Empty<AssetRef>();
[SerializeField] Loadable<Object>[] _assetLoadables       = Array.Empty<Loadable<Object>>();
[SerializeField] SceneRef[]       _sceneRefs       = Array.Empty<SceneRef>();
[SerializeField] LoadableSceneId[]  _sceneLoadables       = Array.Empty<LoadableSceneId>();
[SerializeField] String[]         _sceneNames          = Array.Empty<String>();
  • Pass this one SO to BuildPipeline.BuildContentDirectory

  • On [RuntimeInitializeOnLoadMethod(RuntimeInitializeLoadType.BeforeSceneLoad)] spawn prefab with ContentDIrectory Service that will register contentdirectory with ContentLoadManager.RegisterContentDirectory(contentDirectoryPath); and load my SO with _contentCatalogs = ontentLoadManager.GetRootAssets<ContentDirectoryCatalog>(_contentDirectory);

  • Than pass any asset loading through this service

public override       T?          LoadAssetSync<T>   ( AssetRef @ref ) where T : class  
{
    var asset = GetContentLoadable(@ref).Load(); // Get corresponding Loadable<T> fro AssetRef and Load
    return GetSpecializedAsset<T>(asset); // Make it able to load Components on prefabs directly in other cases just return asset as is
}

More detailed flow you can see when I push all this into Flexy.AssetRefs tomorrow

3 Likes

Released AssetRefs 6.6 on github with support for ContentDirectories

Now you can see exact implementation of it in files:

Hi JesOb,

Thanks for taking the time to share all this great feedback! We’ve spent a lot of time designing and building this new pipeline so it’s great to see that you have identified a lot of architectural improvements. In 6.6, we are mainly targeting static ContentDirectories that ship with the player. In future releases, we will add more features to better enable dynamic content. For the time being, I’ll share a few of the implementation details that I think you might find useful based on your comments

To deliver assets on demand we need to replace them with empty files. While it is easy to do, it is weird step that must not be there. System must init based on files in manifest. Not on actual files in directory.

When you register a ContentDirectory, the available files are enumerated to create a snapshot in order to reduce file look ups and support the optional file extension. When archiving is enabled, we mount all the archives and then perform a similar enumeration. We are exploring a hook point where users can provide the raw data through the virtual file system instead of requiring files in specific folders.

To address your #2 and #3 I will try to explain a bit more of how these files are laid out.

All unity Object types go into ContentFiles and as you noticed we break each asset into 1 or more ContentFile. The ContentFile may reference other artifacts such as the raw texture data, or some other raw binary data. This binary data is always referenced from the content file via a x3 hash of the content it is referencing. ContentFiles can also reference each other, but this is not a content hash reference, it uses IDs derived from the asset source. The manifest file is responsible for mapping the IDs to the content hashes of the content files.

If I understand correctly, it sounds like you are trying to swap out the raw binary data of a texture, but keep the filename the same. This could cause problems because the texture’s content file is built with the expectation of specific data as you are seeing.

How to build few content archives from single ContentDirectory (need deep dive into addresables to see how they do it) and test how it works overall

In the 6.6 release, BuildContentDirectory can take an archiving flag, which will put all artifacts into a single archive. However, as you noticed, Addressables does its own archiving after the build. Addressables loads the ContentLayout.json file, which includes all reference information about artifacts, and then assigns them to archives manually. The archives must follow a strict naming convention in order for them to be properly mounted during content directory registration. For the time being, I would have a look at the Addressables code to learn more.

Multiple Content Directories Behavior

Here’s some extra information about the behavior of registering multiple content directories in 6.6:

  • Each content directory has a list of Loadables

  • When you load a Loadable, the ContentDirectories are iterated and the first match encountered (most recently registered directory) is chosen.

  • Direct references of the Object loaded via Loadable will always be from the Content Directory the Loadable is from and direct references will never cross into another Content Directory.

  • If the same asset is built into two content directories and directly referenced by a loaded Loadable in each, the referenced asset will be loaded into memory twice, once for each content directory.

You can think of the ContentDirectory as a mostly isolated unit where only the Loadable reference can cross between them. So generally we have recommended trying to have fewer, larger ContentDirectories instead of many, small ones. Addressables uses a single ContentDirectory for the entire build. This behavior could be changed in future releases to better enable sharing of assets between ContentDirectories.

2 Likes

Thanks JesOb for being an early adopter of the new build system and all the information and feedback you have posted here.

There is some detail of the expected naming convention here:
UnityDataTools contentdirectory-format

1 Like

Thanks for clarifications :slight_smile:

I will continue to investigate how to support

  • CDN packs
  • DLCs
  • Mods
  • Event Content Overrides

With this new system :slight_smile: and share my findings here :slight_smile:

1 Like

Please add CancellationToken support to LoadAsync and LoadSceneAsync, similar to Object.InstantiateAsync.

While the concept of ContentDirectory seems really great, without cancellation support, it is hard to manage loading efficiency and optimize large-scale games where asset/scene loading needs to be canceled frequently (which was one of the pain points with traditional AssetBundles / Addressables).

1 Like