Problem with sticky catalogs? Why is catalog referring to old bundles and not new ones?

[Endahs]

After losing yet another 12+ hours failing to troubleshoot Addressables issues, I’m humbly begging for help from the community and the Addressables Team. I am using Addressables 1.19.19, and Unity 2021.3.21f. I use “Use Existing Build” Play Mode option, always.

My issue: Updating local assets and rebuilding bundles creates new bundles (great!), and those bundles are saved to a local Apache WWW directory on my machine (also great!), however Addressables continues to refer to the old bundles without getting the new ones (ugh). If you delete the old bundles and leave the new ones you get RemoteProviderExceptions because Addressables continues to look for the old bundles, and won’t get the new ones (double ugh).

I have have several “CORE” bundles that are included with the binary that goes to the store, and these are always saved with the Player. They are saved in /ProjectName/Library/com.unity.addressables and Addressables is supposed to automatically move these into the Unity project at build time. I have many more “DLC” bundles that are saved to a local Apache WWW folder on my machine (used during Dev/Test), and later moved to a Cloud storage bucket (for Prod). The problem of Addressables failing to find newly changed bundles is happening to BOTH types of bundles, whether they are included with the app/player or they are saved/loaded remotely.

Note that I am using Player Version Override to always keep the same catalog name. The idea being that if a remote/Cloud catalog of the same name is updated, the player will download that and become aware of any bundles that have been modified since the app was published to store. This may be important later?

Simplest test case:

• Have some CORE files that are bundled and always included in binary that goes to store, never changes remotely

• Build Addressables: COREBUNDLE1 is built, CATALOG1 is built - works! CATALOG1 knows about COREBUNDLE1

• Change some CORE files

• Rebuild Addressables: COREBUNDLE2 is built, COREBUNDLE1 is deleted, CATALOG1 is rebuilt - fail! CATALOG1 is still looking for COREBUNDLE1!

Things I have tried without success:

• Deleting all the bundles (old and new) and rebuilding them. New bundles are created but Addressables then throws errors saying it can’t find the old bundle names.

• Deleting the TEMP folder in Unity. No change.

• Deleting the local user cache (users/AppData/LocalLow/Unity/CompanyNameAppName/ and users/AppData/LocalLow/CompanyName/AppName/com.unity.addressables). This causes the user to be re-prompted to download the bundles but then you get errors stating that Addressables can’t find the old bundles (even if new ones exist!).

• Using the Addressables menu in the Editor to Clean Build → All, and Clean Build → Build Pipeline. No fix. Addressables still tries to load the old bundles even if they are deleted and there are new ones to download.

• Deleting the the /UnityProject/Library/com.unity.addressables/ and then rebuilding bundles. This can get things moving again sometimes, especially with CORE bundles. It does not work with DLC bundles (the ones remotely saved/loaded).

• Changing the “Player Version Override” value to something new and rebuilding all bundles. This gets me past the issue, however this means I will need to re-publish everything to the store again and defeats the purpose of Addressables. If I then change the value of the catalog name back to the previous name it will start failing again.

The problem?: From my understanding and hours I’ve spent learning how Addressables is supposed to work, this may be a catalog issue? For some reason the client seems to be sticking with an old catalog and is not getting a new catalog, or perhaps the new catalog isn’t being created? This is where I need some help. I can see that the catalog file is being re-written on disk when I rebuild everything, yet whatever catalog is being loaded in-game still refers to the old bundles.

Questions:

• Is there a way to debug out what catalog Addressables is using?

• Is there a way to verify what bundles the loaded catalog is aware of?

• Shouldn’t rebuilding with “Player Version Override” on and without changing the catalog name rebuild the catalog into the same location, but refer to the new bundles? Why is this rebuilt catalog refering to old/deleted bundles instead of the new ones?

• Do you see anything I’m doing wrong here? I’m pulling my hair out here. So close to forking my code and just stripping all Addressables out

Attached screenshots of my Addressables settings:

My 3 sets of paths. Core_Store is always included in binary. DLC_Localhost is for Dev/Test. DLC_Cloud is for Prod.

Addressables settings.

CORE bundle settings that are always included with binary and never change remotely.

DLC bundle that is updated remotely.

[timtunity3d]

Are you using Build->“Update A Previous Build” when you want to update content? Are you uploading the new catalog json and hash files to your web server?

[Endahs]

@timtunity3d

Are you using Build->“Update A Previous Build” when you want to update content? Are you uploading the new catalog json and hash files to your web server?

Hi @timtunity3d , thanks for the reply.

No, I am not using anything to do with updating a previous build and so should have nothing to do with addressables_content_state.bin. If a file changes in a bundle the entire bundle is re-created. This is what I use:

// Clean + rebuild Asset Bundle...
AddressableAssetSettings.CleanPlayerContent( AddressableAssetSettingsDefaultObject.Settings.ActivePlayerDataBuilder );
AddressableAssetSettings.BuildPlayerContent();

Yes I am uploading the new catalog automatically using the “Build Remote Catalog” option to automatically generate a catalog in my local WWW folder. But isn’t this irrelevant?

The issue is that when I modify the “core” bundle (a bundle that is always included locally in the build that goes to the store and never updates from remote) and rebuild it + rebuild the catalog, when I enter play mode Addressables attempts to load the old “core” bundle and fails.

It’s easy to re-create. Start with an existing “core” bundle called “core_assets_all_[hash].bundle” in “Library/com.unity.addressables/aa/Android/Android”. Let’s call this “BundleOLD” for simplicity. I can open the game fine and this bundle is loaded. However, I change a file in the “core” bundle and rebuild, Addressables will delete “BundleOLD” and create a new bundle (let’s call it “BundleNEW”). When I try running the game again Addressables will throw exceptions because it can’t find “BundleOLD”. This is because Addressables deleted it and created “BundleNEW” (expected), however the local catalog is still looking for old the “BundleOLD” (unexpected).

If we ignore the DLC bundles on my local WWW, or the DLC bundles on my Cloud bucket, shouldn’t a local bundle that is designated to build and load locally with the app always load properly? Doesn’t the app always ship with the latest catalog? Even if my local WWW and Cloud WWW is down, shouldn’t the app always have the latest catalog, and therefore always be aware of the latest local “core” bundles?

This is my “core” bundle config that is always included with the game:

Addressables should never need to find and download a remote catalog just to see a bundle that is always included locally, correct?

[Endahs]

To be clear, I have found two solutions to getting new local “core” bundles to be recognized by the local catalog:

1. Change the “Player Version Override”. This works even if local WWW and cloud WWW are offline. But this defeats the purpose of “Player Version Override” and preventing Players from downloading all my bundles every time I change one file in a local bundle.

2. Turn on my local WWW server. Then the player downloads the catalog from local WWW server and sees the new local “core” bundle. But users should not need a remote catalog to tell them about a bundle they already have installed along with the app, as the “core” bundle is always included in the build and never updated remotely.

[pillakirsten]

Hi @Endahs thanks for posting about this issue. In the Editor the local catalog should be getting rebuilt and reference bundles from the latest build. Note that in an actual player build, the local catalog is only built once (when the player is built) and can only be updated by downloading a remote catalog.

First thing I would try is updating Addressables to the latest version (1.21.9), and seeing if the issue persists.

There’s a way to grab all the currently loaded catalogs and their keys through Addressabes.ResourceLocators or by accessing the results of the handle returned by Addressables.InitializeAsync. Alternatively you can just inspect the local catalog directly. If you disable “Compress Local Catalog”, that should build your catalog as a json file.

Another thing you can try is enabling “Disable Catalog Update on Startup”. By default if Addressables is initialized, it will try to download the remote catalog if it’s available. This should ensure that the local catalog is the only catalog that is loaded initially.

If all else fails, I would recommend submitting a bug report with small reproduction project.

[Endahs]

Hi @pillakirsten , thank you for your reply.

Understood that at build time the latest catalog is included in the build along with the latest local CORE bundle. In my case I never update the CORE bundle remotely and the only time a user will receive a new CORE bundle is if the binary in the store is updated. My issue is that after modifying and rebuilding the CORE bundle, the local catalog continues to reference the old bundle. It seems to me that either the local catalog is not being updated, or an older copy of the catalog is being cached somewhere and being used instead of the latest. I admit this is part of the black box of Addressables which is why I’m asking for help.

I will try updating to the latest Addressables as well as disabling automatic catalog update. It’s very odd to me that I can reference my latest local CORE bundle only after I turn my web server on, as this is a bundle that should always be available local and never updates remotely.

Now I’m wondering if Addressables is building the latest catalog in my local WWW folder and not locally? Again, black box, so I’m just guessing what is happening behind the curtains. My local CORE bundle is always set to build locally, and my DLC bundles are all set to build in the WWW folder. Perhaps having a combination of local (never updating) and remote (always updating) bundles is not a use case that Addressable supports?

Local CORE bundle settings:

Remote DLC bundle settings:

I suppose I can just get rid of the local CORE bundle and move all the files into the Resources folder and stop using Addressables for local files…

[Endahs]

Compressing the local catalog or not does not fix this. When I disable compression I can read the json and can confirm the local catalog has the latest changes. The catalog is being updated locally (in Project/Library/com.unity.addresables/aa) and in the WWW folder (simulating remote). Both the local catalog and WWW folder catalog are being updated when bundles are rebuilt, and both contain the change. Keep in mind I am clearing the build cache and rebuilding all bundles before I make every change.

Turning off the “Build Remote Catalog” option DOES get it working again, after I clean the build cache and rebuild everything. On boot up it successfully loads the local catalog and I can see the latest changes in the build.

So I have three workarounds that let me boot successfully after changing the local CORE bundle:

1. I rename the “Player Version Override” and force a new catalog name to be re-generated (unacceptable as users won’t be aware of the new catalog unless I update the entire binary on the store).

2. I turn on webserver or Cloud (unacceptable, as remote should NOT be required to load a local change).

3. I disable “Build Remote Catalog” (unacceptable, as I require remote capabilities for my DLC bundles).

In all these cases it seems Addressables is trying and failing to get the remote catalog, even though I am only booting using the local CORE bundle which is never kept remotely. Obviously I should not need a remote catalog to load resources were (and will always be) built and included at build-time. DLCs are optional and player shouldn’t need a remote catalog to load local resources.

Will try new version of Addressables or manually updating catalog next. Sigh.

[Endahs]

Disabling automatic catalog update does not work either. Same issue: Addressables attempts to load a bundle that no longer exists, because it rebuilt a new bundle and deleted the old one. Obviously there’s a catalog somewhere that is being loaded and continues to reference old, non-existent bundles.

[Endahs]

Updating to latest Addressables (1.21.9) did not change anything either. Same error:

Addressables at this point is set to NOT look remotely for a catalog update, initializes successfully, but fails on the first asset load because it is looking for an old local bundle that no longer exists (because it was rebuilt with a new hashed name).

Current local bundle settings:

And the actual bundle that was just rebuilt in the local Library folder:

Notice the difference in the CORE bundle it is looking for during runtime (ends with “e3f.bundle”) and the bundle that actually exists (ends with “afe.bundle”).

I’m out of ideas, over 40 hours into troubleshooting this, and resigned to start stripping Addressables out of the project. Any suggestions before I axe it??

[Endahs]

You can disregard this post. I removed Addressables and implemented AssetBundles instead for the local bundle, and it’s working perfectly now.

[Epsilon_Delta]

I have similar issue.
If I replace my remote catalog with the catalog of the same name, it is not updated. It looks like the remote catalog is somewhere cached, but I can’t find where.
I deleted my catalog from my server (if I want try to downloaded from the web browser I get “Not Found”), I deleted all data in my Android app, I uninstalled and installed the app, deleted data again. After I run the app it “downloads” the remote catalog successfully. I don’t know where is it cached.
Renaming the catalog and ergo creating new URL resolves the issue but it is unacceptable solution for me.

[Endahs]

Exact problem I had. I was able to re-produce the issue again and again. As you said, the only fix was renaming the catalog, which defeats the purpose of Addressables and prevents anyone with the old catalog from getting updates.

The Addressables team doesn’t frequent the forum often and frequently do not respond or follow up on any issues here. This is in stark contrast to other Unity teams, like Unity IAP. Ultimately I had to remove Addressables for my CORE bundle and “regress” to just using Asset Bundles, but that worked perfectly. You may have better luck than me in getting support, however.

[Epsilon_Delta]

Yeah I never got any answer for any of my issues with addressables. I must say the package is weird. It is a great idea, but the implementation and the vague issues are just… I am a bit disapointed I must say. I have much better experience using other Unity packages at similar stage of maturity. Addressables are now few years old.

[crawfis]

We are having the same issue with the latest Addressables for Unit 2022.3.7. It should reload things if the hash file changes, which it does. We are using Unity’s CCD and the Editor option to build. I can see that is is referencing an old ??assets_all??? file and gets a 404 error. That file is not in the new catalog file and the hash file has been updated. Addressables are just still pretty buggy!!! A shame because it is a great feature. I wonder whether removing the hash codes from the file names such that it just overwrites the files on build will help.

[Plnda]

Got the same damn issue, no and no official response on how to handle this.

[FedeAmihan]

I can just say I have been struggling with this same exact issue for 3 days now. Will just go with Asset Bundles as well (I used them in the past and they were working nicely, the APIs are poor, but there is always the chance to build a wrapper around them)

[CameronND]

Did you guys also upload the catalog .hash file alongside the catalog .json?
You can also open the .hash file in a text editor and change it to something else and then reupload that. The actually hash doesn’t matter as long as its different to the current one. Then the system should download and cache the new catalog.

[FedeAmihan]

Hello CameronND, yes, sure, both the json and the hash are in place and indeed trying this out in the Unity Editor with the Play Mode Script setting set to Use Existing Build works as expected. The issue happens in the build (in my case Android), that uses the catalog that is packed during binary compilation instead of the remote one.

I imagine that the reason is that the new catalog I produce is built from scratch and not via the method “Update A Previous Build” (that I tried once and yes, actually works as expected). The problem with that method, tho, is that it can only be used to update Addressables for a specific version of the build which is inconvenient in our workflow. I will add more details of our use case, in case someone had the same experience and found a solution:

The game is built without most of the addressables for two reasons: keep the binary size small (we ship on Android) and update the content without forcing the users to re-download the game. Our content updates are very frequent, since we have some features that include unique daily content, so it is important that these updates are as seamless as possible, with a low amount of downloaded data and without disrupting the user experience. At the same time we want to support different versions of the game in production to avoid immediately forcing our users to update the app when we release a new version. This means that the addressables that we want to ship will need to be compatible with more than a single binary version of the game, which is not possible with this technique. And, yes, it would be technically possible to store the remote content in a different folder/url on our CDN for each version of the binary, but this would be expensive and impractical and also would force our users to re-download all of the content again if we upgrade the version of the game (since in each folder, we would need to rebuild addressables from scratch, in order to then store the addressables_content_state.bin file and use it for future updates). Imagine a situation in which we have 3 live versions of the game, each one targeting a different addressables url in order to check for its own content updates: we would need to build addressables with “Update A Previous Build” three times, with three different addressables_content_state.bin for each one of them to work properly.

With Asset Bundles there are no such requirements: having a different Hash means that the bundle needs to be downloaded again and that is it. Of course this comes at a price: if we mess up and change a piece of code of the game that is referenced by a prefab in the bundle and we try to use that bundle in a previous version of the client, than it will not work, so the team itself needs to be careful about which bundles updates are retrocompatible with which version of the player, but at least that is on us to decide and not on the tech infrastructure of the Addressables that looks to me like working with a lot of assumptions that are not always true for all businesses.

To complete the full story, I tried to cheat and reference the addressables catalog as if it was coming from a different unity project via the method [Addressables.LoadContentCatalogAsync](https://docs.unity3d.com/Packages/com.unity.addressables@1.20/manual/MultiProject.html). According to Unity’s documentation this can be used to reference (and download) a catalog that was not present in the project used to build the binaries at build time. My thought, here, was that since the method can be used to download a catalog the binary knows nothing about, then the most recent one should be downloaded even if it was generated without “Update A Previous Build”, but the trick does not work fully: the catalog is indeed downloaded, but if it contains the same keys (addresses) than the one that was hardcoded in the build, apparently those take priority and the Addressable system still references old files and hashes. If someone tried this technique and succeeded, I would definitely be curious to know what tricks they used!
Still, it looks to me this trick is not totally worth it even if it was working, since there is another very nasty issue with addressables: once you load a catalog, you can never unload it or re-load it from scratch. The only way to update a catalog at runtime is via [Addressables.UpdateCatalogs](https://docs.unity3d.com/Packages/com.unity.addressables@1.20/manual/LoadContentCatalogAsync.html) but this method relies again on the fact the the update was built via “Update A Previous Build”, otherwise it does nothing. So, if a player puts the app in background, the app is not killed and when the player comes back the server notifies the client of an update to the content, the only way we would have to make Unity download the new stuff would be by killing and restarting the application since the previously loaded sticky catalog would never update itself while the app is running. For this reason, I decided to stop trying to make the update work, since in any case this issue would be a show stopper for our product.

I hope Unity will fix these issues soon and that what I write here will be outdated in no time, but for now, once more, I surrender and go back to Asset Bundles…

[DeathPro]

I encounter exactly the same bug. In Editor, I try to test my Addressables with “Use Existing Build (iOS)”. But somehow Addressables try to load previously generated asset bundles. I also clean the Caches and LocalLow Data and re-build addressable. I got the same error…

Clearing “Asset Bundle Cache” and “LocalLow Data”:

using UnityEditor;
using UnityEngine;

namespace Editor.Addressables
{
    public class ClearAssetBundleCache
    {
        [MenuItem("Tools/Addressables/Clear All Asset Bundle Cache %#&X", priority = 0)]
        public static void ClearAllAssetBundleCache()
        {
            bool isAssetBundleClearCacheSuccess = Caching.ClearCache();

            if (isAssetBundleClearCacheSuccess)
                Debug.LogFormat("All Asset Bundle Cache cleared");
            else
                Debug.LogFormat("All Asset Bundle Cache clear failed");
        }
    }

    
[MenuItem("Tools/Addressables/Clear LocalLow Data %#&C", priority = 1)]
private static void ClearLocalLowData()
{
    // UserProfile = C:\Users\MyUser
    string folderFromUserProfile =
        Path.Combine(Environment.GetFolderPath(Environment.SpecialFolder.UserProfile), "AppData", "LocalLow",
            $"{PlayerSettings.companyName}", $"{PlayerSettings.productName}");

    try
    {
        Directory.Delete(folderFromUserProfile, true);
    }
    catch (Exception e)
    {
        Debug.LogFormat($"{e.Message}");
        throw;
    }

    Debug.Log("LocalLow Data cleared");
}
}

[crawfis]

I think we determined there were (also) problems in switching CCD builds from say Windows to iOS. Had to make sure to rebuild addressables cleanly every time we switched. As well as change the profile for addressables.