Feedbacks, issues and Questions

Unity Engine
,
1

Hello there, wanted to share some feedback on Addressable, and maybe get answers to some of our issues.

It’ll be a long thread, sorry for that. If you think some points should have their seprate threads of bug filled, don’t hesitate to say so.

We’re working on a mobile game that we intend to update regularly with new content. Addressables seemed the way to go when we started investigating, and after a few month of development having moved our assets on using Addressables where we wanted to (all the data that can be updated during the game lifitime), but still having them embedded in the application, it’s time to move on putting most of them on a CDN server so we have a client as small as possible. And with that step comes a bunch of issues.

1 - Auto download

The first point for us is one of the core features of Addressables: if the asset is not present, it is imediately downloaded. Problem is, on mobile phone, I need to display an acceptation popup if the download is too big… I truly wished there was a way to prevent auto-download (for, now, I’m going the hacky way using InternalIdTransformFunc to cut all download tentatives before I said ok).

And if you’re asking “why am I doing this, requesting a file I don’t have already downloaded?”, the answers is related to a dependency issue I will explain later. What I’m willing to do (and supposed to have) is to show my stat screen, which uses some addressable elements that could be dynamically modified using Addressable updates (like background, but alos localization files). The game embedd the group containt the initial version of the file, so we don’t need to download anything theorically to display it (which is true if we don’t consider the issue I got into).

2- “Download All” feature

There is currently no way to use DownloadDependenciesAsync to download all missing bundles at once (and obviously same for GetDownloadSizeAsync). I hacked this by creating a custom BuildScriptPackedMode and adding a tag to all files present in all groups.

3- Catalog managment

I played a bit and did some tentatives on this one. What I need is to be able to update the catalog at a later time, so the game can select a specific version based on a bunch of parameters (which server I’m connecting to, am I in preview mode…).

Proble is that the catalog distant location (including server URL) is set at compile time, and goes through InternalIdTransformFunc only once at first connection tentative AFAIK. After that, there is no way to change which distant catalog I want to use.

I tried to use the additional catalog to bypass this, but unfortunately some files were present in both catalogs if I used this method, resulting in wrong download size comutation and or file collisions.

Also on this part it would be nice to have a catalog rollback or similar feature. It would be nice to correctly handle “partial download cases”:

  • game is on rev 1 with all files locally present
  • new catalog is downloaded, file download on the run
  • game is killed for any reason during downlaod
  • game is restared: we’re on the new catalog, but we don’t have the related files yet… risk of auto download of files prior to getting to the point where we can ask the user.

4- Strange file depencies (bug?)

I think this one is a bug. My packages have something in the likes of:

Pack A

  • Asset 1
  • Asset 2

Pack B

  • Asset 3 (used by Asset )

I have Pack A in local, Pack B is distant at first. Asset 2 uses Asset 3, but not Asset 1.
If I load Asset 1, Pack B is immediately downloaded (and asset 1 is not loaded unitl Pack is here). The reason is clearly because Asset 2 has a dependency to Asset 3, which was proven by doing some researchs and moving assets between packs.

I even found out what could be the issue and may fix it (I insist on the could/may, didn’t test a lot or proff check it a lot), and I don’t know if it was intended, bu I managed to have what seems to be more logical dependecies by altering a bit GenerateLocationListsTask (on 1.17.xx it’s around line 220)

original:

entry.CreateCatalogEntriesInternal(locations, true, assetProvider, bEntry.ExpandedDependencies.Select(x => x.BundleName), null, input.AssetToAssetInfo, providerTypes, schema.IncludeAddressInCatalog, schema.IncludeGUIDInCatalog, schema.IncludeLabelsInCatalog, bEntry.AssetInternalIds);

My hacky, unoptimisez, non guaranteed version

 List<string> depList;
IEnumerable<string> dependencies = null;
input.AssetToFiles.TryGetValue(assetGUID, out depList);
if(depList != null)
{
        dependencies =  depList.Where(k => input.FileToBundle.ContainsKey(k)).Select(k => input.FileToBundle[k]).Select(k => bundleToEntry[k].BundleName);
 }
entry.CreateCatalogEntriesInternal(locations, true, assetProvider, dependencies /*bEntry.ExpandedDependencies.Select(x => x.BundleName)*/, null, input.AssetToAssetInfo, providerTypes, schema.IncludeAddressInCatalog, schema.IncludeGUIDInCatalog, schema.IncludeLabelsInCatalog, bEntry.AssetInternalIds);

5 - (Bug) Initialization lock

I unfortunately have no concrete info or produce step for this one, but I hade several cases where the call to

Addressables.InitializeAsync(); is stuck, the operation handler is stuck in a “waiting to launch” state and never begins, preventing all other operations afterward. At this point we can’t do or load anything, except for checking catalog updates.

I might be wrong, but it seems to only occur if the files are all downloaded, or after doing a full data clear on android device using system menus (the latter in my case always resulted in a “now you must reinstall everything” state). But due to the randomness of this issue I can’t point out what’s going on

6 - (Bug) Hosting & CPU

If you have the Addressable Hosting window visible, it eats a core of the CPU at full charge. No matter if hosting is running or not, and if I have it open as a tab but not visible (another tab on the same group is selected), the problem does not occur.

7- Data encryption

As far as I know, there is no encryption of the file on your side, not even a basic one (we have to do it ourselves). A lot of people are asking for it, so we are: preventing data mining or tampering is important for us.

8- Pack separately option & directories

The Pack separately option as it shown in tooltip pack together all files in a given directory. It would be nice to have another option to pack separately all the files in those folders : some groups are left managed by folders so non-tech can more easily uses the system, yet for packaging, update & download reasons we’d like to stil have them packed separately.

I did a quick test to try to do it, and it seems to be doable easily in BuildScriptPackedMode.PrepGroupBundlePacking (I have yet to confirm it 100 % works). But as the function is private with no way to override it, changing it results in needing to have a local copy of addressable and forget about updates.

9 – More export override option

You’ve prepared a bunch of features and virtal functions so we can override part of the build process, it’s nice. It would be great to have more private stuff moved to protected (so we can call them in our overrides) or virtual (so we can specialize them).

10 – The Local group export directory

Using Profiles, you can change the export directories, which is nice. But if I set something other than the default value in LocalBuildPath, it seems to me that the files won’t be embedded in the build. It would be nice to allow us to specify a folder that will still be embedded, as the default directory is inside « Library » which may cause issues with source control, auto build generation and so on.

Thank you for reading everything. I know I did a big wall of text. If anybody has hints regarding my issues it’ll be highly welcomed.

2

Thanks! I’ll flag for the team to have a look. For any bugs you think you might have encountered, I strongly encourage that you go ahead and file a bug report for us on those. :slight_smile:

3

Hey @LoicLer thanks for the very detailed feedback. I’m going to comment on some things but I’ll also pass this along to more members of the team so they can chime in if they have more info.

First of all, I see you mention 1.17.xx as the version you’re on. For some of your issues I would try updating to 1.18.4+ if you’re able. At least to a version of 1.18.xx and see if the minor version bump exposed any of the built script functionality you need.

1. Auto Download
This is an interesting use case. I’m not sure if we’ll add this feature explicitly, though it’ll be worth discussing internally. You said you have a workaround, which is great! But I’d be more curious what tools you think we could provide that would give you the capabilities you’re looking for and not feel hacky. I assume what you need is more complicated than

var getDLSizeHandle = Addressables.GetDownloadSizeAsync(...);
if(!getDLSizeHandle.isDone)
yield return getDLSizeHandle; //only needed if init hasn’t happened yet, otherwise this op completes synchronous.
if(getDLSizeHandle.Result > MaxDownloadSize)
{
    if(DisplayAcceptanceDialogue())
        Addressables.LoadAssetAsync<T>(...); //Or whatever
}
else
    Addressables.LoadAssetAsync<T>(...); //Or whatever

2. Download All
So, in the current iteration of Addressables, what you’re doing is probably what I would recommend you do. Likely a temporary tag that’s defined as some constant in Addressables that we’d use internally as part of a DownloadAllAsync(); API would be very close to how we’d implement it I imagine.

3. Catalog Management
I’d need to look and double check if this does what you’re asking but we do have a CheckForCatalogUpdates and UpdateCatalogs API. These should also feed through the TransformInternalIdFunc when requesting the catalogs.

As far as the catalog rollback, yeah that’s a good idea.

4. Strange File Dependencies
This scenario is actually covered in the docs here Memory management | Addressables | 1.18.19. Generally, if you have issues with this behavior we’d recommend packing that Group as “Pack Separately” or break those assets into their own Groups.

However, since you did provide a potential solution, we can take a look at incorporating this into an Addressables 2.0. I doubt we add this to the current 1.x of Addressables since that would be a pretty major behavior change in my mind.

5. Initialize Lock
Yeah this sounds like a bug. Try updating Addressables and make sure there isn’t a fix in there. If you still are encountering the problem please submit a bug report with a project that reproduces the issue.

6. Hosting and CPU
Same thing, try updating if you can but this sounds like a bug. I recommend updating for this one because this sounds very near to something we’ve fixed in a semi-recent version if I’m not mistaken.

7. Data Encryption
Yes, we’re very aware this is something a lot of users need. Like the Auto Download section we’d be more likely to implement more general purpose tools that would enable users to implement their own specific encryption solutions.

8. Pack Separately & Directories
When we do an Addressables 1.19.x we can look into exposing PrepGroupBundlePacking so you’re able to do this.

9. More export override
This is one I’d say definitely check out 1.18.x since we exposed a bunch of stuff in the build scripts to help with extensibility. If you still find yourself too constrained, file a bug with the API’s you feel should be overridable and we’ll look into it for sure.

10. Local group export
This sounds very familiar. I think this has been discussed recently. Likely what you’re seeing is an issue with how we copy files into StreamingAssets as part of the Player build. We need to be respecting the local build path and not trying to point directly to the Library. The actual file copying is done using Addressables.PlayerBuildDataPath and not the “LocalBuildPath” of the profile. There was a ticket to look into doing this at one point, though I think there were some technical issues around changing it. I’ll see about digging that up and maybe there’s something we can do.

4

Thanks a lot for taking the time to answer to my various points. I’ll try to complete my original post as much as possible here.
I just saw I forgot to mention the version we’re currently using, so just for reference we’re on 1.17.17 (after a while on 1.9), and did not move to 1.18 because of the changes we made directly in the code (especially the packing one)

1. Auto Download
Indeed, it’s a bit more complicated that youre example, as requesting several assets would produce several popups. maybe a kind of callback so we can aggregate all missing assets?

But at first, Simply having the Asset loading returns an error like “Asset not available” or “Download requested” would be sufficient. We’ve done our best to handle packaging so the requested assets to display start screen and elements prior to the download step are embedded in the game (or at least have a version embedded even if we update them afterward), except for the dependencies due to point 4.

2. Download All
I’ll keep my version then for now. Just for reference, I also automatically added a tag with the group name, so I can more easily manage when and how I want to download a group. Problem of doing it ourselves is that if we move files, the old group tag will stay unless we manually remove it.

We’re intending of using this to prioritize our download, so we know we have some parts up-to-date and let the player continue while downloading the remaining parts.

For reference, another kind of usage would be this one (not our case currently but could be expected): imagine a game with lots of voices. having the group tag allow us to let the player decide to download all sounds at once from a menu if he prefers to (he’s currently on a wifi spot and have enough space to have everything on his device, versus someone else who’s short on space but always have a good internet access so don’t mind a on-demand download)

3. Catalog Management
Indeed, both goes through TransformInternalIdFunc, at least the 1st time or until it gets a valid response.

Our case that seems not to work is that we have a tag in the catalog name that let us select a specific remote catalog (this tag is replaced using TransformInternalIdFunc). If we dynamically update this tag afterward and do a new request to CheckForCatalogUpdates, it seems the TransformInternalIdFunc is not called and the previous url will always be used.

4. Strange File Dependencies
I missed the point in the doc indeed (or more I did not understand it). Unfortunately, the group separately or group break is really hard in our case due to the dependency graph, and the fact that we were not aware of the dependency until I discovered a huge download at game start.

Is there a way to see the package dependencies? I don’t think the Analyze tools show us that information. It could help prevent the scenario, or at least try to ease it.

Yet a more precise dependency like I did try to implement would be really great and helpful to a lot of devs I think (I don’t know if you’ve checked my code, if you have any feedbacks on it it’d be greatly appreciated).

5. Initialize Lock
I could not produce it again for the past 2 weeks, but we’re monitoring this one on our side. If the issue occurs again we’ll try to find a more precise produce step and let you know.

6. Hosting and CPU
I’ll try to upgrade on 1.18 if I can, but it’s happening on 1.17 so close to last release (or maybe I’ll wait for 1.19 if you expose more stuff that would ease the upgrade for us).

7. Data Encryption
That would be great yes. Hoping you can work on this on a near future :slight_smile:

8. Pack Separately & Directories
I’d appreciate having it exposed indeed (or having a new option for it if you think devs could use both ‘pack per folder’ and ‘pack per files in the folder’). If it helps I can share what I did, the only minor issue of my custom code is that the bundles names are not based on the asset name, but use a hash/id. I don’t care at all (on the contrary it helps mitigating datamining as the names are less readable), but some devs could have a different point of view.

9. More export overrid
Yes, thanks for thoses exposition. Overriding you classes instead of rewriting them is always better, as it’s way easier to upgrade the package (main reason I did not try 1.18 yet).
We currently bypassed a lot of our issues with non exposed stuff by going post-process, but it’s way less efficient and easy to use.

For reference, here are the changes I did or wanted to do in addressables 1.17:

  • GenerateLocationListsTask.RunInternal: already discussed (more precise dependency graph)

  • BuildScriptPackedMode.PrepGroupBundlePacking: already discussed (pack per file in folder)

  • AddressablesDataBuilderInput.PlayerVersion: In the end I’m not using it, but I did put it in public set. Reason is that I wanted a custom catalog name, but one that is automatically generated on build. What you gave to us is a property in the settings (player version override), but at the time my build script is initialized (and I can access the settings property), it’s already too late. In the end I did a post process copy of the catalog using the name I wanted.

10. Local group export
It would be great if this point moves to a resolution. For reference (and other threads are talking about this), I’m currently working on our “builder farm” to export the addressable and build generation and have a central, shared, and more “reliant” packaging process, and the fact that localgroup assets are in the Library folder (and as such not shareable between different copies of the project) made me hit a dead point where I did not succeed in having 2 copies of the project into producing compatible bundles & catalogs (each copy of the project, wether on the same machine or on a different one, is making bundles with different names/hashes so all files are duplicated. I know taht addressable_content_state.bin is supposed to help preventing that, but currently as it has been explained there’s an issue when you have localGroups so I could not get the system to work (waiting for more feedbacks or a fix on this point before spending more time on trying to ake my builders works perfectly, and in the meanwhile limiting to a single point the package generation for the release clients).

5

This is awesome, thank you for the feedback. I’m going to make a few tickets to address as much of this as we’re able. I can’t make any promises or guarantees, of course, but we’ll definitely look into it.