Dependency cache check doesn't use resolved paths?

Unity Engine
,
1

This is a bit of supposition on our part. We have quite a complex setup.

In our case each prefab is a separate .bundle.

Prefab A & Prefab B may share a dependency (e.g. a Material “brickMat”). In this example, the Material is also in its own .bundle.

Prefab A & Prefab B may be published (built & uploaded to S3) independently and at separate times.
Say Prefab A gets published in January and lives at http://…/January/A.bundle
Once uploaded, the shared material gets uploaded to http://…/Shared/brickMat.bundle

Then, Prefab B gets published in Feb to: http://…/February/B.bundle and at this point, the brickMat.bundle is overwritten at …/Shared/brickMat.bundle.

At runtime we use InternalIdTransformFunc to resolve the paths as appropriate. Our build path for our addressables are essentially constant and we resolve at runtime.

The issue
Loading one addressable after the other (no matter the order), on the second load we receive the error
The AssetBundle 'https://..../Shared/brickMat.bundle' can't be loaded because another AssetBundle with the same files is already loaded.
Despite the bundle being requested from exactly the same path both times and it being the same bundle.

Our theory is that a load is attempted on the bundle, even though it has already been downloaded because the check if it was previously downloaded is done via another internal id or else pre-transformed path. This makes some sense given that indeed, brickMat.bundle may have changed between when Prefab A & B were published so have a different catalog and perhaps internal id?

In our case this behaviour is ‘safe’ by policy as we ensure a shared bundle (eg brickMat.bundle) only contains changes that we consider compatible with either A & B.

We would super appreciate the ability to prevent this error. At the very least, the ability to handle the exception.

Although tangential I should note that we already have “Unique Bundle IDs” enabled.

N.B. If the setup above sounds a bit odd it’s because I’ve tried to simplify the example. Happy to hop on a call if more useful.

Many thanks in advance.

2

Hey, so it sounds like to me that the requested IResourceLocation is failing an equivalency check somewhere that it shouldn’t. What I think is happening is that the first prefab is request and as part of that it has a dependency on the brickMat bundle, so that location is also downloaded + asset bundle loaded. Then, when the next prefab is requested it has that dependency and so Addressables looks in the build cache and sees you have a brickMat location currently loaded, but due to some discrepancy in the actual IResourceLocation, it doesn’t see it as being the same. Then when it actually downloads the bundle, since something called matBrick.bundle is already loaded, it throws an error.

tl;dr yeah, I think your theory is correct.

If brickMat.bundle is changing between the A and B upload. If the internal Id is different that could definitely be what’s causing it. If the provider Id, or requested resource type is different that can also be why the two locations aren’t being detected as the same.

I think what we need to do on our end is modify that comparison and make sure that the transformed internal id is what’s used to compare at runtime.

As a workaround, it might be possible to just change the bundle naming mode to Filename before you build (on the Group Schema) so that asset hashes aren’t included in the internal id. Then the comparison should be the same.

1 Like
3

Apologies for being slow to reply - really appreciate the comprehensive response.
Since your message we have tried setting naming mode to both Filename & Hash of filename. The forward slashes in the Filename were causing some issues in our pipeline so right now we can’t test that out. After changing from Hash of asset bundle to Hash of Filename it seems we’re seeing the error less. We still see it on occasion but it’s tricky to tell right now if that’s due to older catalogues.
You mentioned using Filename should mean asset hashes aren’t included in the internal id. If we use ‘Hash of Filename’ is that still true? If not, we could look at fixing up our pipeline to deal more sensibly with the forward slashes.

Many thanks for your help - much appreciated. Needless to say, it would be great if this was fixed in the package, thank you.

4

Hey, so “Hash of Filename” also shouldn’t include the asset hash. It’ll just replace the filename with the hash of that filename, the asset hash doesn’t come into play.

I agree it’s something we should try and address. I actually have a ticket for this that I’m looking at right now. It’s not quite as straight forward as I’d hoped, but we’ll see what we can get figured out.

1 Like
5

Anything you could do towards it would be greatly appreciated, thank you. Going by our tests, changing the naming mode didn’t help and we’ve had to fallback to disabling caching for the Group that encompasses all our ‘shared’ asset bundles. I originally thought it was a somewhat harmless error as things appeared to work anyway but have since realised / come to believe believe there are real issues and in our case certain shaders are sometimes not resolved. Disabling caching is a work around we can live with for now but not ideal long-term. Many thanks again.

1 Like
6

Just an update. We’re still having issues and it turns out disabling the caching didn’t work unfortunately. At the moment we’re trying to work out if issues we’re seeing with missing materials & missing shaders are connected and how - though it does seem to all be around shared assets so I hope this cache issue is the underlying reason why.

7

So, we’re discussing this issue right now, as it were. Do you load multiple content catalogs by chance? Or do you just use the default one that’s loaded? I feel like originally I underplayed the Unique Bundle IDs significance as well. That may be playing more of a role than I originally thought.

8

In case it helps I’m attaching screenshots of our specific configurations. First is the settings (Default Local Group) for what we call ‘elements’. At one point this could contain Prefab A & B, at another time only Prefab B for example.
Second is our Duplicate Asset Isolation group (aka what I call our Shared assets group).

We do load multiple content catalogs, yes. We also attempt to support mid-run updates to make things extra spicy. So we do have Unique Bundle IDs enabled.

Given both Prefab A & B use brick.mat. If I publish Prefab A & B together (part of the same content catalog), they both load fine.
If I then e.g. subsequently publish (meaning Build then upload to S3) Prefab B only, after successfully loading the new catalog for it, when loading the asset via the addressables key, I get the aforementioned errors from the dependencies (i.e. brick.mat). i.e.

The AssetBundle 'xxx.bundle' can't be loaded because another AssetBundle with the same files is already loaded.
UnityEngine.AddressableAssets.Addressables:LoadAssetAsync<UnityEngine.GameObject> (UnityEngine.ResourceManagement.ResourceLocations.IResourceLocation)

Is there any more info I can provide? More than happy to hop on a call at some point if helpful.

7952314--1018306--upload_2022-3-9_18-14-31.png
7952314--1018303--upload_2022-3-9_18-14-2.png7952314--1018309--upload_2022-3-9_18-16-43.png

Notes
I recently enabled CRC checks as an experiment.
I also tried updating from 1.19.18 to 1.19.19 but no difference.
I tried setting the cache clear behaviour to ‘when new version loaded’ as an experiment but that generated errors about being unable to delete existing cache files.

1 Like
9

Also if it helps, initial tests indicate that disabling Unique Bundle IDs doesn’t appear to make a difference.

1 Like
10

Just wanted to update this while I’m thinking of it. We have something slotted for the upcoming release that should fix this issue. The main thing to note is that, in order for your shared material to still be referenced by other your other prefabs, you won’t be able to use Unique Bundle IDs. It has to do with how files reference each other. If Unique Bundle IDs is on, the bundle that still thinks it’s pointing to the original texture won’t be able to find the reference for the material after it’s updated. Only if the CAB file name (the internal name of the asset bundle) and the object id are unchanged, would both prefabs still be able to reference a material after it’s updated but one of the prefab bundles aren’t.

I hope that makes some sense. It could also be that your specific setup does allow you to use Unique Bundle IDs, I just wasn’t able to in my repro setup.

But, as far as having the issue of making TransformInternalid point to the same bundle in multiple ways, that should be resolved in the upcoming release.

1 Like
11

That’s great news, thank you. Despite my delayed response we are very eager to get this fixed.

We’ve been doing a lot more exploration and more systematic debugging of our issues.

All previous observations seem to hold but in addition we discovered a large thing tripping us up is that with Unique Bundle Ids enabled, the internal ids used for shared assets become no longer deterministic across machines. Quite unsure if this is obvious, something I missed in the docs or else a bug, but it was a big part of our issues. Missing shaders and materials now explained. So we’ve had to disable the unique bundle id option anyway.

Many thanks for your help and support with this - looking forward to the next release!

12

A bit delayed but @davidla_unity I just wanted to thank you for doing this fix and to confirm that with 1.20 it solved our error which has helped hugely - not least our automated testing. Thanks again.