Unity Accelerator Upload / Caching Issues

Unity Engine
1

Hi, we are seeing that some Assets are always reimported and not downloaded from the Unity Accelerator and I have examined this strange behaviour in the log / disc access:

Client with force reimport and upload to cacheserver

Done importing asset: 'Assets/Resources/FX/FX_Resources/model/MOUNT_SWAMPDRAGON_SHED_SKIN_03.fbx' (target hash: '8aada8d096289f6a3d350155ef6fcf43') in 0.784921 seconds
RemoteAssetCache::AddArtifactToCacheServer - artifactKey='Guid(a707bad777e7eca42b3c4ceb295b64a9) Importer(-1,00000000000000000000000000000000)' Target hash='8aada8d096289f6a3d350155ef6fcf43' Key (and library path)='Library/Artifacts/cd/cd85fa826b40440b1774dc2556749f09'
RemoteAssetCache - Upload - Artifact - success:true, namespace:defaultmetadata, key:cd85fa826b40440b1774dc2556749f09```

CacheServer log:
```{"level":"debug","ts":"2022-03-14T12:07:36.537Z","msg":"received putRequest","agent_id":"unityaccelerator.office.albion.zone","agent_name":"unityaccelerator.office.albion.zone","component":"pbservice","subprocess_id":5,"conn_id":4401,"remote":"10.23.0.105:54567","segment_id":7,"namespace":"defaultartifacts","key":"cd85fa826b40440b1774dc2556749f09","size":160064,"initial blob hash received":0}
{"level":"debug","ts":"2022-03-14T12:07:36.539Z","msg":"putRequest closing cache file","agent_id":"unityaccelerator.office.albion.zone","agent_name":"unityaccelerator.office.albion.zone","component":"pbservice","subprocess_id":5,"conn_id":4401,"remote":"10.23.0.105:54567","segment_id":7,"namespace":"defaultartifacts","key":"cd85fa826b40440b1774dc2556749f09","size":160064,"final blob hash used with close":0}
{"level":"debug","ts":"2022-03-14T12:07:36.539Z","msg":"handled putRequest","agent_id":"unityaccelerator.office.albion.zone","agent_name":"unityaccelerator.office.albion.zone","component":"pbservice","subprocess_id":5,"conn_id":4401,"remote":"10.23.0.105:54567","segment_id":7,"namespace":"defaultartifacts","key":"cd85fa826b40440b1774dc2556749f09","size":160064,"final blob hash":0,"status":"ok","status_code":0}

Actual file modifications: DIFFERENT KEY

26484978 160 -rw------- 1 root root 160064 Mar 14 12:07 ./cachedb/e24b6d24ba7d7918e08be3d9b15d5fd8/4776ce65bea19c14a11c5e394f9629c2/05/05a60420192c246c92e50068cadc0c97```

Why are the artifacts saved in the cachedb with a different key as filename? Is this normal behaviour?

Thanks for your support.
2

Hi :slight_smile:

Because the key that the accelerator receives can in theory be anything(special chars, etc.), we MD5 the key and store the data under that. So that means the key ‘cd85fa826b40440b1774dc2556749f09’ would become ‘05a60420192c246c92e50068cadc0c97’ when you need to look it up.

Hope that helps.

It sounds like an importer is generating non deterministic results. Are you seeing the ‘target hash’ or library path change from import to import?

3

thanks, the target hash is changing but i don’t see any importer changing anything.
In a vanilla example with the asset (same meta file, same guid) the hash seems to be stable.
Deleting all our AssetProcessors also gives another target hash than the asset in the new sample project.

Can you explain what exactly is considered for generating the hash?

4

From the docu it says: source asset path, source asset, meta file, target platform and importer version
but what in detail?
source asset path: I assume path below Assets?
source asset: md5 checksum?
where does it take importer version from?

5

Yes, we also depend on the build target. Additionally we have custom dependencies that can be setup through an scripted importer. AssetPostProcessors can also be a source of non deterministic results because it can modify the dependencies of an import.

I believe ‘Assets/’ is part of the path.

We use SpookyV2 as the hash algorithm.

Versioning for native importers, ModelImporter is native, it’s hardcoded on the native side. For scripted importers it’s through the attribute where you also define the extension it should be associated with.

It would be interesting to see the difference between the two import results. You can use the ‘binary2text’ tool to get a readable representation of an imported asset. We also have the consistency checker tool in 20.3 which detects when importers generate non deterministic results. Doc is here https://docs.unity3d.com/Manual/ImporterConsistency.html. It’s only available from 20.3 and up, and it looks like you might be on 19.4? However, the debugging section of the doc might be helpful in terms of making a diff of the import result with ‘binary2text’.

6

Thats a good idea to look at the artifact differences of the imported FBX file. Thanks!

The only difference is the m_RootOrder of the Transform Component:

7968597--1022037--model_diff.png
7968597--1022037--model_diff.png1873×647 58.7 KB

What can influences this? The Fbx looks normal, actually it has only one mesh.
I could also send you the FBX and meta file for the importer settings.

7

We actually fixed this issue with ‘m_RootOrder’ being non deterministic in 21.2. But it never got backported. I’ve initiated the backport process for it to go into 20.3. What version are you using? A question if it should be backported further.

8

Thanks, we are in the process of upgrading to 2020.3., waiting right now for the AssetBundle Build Fix in 2020.3.31.
So i think its ok for us if it will be fixed in 2020.

Thanks for your support!

9

Actually is there anything we can do as a workaround? our buildtimes are much longer with this bug rightnow.

10

Ok, cool! We’ll “just” backport it to 20.3 for now then.

I’m not aware of a workaround sadly. The issue was around how the model importer would not get run in a clean state, and this would make it difficult to do a workaround.