Import getting stuck after upgrading to 2021.2.2f1

Unity Engine
1

Since upgrading to 2021.2.2f1 and the introduction of multi-process importing, we are seeing issue where imports get stuck in a cycle of waiting for workers that eventually times out. It seems to be worse in our Jenkins CI pipeline. We are currently building for PS4/PS5.

Once the import process get’s stuck, its just hundreds of lines of this until I restart the job.

01:24:28 [OnDemand] Worker ##: WorkerState(Connecting), ImporterState(Initializing) worker timed out connecting with editor - while handling artifactKey Guid(00000000000000000000000000000000) Importer(-2,00000000000000000000000000000000) - path  - see Logs/AssetImportWorker###.log

I’ve tried disabling AV and firewall entirely - it seems to help but doesn’t stop the issue entirely.

Wondering if anyone else is seeing this, and if they know any workarounds. I can’t find anything relevant in the patches notes, or on Unity’s bug tracker.

Here is an example of a worker that times out:

nothing useful in the log files, just the worker process' launch diag.

Using pre-set license
Built from '2021.2/staging' branch; Version is '2021.2.2f1 (5e2b1e92c7f8) revision 6171422'; Using compiler version '192829333'; Build Type 'Release'
OS: 'Windows 10 Pro; OS build 19042.1348; Version 2009; 64bit' Language: 'ja' Physical Memory: 65414 MB
BatchMode: 1, IsHumanControllingUs: 0, StartBugReporterOnCrash: 0, Is64bit: 1, IsPro: 0

COMMAND LINE ARGUMENTS:
C:\Program Files\Unity\2021.2.2f1\Editor\Unity.exe
-adb2
-batchMode
-noUpm
-name
AssetImportWorker355
-projectPath
C:/jenkins/build-environment/ps5-01/master_build_ps5
-logFile
Logs/AssetImportWorker355.log
-srvPort
64740
Successfully changed project path to: C:/jenkins/build-environment/ps5-01/master_build_ps5
C:/jenkins/build-environment/ps5-01/master_build_ps5
[UnityMemory] Configuration Parameters - Can be set up in boot.config
    "memorysetup-bucket-allocator-granularity=16"
    "memorysetup-bucket-allocator-bucket-count=8"
    "memorysetup-bucket-allocator-block-size=33554432"
    "memorysetup-bucket-allocator-block-count=8"
    "memorysetup-main-allocator-block-size=16777216"
    "memorysetup-thread-allocator-block-size=16777216"
    "memorysetup-gfx-main-allocator-block-size=16777216"
    "memorysetup-gfx-thread-allocator-block-size=16777216"
    "memorysetup-cache-allocator-block-size=4194304"
    "memorysetup-typetree-allocator-block-size=2097152"
    "memorysetup-profiler-bucket-allocator-granularity=16"
    "memorysetup-profiler-bucket-allocator-bucket-count=8"
    "memorysetup-profiler-bucket-allocator-block-size=33554432"
    "memorysetup-profiler-bucket-allocator-block-count=8"
    "memorysetup-profiler-allocator-block-size=16777216"
    "memorysetup-profiler-editor-allocator-block-size=1048576"
    "memorysetup-temp-allocator-size-main=16777216"
    "memorysetup-job-temp-allocator-block-size=2097152"
    "memorysetup-job-temp-allocator-block-size-background=1048576"
    "memorysetup-job-temp-allocator-reduction-small-platforms=262144"
    "memorysetup-temp-allocator-size-background-worker=32768"
    "memorysetup-temp-allocator-size-job-worker=262144"
    "memorysetup-temp-allocator-size-preload-manager=33554432"
    "memorysetup-temp-allocator-size-nav-mesh-worker=65536"
    "memorysetup-temp-allocator-size-audio-worker=65536"
    "memorysetup-temp-allocator-size-cloud-worker=32768"
    "memorysetup-temp-allocator-size-gi-baking-worker=262144"
    "memorysetup-temp-allocator-size-gfx=262144"
Refreshing native plugins compatible for Editor in 1188.84 ms, found 7 plugins.
Preloading 0 native plugins for Editor in 0.00 ms.
Initialize engine version: 2021.2.2f1 (5e2b1e92c7f8)
[Subsystems] Discovering subsystems at path C:/Program Files/Unity/2021.2.2f1/Editor/Data/Resources/UnitySubsystems
[Subsystems] Discovering subsystems at path C:/jenkins/build-environment/ps5-01/master_build_ps5/Assets
GfxDevice: creating device client; threaded=0; jobified=0
Direct3D:
    Version:  Direct3D 11.0 [level 11.1]
    Renderer: NVIDIA GeForce GTX 1660 Ti (ID=0x2182)
    Vendor:   NVIDIA
    VRAM:     5991 MB
    Driver:   30.0.14.7111
Initialize mono
Mono path[0] = 'C:/Program Files/Unity/2021.2.2f1/Editor/Data/Managed'
Mono path[1] = 'C:/Program Files/Unity/2021.2.2f1/Editor/Data/MonoBleedingEdge/lib/mono/unityjit-win32'
Mono config path = 'C:/Program Files/Unity/2021.2.2f1/Editor/Data/MonoBleedingEdge/etc'
Using monoOptions --debugger-agent=transport=dt_socket,embedding=1,server=y,suspend=n,address=127.0.0.1:56620
Begin MonoManager ReloadAssembly
Registering precompiled unity dll's ...
Register platform support module: C:/Program Files/Unity/2021.2.2f1/Editor/Data/PlaybackEngines/WindowsStandaloneSupport/UnityEditor.WindowsStandalone.Extensions.dll
Registered in 0.050056 seconds.
Native extension for WindowsStandalone target not found
Refreshing native plugins compatible for Editor in 646.83 ms, found 7 plugins.
Preloading 0 native plugins for Editor in 0.00 ms.
Mono: successfully reloaded assembly
- Completed reload, in  1.691 seconds
Domain Reload Profiling:
    ReloadAssembly (1692ms)
        BeginReloadAssembly (256ms)
            ExecutionOrderSort (0ms)
            DisableScriptedObjects (0ms)
            BackupInstance (0ms)
            ReleaseScriptingObjects (0ms)
            CreateAndSetChildDomain (1ms)
        EndReloadAssembly (1187ms)
            LoadAssemblies (227ms)
            RebuildTransferFunctionScriptingTraits (0ms)
            SetupTypeCache (114ms)
            ReleaseScriptCaches (0ms)
            RebuildScriptCaches (27ms)
            SetupLoadedEditorAssemblies (831ms)
                LogAssemblyErrors (0ms)
                InitializePlatformSupportModulesInManaged (35ms)
                SetLoadedEditorAssemblies (0ms)
                RefreshPlugins (647ms)
                BeforeProcessingInitializeOnLoad (1ms)
                ProcessInitializeOnLoadAttributes (107ms)
                ProcessInitializeOnLoadMethodAttributes (41ms)
                AfterProcessingInitializeOnLoad (0ms)
                EditorAssembliesLoaded (0ms)
            ExecutionOrderSort2 (0ms)
            AwakeInstancesAfterBackupRestoration (0ms)
Platform modules already initialized, skipping
Registering precompiled user dll's ...
Registered in 0.378416 seconds.

Cheers.

2

I tried increasing the idle worker timeout to 120s, which seems to allow 1/20 workers to survive. Not sure if it’s related at all, may be a fluke…

Will continue to investigate.

3

bump

4

bump

5

bump

6

bump

7

I am running into this as well on a Ubuntu VM. But not with every project. Have you have any luck?

8

Unfortunately not. Just running Jenkins on a server with anti-virus completely disabled.

9

I’m having this issue in 2021.2.17.f1

Sad to see there has been no response to the previous post.

10

Looking into the Logs/AssetImportWorker###.log in the project folder may give a clue

11

Hi jdrewson,

That’s what the example log is in the original post - and every single one is the same. Unless I’m missing something I don’t think there is anything of note there.

Kind regards,
James

1 Like
12

Ummm, it’s exactly as described above. It just says it’s timed out.

13

How long does it take for the editor to start up a preimported version of the project in question on the machine in question? The workers have a kill timeout of 30 seconds if they do not report back to the editor.

14

When it works it loads my project in under 2 minutes.

And the workers are timing out, but as the OP mentioned, it just starts a new one.

15

My unity version is 2021.3.0 macOS. Same problem.
I delete my layout cache(UserSettings/Layouts).seems better.
But stuck at some very big model importing. Delete the model. I finally get into my project…

16

As a quick mitigation fix we are bumping the timeout from 30 secs to 5 mins and backporting to misc Unity versions. We are also planning to do a less hardcoded fix to be released later.

17

Good to hear, as I’m still having this issue and it’s driving me crazy.

Digging through my AssetWorkers and Editor log, this always comes up.

Begin MonoManager ReloadAssembly
Symbol file LoadedFromMemory doesn't match image G:\Unity Projects\Swordsfall ARPG Main 8.0\Library\PackageCache\com.unity.visualscripting@1.7.8\Editor\VisualScripting.Core\Dependencies\DotNetZip\Unity.VisualScripting.IonicZip.dll
Symbol file LoadedFromMemory doesn't match image G:\Unity Projects\Swordsfall ARPG Main 8.0\Library\PackageCache\com.unity.visualscripting@1.7.8\Editor\VisualScripting.Core\Dependencies\YamlDotNet\Unity.VisualScripting.YamlDotNet.dll
18

Same with 2021.3.2f1, all AssetImporter log ends with “Begin MonoManager ReloadAssembly”.

19

We have this problem when running the Unity editor in headless mode on an M1 Mac Studio. The log is full of:

01:40:37 Starting new worker id: 0 with log in
01:40:37 Starting new worker id: 1 with log in
01:41:12 [OnDemand] Worker 0: WorkerState(Connecting), ImporterState(Initializing) worker timed out connecting with editor - while handling artifactKey Guid(00000000000000000000000000000000) Importer(-2,00000000000000000000000000000000) - path - see Logs/AssetImportWorker0.log
01:41:12 Starting new worker id: 2 with log in
01:41:12 [OnDemand] Worker 1: WorkerState(Connecting), ImporterState(Initializing) worker timed out connecting with editor - while handling artifactKey Guid(00000000000000000000000000000000) Importer(-2,00000000000000000000000000000000) - path - see Logs/AssetImportWorker1.log
01:41:12 Starting new worker id: 3 with log in
01:41:41 [OnDemand] Worker 2: WorkerState(Connecting), ImporterState(Initializing) worker timed out connecting with editor - while handling artifactKey Guid(00000000000000000000000000000000) Importer(-2,00000000000000000000000000000000) - path - see Logs/AssetImportWorker2.log
01:41:41 Starting new worker id: 4 with log in
01:41:41 [OnDemand] Worker 3: WorkerState(Connecting), ImporterState(Initializing) worker timed out connecting with editor - while handling artifactKey Guid(00000000000000000000000000000000) Importer(-2,00000000000000000000000000000000) - path - see Logs/AssetImportWorker3.log

entries, and when inspecting the /Log output for the workers they all look like this:

Unity Editor version: 2022.1.9f1 (07e076b6d414)
Branch: 2022.1/staging
Build type: Release
Batch mode: YES
macOS version: Version 12.4 (Build 21F79)
Darwin version: 21.5.0
Architecture: arm64
Running under Rosetta: NO
Available memory: 32768 MB
Using pre-set license
Pro License: YES
2022-07-24 22:24:48.934 Unity[22428:558830] +[NSXPCSharedListener endpointForReply:withListenerName:]: an error occurred while attempting to obtain endpoint for listener 'ClientCallsAuxiliary': Connection interrupted

We are using the Unity Accelerator and the Unity instance is invoked as such:

"${UNITY_PATH}" \
-projectPath . \
-buildTarget ${BUILD_TARGET} \
-logFile - log.txt \
-EnableCacheServer \
-cacheServerEndpoint "${UNITY_ACCELERATOR_ADDRESS}" \
-cacheServerEnableDownload \
-cacheServerEnableUpload \
-username xxx \
-password xxx \
-batchmode \
-nographics \
-quit \
-silent-crashes

This happens always after the Library has been cleaned, and requires a few runs before it starts working again.
Please help us as this is pretty much a fatal error for our CI.

Thank you!

20

Update: The error goes away if I log in on the server, at which point I see the Unity splash screen flash. The -nographics option is clearly not working.