Dynamic libraries crashing in Editor since 2019.2, working in Standalone

Unity Engine
1

Hi,

We’ve been having some issues with our macOS Unity app that contains a plugin in .bundle format which packages a static library. For about a year now we have also packaged several dynamic libraries in this same bundle. Our plugin executable and the dylib’s are on the same level like this:

Plugin.bundle
-Contents
--MacOS
---Plugin (executable)
---1.dylib
---2.dylib
---etc.

For about a year now we only had to add the @loader_path entry to the dylib’s RPATH to make it work. The code that internally loads the dynamic libraries uses dyld / dlopen.

However since Unity version 2019.2. (and all versions newer than this), this structure no longer works for us. It crashes after about 30 seconds since we hit play in the Editor, without any useful logs or information. Attaching the debugger to Unity allows us to see exactly where it crashes, but we’re at a loss why this part of the code would fail.

This exact same project with the exact same structure works fine if I go to Build Settings->Create Xcode project, and run it as a standalone app. It only fails in the editor.

The release log for version 2019.2 states:

From this I can presume that some modifications were made to the way dylib’s are handled on macOS in this version, because 2019.1.14, 2019.1.5 and so on work fine for us. As for macOS version, we are on 10.15 (Catalina).

I’ve spent about 10 days now trying to debug this from our side and feel like I exhausted all options. Hoping that someone with internal knowledge on what exactly was changed can chime in and tell us if we maybe need to package dylib’s differently now, or use a different RPATH, or something like that in order for this to continue working in the editor.

Thank you

2

Bump.

3

No one has information on what exactly changed from 2019.1.14 to 2019.2.0 regarding handling of .dylib’s?

libtbb.dylib and libtbbmalloc.dylib were added to Unity Frameworks folder in this version and start to be used for some reason. But I have excluded them as the source of the problem.

Essentially our dylib’s load some files from disk and this goes OK, but later attempting to access that data in memory throws an EXC_I386_GPFLT exception. Once again, the same exact project works fine as a standalone build, it is only failing in editor.

4

Well, we managed to find the source of the problem, so for any lost souls who may wander into this themselves…

There is apparently a bug in the OpenImageDenoise library that ships with Unity related to detection of AVX-512 on macOS.

This bug was later fixed, in OIDN v.1.2.3:

Unfortunately Unity uses an older version, 0.9.0, which still has this bug. Another Intel library that we use happens to also invoke MKLDNN functions (which are also used in OpenImageDenoise) and these somehow ended up being resolved to the Unity file (since it contains the same symbols), and not to the one we supplied, hence it was trying to load the functions from libOpenImageDenoise.dylib. This triggered the AVX-512 crash.

Simply replacing the version of libOpenImageDenoise.dylib files located in
/Applications/Unity/Hub/Editor/2019.4.14f1/Unity.app/Contents/Frameworks
With the new v1.2.3 (or newer) dylib’s was enough to eliminate the crash.

I’ve filed a Bug Report so hopefully at some point Unity starts shipping with the new version of OIDN which fixes this.

We dumped 2-3 weeks of frustrating troubleshooting into this so I hope this post maybe helps save someone from a similar ordeal.

1 Like