Interop to native .mm not working when making xcode build in release (2019.3.10f)

Unity Engine
, ,
1

We are using the newer Generate OSX Xcode Project facility, in Unity 2019.3.10f. We are also using the ability to attach a .mm file to the XCode project to facilitate our GameCenter access needs, using native code. We use normal C#/C Interop features to talk to that native code. This works fine for IOS in both debug/release and also works fine for OSX, when in debug.

However, when generating a release build, the interop appears to break. We are definitely not generating a Development build when using the release config in XCode on OSX - doing so gives errors on the console when interop cannot find the C Functions it is looking for. When we turn off the development flag in unity build settings and use a release build of the resulting XCode project, those console errors go away. However, while they go away, the C functions are never called and, in fact, the C# code that calls them using Interop will crash internally and abort that current C# processing thread, albeit silently.

We are using a dll name of “__Internal” for all uses of the .mm native code function calls, which appears to work for IOS debug/release, IOS debug/release and OSX debug, but I suspect this doesn’t work for release on OSX. What should this value be for OSX release?

If it is supposed to be “__Internal”, can Unity give any other details as to why the interop fails when building an OSX non development unity build, and release build in XCode? And what we can do about it?

2

Where did you put those .mm files in? Are they in your Unity Project? Does this also crash when you’re not using “Generate Xcode project” setting?

3

OK, so, I’m the engineer having this issue.

So the .mm files are in a directory off Assets/Plugins/OSX (the IOS versions are placed in Assets/Plugings/IOS).

Unity usually has a method of stating which builds these .mm files are attached to, treating them as DLL’s, however while this works for IOS builds (the appropriate .mm files get included in the generated XCode project as well as required frameworks), this approach doesn’t work at all for OSX XCode project generation. I get around this by actually editing the generated project using a post processing step, where I load up the generated XCode project and inject my files into directly, and set frameworks and bundle ID’s and such.

This works. We can then build the OSX project in XCode as a debug release and the C functions inside of the native .mm files are accessed using Interop just fine.

It’s only when we are running a release build (which is what the archive feature does by default inside of XCode) that Interop suddenly stops working. As mentioned - we made sure that the XCode project we generate from inside Unity does NOT have the development flag set, because there is very definitely an Interop issue when we generate a development XCode project and then try and build it as release in XCode; we get a ton of rolling red errors on the console in the app when accessing any of the native C functions. When we build a release build in XCode of an OSX project generated from Unity WITHOUT the development flag set, these visual error messages go away, however, Interop still doesn’t work, and silently crashes the C# thread on which they are called.

Hence looking for help here.

This is kind of critical to us here, and we need to work how to get this to work.

4

Argh! This is actually a known bug in Unity. We fixed it in 2019.4.4f1. Are you able to upgrade your Unity version to 2019.4.4f1?

I suspect the whole interop thing with debug/release is having issues because when you edit generated Xcode project, those .mm files get compiled into the app executable, rather than GameAssembly.dylib (which is where all the other generated C++ code goes). [DllImport(__Internal)] is meant to only invoke code that’s in the same GameAssembly.dylib file.

5

Ok, so… it does appear to work fine with IOS, athough I do notice that what is generated for IOS is very different from what is generated for OSX; none of the native classes for the C# appear to be in the OSX build, like they are in the IOS one.

And this does work for Debug. It’s only release that has this issue, which would lead me to believe there’s some setting for non development builds that’s not set correctly, to make interop work. My suspicion is that the development ON flag and release build, while handling the error reporting correctly, is not actually setting the interop capability correctly. The end result being we don’t see errors being reported when development bits are not set and we build release in XCode, but the actual problems with interop not working, when you have a development build trying to be compiled to release in XCode are still there.

6

Correct, iOS links all the code into the executable, whereas macOS builds use several dynamic libraries (one for the engine - UnityPlayer.dylib, one for IL2CPP generated code - GameAssembly.dylib and one for the entry point - the executable.

Did you mark your methods with attribute visibilty default in the .mm file?

Umm… by the way, you are using IL2CPP scripting backend right?

7

No. The whole point of the exercise is that I shouldn’t have to. The entire point here is that I should be able to just mark files for attachment in the XCode OSX project (like I do in IOS) and their functions be available for interop inside of C#, using DLLIMPORT.

I don’t on IOS, and since OSX now generates XCode projects, it follows that I shouldn’t have to on OSX either. And indeed, in debug, those functions are available, so the assumption is sound. It’s only in release it is not, which suggests a bug in how interop connections are generated in OSX projects for release.

8

Oh, ignore everything I said above regarding GameAssembly.dylib then. It doesn’t apply to Mono scripting backend.

I’m not exactly sure whether that is supported to be perfectly blunt with you. I don’t think anyone ever tested this. This workflow has only really been defined for IL2CPP scripting backend: Unity - Manual: Use IL2CPP with macOS

Looking at the source code for Mono, doing [DllImport(“__Internal”)] is really the same as doing [DllImport()]: mono/mono/metadata/loader.c at unity-2019.4-mbe · Unity-Technologies/mono · GitHub

Following that, Mono calls dlsym with the executable module handle and your function name. For that to succeed, the function has to be “exported” from the binary.

There are two ways to export symbols on Mac:

  1. Pass -fvisibility=default to the compiler (as opposed to passing -fvisibility=hidden). This makes ALL the symbols in the executable to be exported.
  2. Annotate your functions with “attribute((visibility(“default”)))”. This will selectively export them.

I suspect that either the -fvisibility flag is different in release and debug builds, or in Release builds your function gets stripped away because the compiler thinks it is unused. That’s what I’d check next.

1 Like
9

I’d also try adding “–export-dynamic” to the linker flags. I might make it not strip away this “unused” symbol.

1 Like
10

That’s terrific feedback. I would actually go directly in and try this, however we have found that 2019.4.4f does actually fix this issue, and assuming nothing else is broken in that build, we are going to migrate to that.

11

Argh.

So yeah, this is NOT fixed in 2019.4.4f1.

In fact, nothing is fixed. The auto inclusion of the .mm file does not work, despite the inspector for the .mm file pane having changed a bit to look like it should be including it. Doesn’t matter if you do a development build or not, it’s not there in the XCode project.

I can still do it in the post process, and like 2019.3.10, Development/Debug interop works, and in NonDevelopment / release, it does not, and fails silently.

Unfortunately nothing has been fixed in 2019.4.4f1 at all, I’m afraid.

Question -

Pass -fvisibility=default to the compiler (as opposed to passing -fvisibility=hidden). This makes ALL the symbols in the executable to be exported.

Where do I do this in the project files? I am opening the project file inside of the post processor

e.g.

            // Initialize PbxProject
            string projectPath = pathToBuiltProject + "/project.pbxproj";
            PBXProject pbxProject = new PBXProject();
            pbxProject.ReadFromFile(projectPath);

            string targetGuid = pbxProject.TargetGuidByName("blahblah");

            pbxProject.SetTeamId(targetGuid, "XXXXXXXX");
            pbxProject.SetBuildProperty(targetGuid, "PRODUCT_BUNDLE_IDENTIFIER", "com.tnbgames.blahblah");

            string pathToMMFile = Application.dataPath + "/Plugins/OSX/GiantsGameCenter.mm";
            string projectNewPath = "Plugins/OSX/GiantsGameCenter.mm";

            string fileGuid = pbxProject.AddFile(pathToMMFile, projectNewPath);
            pbxProject.AddFileToBuild(targetGuid, fileGuid);
            File.WriteAllText(projectPath, pbxProject.WriteToString());


            ProjectCapabilityManager manager = new ProjectCapabilityManager(
            projectPath,
            "blah_blah.entitlements",
            "blah blah"
            );

            manager.AddiCloud(true, false, null);
            manager.AddGameCenter();
            manager.WriteToFile();

I’m imagining it looks something like
pbxProject.AddBuildProperty(targetGuid, “OTHER_LDFLAGS”, “-all_load”);

but I don’t know the exact syntax - got any idea what that would look like? The documentation on this is a bit sparse, to say the least.

Failing that, where does this

  1. Annotate your functions with “attribute((visibility(“default”)))”. This will selectively export them.

actually go? Are we talking about in the .mm file, where the externed C functions are defined?

12

The fix was targeted at IL2CPP scripting backend. I’m pretty sure it’s fixed there. Sorry, when I wrote that I thought you were using IL2CPP scripting backend because as I said, this feature wasn’t meant for Mono scripting backend at all.

I highly recommend you don’t actually use that flag in production.

Yes. You pretty much do this:

extern "C" __attribute__((visibility("default"))) void DoSomething()
{
    // actual code
}

You might also have to specify “export_dynamic” flag in OTHER_LDFLAGS. I’m not exactly sure how to add that to Xcode project either (and I don’t have access to a Mac at this very moment) but I would suggest trying to add that flag via UI, saving the project, opening up the file and checking where it added it. Hopefully you’re able to work backwards from that to see how to add it via Xcode API.

13

OK. Thought you’d like some feedback.

    extern "C" __attribute__((visibility("default"))) void DoSomething()
    {
        // actual code
    }

Works like a champ. I wrapped my C functions in the .mm file, and they now work in release mode! So yay! Well done…

However…

When I do an archive of the OSX build (which is building release), back to the same old problem.

I did figure this one out on my own though.

Archiving has a Strip step in it. This removes all those C functions, because the strip system doesn’t know they are called from embedded C#, dynamically loading the lib. There are project settings in XCode that enable you to turn off post process strip functions are in the Deployment settings of Build Settings (easiest if you just search for “strip” in the build settings.

If you turn off “Deployment post processing” for all settings, then the strip set doesn’t happen in the archive build steps, and the functions work great in a distributed build.

I’m not attempting to figure out how to force this setting inside of the XCode Project editing I do, post Unity generation of an XCode project.

So yeah, this last one is not an Unity issue, although once I figure out what the setting is, I would advise Unity to force set this in the OSX project by default, so no one else runs into this.

14

Thanks!

15

Right, so the PBXProject class appears to be incomplete and quite a lot of what it does offer doesn’t work - not entirely surprised, that seems how Unity is these days - so the way I got this to work is like this.

public class XcodeSettingsPostProcesser
{

    [PostProcessBuildAttribute(0)]
    public static void OnPostprocessBuild(BuildTarget buildTarget, string pathToBuiltProject)
    {
        // Stop processing if targe is NOT iOS
        if (buildTarget == BuildTarget.StandaloneOSX)
        {
            // Initialize PbxProject
            string projectPath = pathToBuiltProject + "/project.pbxproj";
            PBXProject pbxProject = new PBXProject();
            pbxProject.ReadFromFile(projectPath);

string projectAsString = pbxProject.WriteToString();
            projectAsString = projectAsString.Replace("SDKROOT = macosx;", "SDKROOT = macosx;\n\t\t\t\t\tSTRIP_INSTALLED_PRODUCT = NO;");

            File.WriteAllText(projectPath, projectAsString);
        }
    }
}

It’s the STRIP_INSTALLED_PRODUCT = NO; part that stops the strip process on the archive step.

Once that is in place inside the Unity post process class, it all works.