Using Unity as a library in native iOS/Android apps

In some scenario developers using native platform technologies (like Android/Java & iOS/Objective C) want to include in their apps/games features powered by Unity (usually for 3D/2D Real Time Rendering), like AR experience, interaction with 3D models, 2D mini games and more.

Starting with Unity 2019.3.a2, we are introducing a new feature to use Unity as a library in native apps by integrating the Unity runtime components and your content in a native platform project. The Unity Runtime Library exposes controls to manage when and how to load/activate/unload within the native application.

Please note that using Unity as a library requires that you deeply understand the architecture of Android/Java & iOS/Objective C applications.

Limitations
While we tested many scenarios for Unity as a library hosted by a native app, Unity does not control anymore the lifecycle of the runtime, so we cannot guarantee it’ll work in all possible use cases.

For example:

  • Unity as a Library supports rendering only full screen, rendering on a part of the screen isn’t supported.
  • Loading more than one instance of the Unity runtime isn’t supported.
  • You may need to adapt 3rd party Plug-ins (native or managed) to work properly.

How it works
The build process overall is still the same, Unity creates the iOS Xcode and Android Gradle projects, but to enable this feature we changed the generated iOS Xcode and Android Gradle projects, they now have the following structure:

  • A library part (iOS framework and Android Archive (AAR) file) that includes all source & plugins
  • A thin launcher part that includes app representation data and runs library

Step by step explanations on how to include the iOS / Android library part into your native application when needed.
The rest of this post describes the Xcode / Android Gradle project changes.

iOS
Main functionality revolves around Xcode project to have additional target named UnityFramework. This target includes source/plugins and dependent frameworks and produce UnityFramework.framework file. Launch Screen, Xib, icons, Data, etc. stays in Unity-iPhone target. Unity-iPhone target will have single dependency on UnityFramework target.

UnityFramework provides simple API to manage player from native side to load/unload, pause /unpause when needed, send a message to game object and set bundle where player Data folder is located this way you can have framework file with everything Unity player needs to run in one place.

Change
Unity as a library feature brings some change to API and Xcode project structure. For the most use cases everything will work as before but if you developing or using plugins, custom BuildPostProcessor, PBXProject, CI there is risk these changes affecting your project even if you do not use Unity as a Library feature.

1. PBXProject
PBXProject.GetUnityTargetName and pbxProject->TargetGuidByName(“Unity-iPhone”) are obsolete. Instead use either pbxProject->GetUnityFrameworkTargetGuid() (for source, plugins, dependent frameworks, source build options) or pbxProject->GetUnityMainTargetGuid().
Any use of these obsolete functions will throw an exception.

// Obsolete
string targetGuid = proj.TargetGuidByName("Unity-iPhone");
string targetGuid = proj.TargetGuidByName(PBXProject.GetUnityTargetName());

// Instead call one of these
string targetGuid = proj.GetUnityFrameworkTargetGuid();
string targetGuid = proj.GetUnityMainTargetGuid();

How to support both old and new code paths in your package or custom build postropressor ?

  • Rely on plugin importer capabilities whenever possible. Egz. to specify additional frameworks
  • Use reflection:
string mainTargetGuid;
string unityFrameworkTargetGuid;
       
var unityMainTargetGuidMethod = proj.GetType().GetMethod("GetUnityMainTargetGuid");
var unityFrameworkTargetGuidMethod = proj.GetType().GetMethod("GetUnityFrameworkTargetGuid");
               
if (unityMainTargetGuidMethod != null && unityFrameworkTargetGuidMethod != null)
{
    mainTargetGuid = (string)unityMainTargetGuidMethod.Invoke(proj, null);
    unityFrameworkTargetGuid = (string)unityFrameworkTargetGuidMethod.Invoke(proj, null);
}
else
{
    mainTargetGuid = proj.TargetGuidByName ("Unity-iPhone");
    unityFrameworkTargetGuid = mainTargetGuid;
}

2. ProjectCapabilityManager
ProjectCapabilityManager is now accepting guid of a target.

3. xcodebuild
Some build settings have suffixes to specify exact target, they are: *_APP for application target and *_FRAMEWORK for framework target. When building with xcodebuild use suffixed version for:

  • PRODUCT_NAME → PRODUCT_NAME_APP
  • PROVISIONING_PROFILE → PROVISIONING_PROFILE_APP
  • PROVISIONING_PROFILE_SPECIFIER → PROVISIONING_PROFILE_SPECIFIER_APP
  • OTHER_LDFLAGS → OTHER_LDFLAGS_FRAMEWORK

Example
Example on how to integrate Unity as a Library into native iOS application.

Android
With Unity as a Library, the structure of the Android Gradle project has changed. If you are using a custom mainTemplate.gradle or AndroidManifest.xml, these changes might affect your project or unity plugin. Previously, Unity used a single gradle module to build an Android app. Now, Unity creates a gradle project with two modules - a unityLibrary module and a launcher module.

Change
The unityLibrary module contains the Unity runtime and project data. This module is a library that can be easily integrated into any other gradle project. It allows embedding Unity into existing native Android applications. The launcher module contains all the icons and the application name. It is a simple Android application that launches Unity.

1. Gradle Project Structure
Unity as a Library have new gradle project structure that better match best practices of android studio project structure: a root folder that includes some sub-projects that can be built and used individually. This makes importing a unity android studio project much easier.


2. Gradle template workflows
Gradle templates describe and configure how to build the android app with gradle. Each gradle template represents a gradle project. Gradle projects can include and depend on other gradle projects.

Unity as a Library gradle templates look like this:

  • baseProjectTemplate.gradle - contains configuration that is shared between all other templates/gradle projects (repositories and the dependency on the android gradle plugin);
  • launcherTemplate.gradle - contains instructions how to build the android application (bundling, signing, apk splitting). Depends on the unityLibrary project. Outputs either an APK or an app bundle;
  • mainTemplate.gradle - contains instructions how to build unity as a library. Outputs a .aar. Users can provide a custom template to override the Unity template (editor setting);
  • libTemplate.gradle - works the same way as previously.

3. Manifest workflows
Unity as a Library changes the way unity android application manifests work. Previously, there was a single AndroidManifest.xml that was used to provide the android app with icons, activities, permissions and other settings.

With the new system, we have two manifest files instead of one:

  • LauncherManifest.xml - responsible for icons, app name, starting activity (and its intents), install location, supported screen sizes, setting ‘isGame’;
  • LibraryManifest.xml - responsible for declaring the unity activity, permissions, theme used by the unity activity, vr modes, vr performance, making the activity non-resizable (for VR), setting max aspect ratio, reacting to configuration changes, orientations, launch modes, android UI hardware acceleration, used features (like gamepad or graphics API), notch support. This manifest can be overridden by providing a custom AndroidManifest.xml inside the Plugins/Android folder.

4. IUnityPlayerLifecycleEvents
Since 2019.3.b4+ IUnityPlayerLifecycleEvents provides a way to react to two important lifecycle events of Unity Player:

  • Unload - IUnityPlayerLifecycleEvents.onUnityPlayerUnloaded will be called when unity player will be unloaded though Application.Unload or UnityPlayer.unload(). (UnityPlayer is in unloaded/pause state atm.)
  • Quit - IUnityPlayerLifecycleEvents.onUnityPlayerQuitted is called when Unity player did quit. Please note process where unity was running will be killed after this call.
    You can pass instance of IUnityPlayerLifecycleEvents to UnityPlayer constructor or override methods in subclasses of UnityPlayer or UnityPlayerActivity

Example
Example on how to integrate Unity as a Library into native Android application.

18 Likes

Thanks for the detailed description, excited to try this out!
Are there any plans to also officially support a similar workflow on UWP and in particular for holographic apps?
I understand there are unofficial ways/hacks to embed Unity in native UWP apps, but I’m wondering if there will also be an officially supported workflow.

Many Thanks!

1 Like

This is a great feature to see officially supported! Now I can get rid of my custom Gradle file which was doing the same thing.

One request: It would be nice to see the sourceCompatibility and targetCompatibility values of the unityLibrary module set as low as possible to ensure compatibility with a wider range of applications. It looks like they could be 1.6 without any warnings/errors. (1.5 is technically achievable without code changes but it is deprecated)

The plugin fails to run unless the “game_view_content_description” Android String resource is defined. Why is that string required?

How can unity as a library be used in reactnative?

3 Likes

That string is required by Google as an alternative text for the game view when using accessibility features.

It is defined in values/strings.xml. If you are seeing issues with regard to it - please submit a bug report. Thanks!

1 Like

Cool, I just wanted to know why - it isn’t a problem for me to define it.

Another issue is that Android x86 support is gone in 2019.3 which makes it difficult to run any app that uses the plugin in an emulator. (the ARM emulators are very slow because they ware running on x86 machines mostly) So now my app just got a lot harder to iterate on. ;(

Any chance that the x86 removal can be re-thought?

1 Like

There are no plans for bringing back x86 at the moment.

I previously posted on the Android example thread but it might be worth posting here with more info since there seems to be more momentum here.

I was tinkering with the example Android app and I noticed that when any of the Unload buttons are used, the app terminates. I see that the unload is being attempted by calling the finish() method on the Unity activity. I would expect that this would just destroy the activity in question and not quit the app. I verified this by adding a new, non-Unity activity to the example app and a new button that would open the newly created activity from the main activity. The new activity does finishes without quitting the app.

It appears that the plugin quits (or maybe crashes?) the app when it is destroyed. @Yury-Habets Can you investigate?

Edit: not sure if it helps but here are the logs from the device:
Logs

06-13 11:11:46.307 532-580/system_process I/LoggerMetricsFactoryImpl: PhoneWindowManager:ActivityChanged:com.unity.mynativeapp=1.0;CT;1:NR
06-13 11:11:46.312 6984-6984/com.unity.mynativeapp I/[MALI][Gralloc]: [-]r_hnd:0xf7ad6960, fd:56, ion_hnd(0x5), req_format(0x1), int_fmt(0x1) byte_stride(5120), flags(0x4), usage(0x933), size(4096000), alloc_size(4096000) 1280(1280)X800(800) pid(6984) sec (0)
06-13 11:11:46.312 532-560/system_process I/ActivityManager: Displayed com.unity.mynativeapp/.MainActivity: +63ms
06-13 11:11:46.312 6984-6984/com.unity.mynativeapp I/HAL: loaded HAL id=gralloc path=/system/lib/hw/gralloc.mt8163.mali.so hmi=0x0 handle=0xe3ac0004
06-13 11:11:46.312 6984-6984/com.unity.mynativeapp I/[MALI][Gralloc]: [-]r_hnd:0xf7aceb90, fd:60, ion_hnd(0x6), req_format(0x1), int_fmt(0x1) byte_stride(5120), flags(0x4), usage(0x933), size(4096000), alloc_size(4096000) 1280(1280)X800(800) pid(6984) sec (0)
06-13 11:11:46.312 6984-6984/com.unity.mynativeapp I/HAL: loaded HAL id=gralloc path=/system/lib/hw/gralloc.mt8163.mali.so hmi=0x0 handle=0xe3ac0004
06-13 11:11:46.313 196-1618/? I/[MALI][Gralloc]: [-]hnd:0x7f999983c0, fd:54, ion_hnd(0x8), req_format(0x1), int_fmt(0x1) byte_stride(5120), flags(0x4), usage(0x933), size(4096000), alloc_size(4096000) 1280(1280)X800(800) pid(196) sec (0)

--------- beginning of vitals
06-13 11:11:46.314 532-631/system_process I/Vlog: performance:warm_app_warm_transition_launch_time:fgtracking=false;DV;1,key=com.unity.mynativeapp;DV;1,Timer=62.0;TI;1,unit=ms;DV;1,metadata=com.unity.mynativeapp/.MainActivity!{“d”#{“app_version”#“1”}"m"#{"context"#"S1T0A1D0I0L0E0M99"“prev”#“com.unity.mynativeapp/.MainUnityActivity”}};DV;1:HI
06-13 11:11:46.314 532-631/system_process I/Vlog: performance:warm_activity_launch_time:fgtracking=false;DV;1,key=com.unity.mynativeapp;DV;1,Timer=62.0;TI;1,unit=ms;DV;1,metadata=com.unity.mynativeapp/.MainActivity!{“d”#{“app_version”#“1”}"m"#{"context"#"S1T0A1D0I0L0E0M99"“prev”#“com.unity.mynativeapp/.MainUnityActivity”}};DV;1:HI
06-13 11:11:46.315 218-295/? D/AALService: enableAALEvent: 0 → 0
06-13 11:11:46.315 196-1618/? I/[MALI][Gralloc]: [-]hnd:0x7f99998460, fd:55, ion_hnd(0xd), req_format(0x1), int_fmt(0x1) byte_stride(5120), flags(0x4), usage(0x933), size(4096000), alloc_size(4096000) 1280(1280)X800(800) pid(196) sec (0)
06-13 11:11:46.316 6984-6984/com.unity.mynativeapp I/[MALI][Gralloc]: [-]r_hnd:0xf7f732f0, fd:63, ion_hnd(0x2), req_format(0x1), int_fmt(0x1) byte_stride(5120), flags(0x4), usage(0x933), size(4096000), alloc_size(4096000) 1280(1280)X800(800) pid(6984) sec (0)
06-13 11:11:46.317 6984-6984/com.unity.mynativeapp I/HAL: loaded HAL id=gralloc path=/system/lib/hw/gralloc.mt8163.mali.so hmi=0x0 handle=0xe3ac0004
06-13 11:11:46.319 6984-6984/com.unity.mynativeapp D/FaxInstantSearchPwCallback: setInstantSearchActivityInfo()
06-13 11:11:46.319 6984-6984/com.unity.mynativeapp D/FaxInstantSearchPwCallback: setInstantSearchActivityInfo()
06-13 11:11:46.319 196-622/? I/[MALI][Gralloc]: [-]hnd:0x7f998b7db0, fd:64, ion_hnd(0xf), req_format(0x1), int_fmt(0x1) byte_stride(5120), flags(0x4), usage(0xb00), size(4096000), alloc_size(4096000) 1280(1280)X800(800) pid(196) sec (0)
06-13 11:11:46.322 196-622/? I/[MALI][Gralloc]: [+]hnd:0x7f998b7db0, fd:54, ion_hnd(0x8), req_format(0x1), int_fmt(0x1) byte_stride(3200), flags(0x4), usage(0xb00), size(4096000), alloc_size(4096000) 800(800)X1280(1280) pid(196) sec (0)
06-13 11:11:46.323 6984-7012/com.unity.mynativeapp I/HAL: loaded HAL id=gralloc path=/system/lib/hw/gralloc.mt8163.mali.so hmi=0x0 handle=0xe3ac0004
06-13 11:11:46.324 6984-7012/com.unity.mynativeapp I/[MALI][Gralloc]: [+]r_hnd:0xf7ad6960, fd:49, ion_hnd(0x2), req_format(0x1), int_fmt(0x1) byte_stride(3200), flags(0x4), usage(0xb00), size(4096000), alloc_size(4096000) 800(800)X1280(1280) pid(6984) sec (0)
06-13 11:11:46.329 532-631/system_process I/Vlog: performance:SkippedVsyncsBucket1:fgtracking=false;DV;1,key=com.unity.mynativeapp;DV;1,Counter=5.0;CT;1,unit=count;DV;1,metadata=com.unity.mynativeapp/.MainActivity!{“d”#{“app_version”#“1”}"m"#{"context"#"S1T0A1D0I0L0E0M99"“prev”#“com.unity.mynativeapp/.MainUnityActivity”}};DV;1:HI
06-13 11:11:46.329 532-631/system_process I/Vlog: performance:SkippedVsyncsBucket2:fgtracking=false;DV;1,key=com.unity.mynativeapp;DV;1,Counter=1.0;CT;1,unit=count;DV;1,metadata=com.unity.mynativeapp/.MainActivity!{“d”#{“app_version”#“1”}"m"#{"context"#"S1T0A1D0I0L0E0M99"“prev”#“com.unity.mynativeapp/.MainUnityActivity”}};DV;1:HI
06-13 11:11:46.329 532-631/system_process I/Vlog: performance:FrameDropTotal:fgtracking=false;DV;1,key=com.unity.mynativeapp;DV;1,Counter=7.0;CT;1,unit=count;DV;1,metadata=com.unity.mynativeapp/.MainActivity!{“d”#{“app_version”#“1”}"m"#{"context"#"S1T0A1D0I0L0E0M99"“prev”#“com.unity.mynativeapp/.MainUnityActivity”}};DV;1:HI
06-13 11:11:46.330 532-631/system_process I/Vlog: performance:FramesTotal:fgtracking=false;DV;1,key=com.unity.mynativeapp;DV;1,Counter=347.0;CT;1,unit=count;DV;1,metadata=com.unity.mynativeapp/.MainActivity!{“d”#{“app_version”#“1”}"m"#{"context"#"S1T0A1D0I0L0E0M99"“prev”#“com.unity.mynativeapp/.MainUnityActivity”}};DV;1:HI
06-13 11:11:46.330 532-631/system_process I/Vlog: performance:FirstFramesTotal:fgtracking=false;DV;1,key=com.unity.mynativeapp;DV;1,Counter=2.0;CT;1,unit=count;DV;1,metadata=com.unity.mynativeapp/.MainActivity!{“d”#{“app_version”#“1”}"m"#{"context"#"S1T0A1D0I0L0E0M99"“prev”#“com.unity.mynativeapp/.MainUnityActivity”}};DV;1:HI
06-13 11:11:46.330 218-295/? D/AALService: enableAALEvent: 0 → 0
06-13 11:11:46.351 218-295/? D/AALService: enableAALEvent: 0 → 0
06-13 11:11:46.367 218-295/? D/AALService: enableAALEvent: 0 → 0
06-13 11:11:46.383 218-295/? D/AALService: enableAALEvent: 0 → 0
06-13 11:11:46.400 218-295/? D/AALService: enableAALEvent: 0 → 0
06-13 11:11:46.416 218-295/? D/AALService: enableAALEvent: 0 → 0
06-13 11:11:46.433 218-295/? D/AALService: enableAALEvent: 0 → 0
06-13 11:11:46.449 218-295/? D/AALService: enableAALEvent: 0 → 0
06-13 11:11:46.465 218-295/? D/AALService: enableAALEvent: 0 → 0
06-13 11:11:46.482 218-295/? D/AALService: enableAALEvent: 0 → 0
06-13 11:11:46.498 218-295/? D/AALService: enableAALEvent: 0 → 0
06-13 11:11:46.515 218-295/? D/AALService: enableAALEvent: 0 → 0
06-13 11:11:46.533 218-295/? D/AALService: enableAALEvent: 0 → 0
06-13 11:11:46.544 218-295/? D/AALService: enableAALEvent: 0 → 0
06-13 11:11:46.556 196-196/? I/[MALI][Gralloc]: [-]hnd:0x7f999215c0, fd:49, ion_hnd(0x5), req_format(0x1), int_fmt(0x1) byte_stride(3200), flags(0x4), usage(0xb00), size(4096000), alloc_size(4096000) 800(800)X1280(1280) pid(196) sec (0)
06-13 11:11:46.559 196-196/? I/[MALI][Gralloc]: [-]hnd:0x7f998f0ab0, fd:56, ion_hnd(0x3), req_format(0x1), int_fmt(0x1) byte_stride(5120), flags(0x4), usage(0x933), size(4096000), alloc_size(4096000) 1280(1280)X800(800) pid(196) sec (0)
06-13 11:11:46.563 218-295/? D/AALService: enableAALEvent: 0 → 0
06-13 11:11:46.651 532-1247/system_process I/WindowState: WIN DEATH: Window{3d0c9d1a u0 com.unity.mynativeapp/com.unity.mynativeapp.MainActivity}
06-13 11:11:46.651 196-239/? I/[MALI][Gralloc]: [-]hnd:0x7f99921240, fd:65, ion_hnd(0x10), req_format(0x1), int_fmt(0x1) byte_stride(5120), flags(0x4), usage(0xb00), size(4096000), alloc_size(4096000) 1280(1280)X800(800) pid(196) sec (0)
06-13 11:11:46.653 196-239/? I/[MALI][Gralloc]: [-]hnd:0x7f99934b20, fd:59, ion_hnd(0x7), req_format(0x1), int_fmt(0x1) byte_stride(3200), flags(0x4), usage(0xb00), size(4096000), alloc_size(4096000) 800(800)X1280(1280) pid(196) sec (0)
06-13 11:11:46.656 218-295/? D/AALService: enableAALEvent: 0 → 0
06-13 11:11:46.656 196-196/? I/[MALI][Gralloc]: [-]hnd:0x7f998b7db0, fd:54, ion_hnd(0x8), req_format(0x1), int_fmt(0x1) byte_stride(3200), flags(0x4), usage(0xb00), size(4096000), alloc_size(4096000) 800(800)X1280(1280) pid(196) sec (0)
06-13 11:11:46.661 532-26020/system_process I/WindowState: WIN DEATH: Window{23b4e31f u0 com.unity.mynativeapp/com.unity.mynativeapp.MainUnityActivity}
06-13 11:11:46.652 6984-7027/? I/Kernel: [1381706.680685] <3> (3)[7027:SendFrameDataTa][name:ion&][ION]warning: release handle @ client destroy: handle=ffffffc02a9b4500, buf=ffffffc01fb64600, ref=2, size=4096000, kmap=0
06-13 11:11:46.652 6984-7027/? I/Kernel: [1381706.680732] <3> (3)[7027:SendFrameDataTa][name:ion&][ION]warning: release handle @ client destroy: handle=ffffffc019cda900, buf=ffffffc01fb65680, ref=1, size=4096000, kmap=0
06-13 11:11:46.652 6984-7027/? I/Kernel: [1381706.682188] <3> (3)[7027:SendFrameDataTa][name:ion&][ION]warning: release handle @ client destroy: handle=ffffffc03628f200, buf=ffffffc054f66900, ref=7, size=3833856, kmap=0
06-13 11:11:46.671 224-224/? I/Zygote: Process 6984 exited due to signal (11)
06-13 11:11:46.679 532-1261/system_process I/ActivityManager: Process com.unity.mynativeapp (pid 6984) has died
06-13 11:11:46.679 532-1261/system_process D/ActivityManager: cleanUpApplicationRecord – 6984
06-13 11:11:46.680 532-1261/system_process W/ActivityManager: Force removing ActivityRecord{314ca5b u0 com.unity.mynativeapp/.MainActivity t746}: app died, no saved state
06-13 11:11:46.680 532-631/system_process W/Eve: MEMORY_LEVEL: LMK [6984,0]
top=com.unity.mynativeapp/.MainActivity launching=false
prev=com.unity.mynativeapp/.MainUnityActivity screen_on=true thermal=0
adb=true downloading=false
memory=0 available=623972KB free=166716KB
06-13 11:11:46.682 532-1309/system_process I/PackageRecencyCallback: Queuing notifcation to package: ComponentInfo{com.amazon.kindle.cms/com.amazon.kindle.cms.MaintenanceService$PackageRecencyListener}
06-13 11:11:46.683 532-631/system_process W/Eve: Lmk did not kill com.unity.mynativeapp

It looks like the process dies die to signal 11. (SEGFAULT)

1 Like

More info.

The above issue appears when I used the Unload button from within the overridden Unity activity.

The process seems to just kill itself when the unload button is pressed from the main activity:
06-13 11:28:49.179 7746-7746/com.unity.mynativeapp I/Process: Sending signal. PID: 7746 SIG: 9

The kill() call was there in older Unity versions. Which one are you using?

2019.3.0a5. If I look inside the UnityPlayer class, the kill call is still there.

Looks like one of my coworkers also replied to the Android thread about this: Integration Unity as a library in native Android app - Unity Engine - Unity Discussions

Edit: @estradap Put in a bug report for this: 1163573

I have a question about how this works.

Is it possible to have a native application that controls everything about ads (dependencies) and then communicate from Unity to the ads application to show banners, interstitial and RewVideos?

Or this is just to render Unity while another native application is active?

Thanks!

Wonderful update, any word on if there will be a solution to integrate with Xamarin?

Thanks for spotting up this issue, we going to investigate the case and update/fix example/unity accordingly.

We did not investigated integration path to Xamarin. Found this https://unitylist.com/p/n2u/Xamarin-With-Unity-App example seems to be a positive prove

Hard to say without having all the details but i do not see reasons why it would be impossible.

Hi PavelLu! Thanks for the great update. I know you wrote, ‘Loading more than one instance of the Unity runtime isn’t supported.’ The thing that I did not understand does that mean it is still impossible to integrate an AR game to native IOS app?

It is possible now to integrate Unity project with AR to native iOS app, follow iOS integration example and it should work.

Hi, I was wondering, would this work for tvOS as well?