Native Plugins / Message Bridge To / From Unity from Swift / ObjC

Unity Engine
, ,
1

Hi, I am trying to do a few wonky things to accomplish something specific but simple.

I would like two way communcation between Unity and some Native SwiftUI views.

I can get SwiftUI Views rendering in tandem with Unity by injecting a Scene (that contains a window group) Reference into UnityPolySpatialApp, and manually opening it by name in the init method.

I then get to the point where there seems to be two options for passing messages to Unity from Swift.

  1. Using unityFrameWork.sendMessageToGO using a snippet like the following, and a Gameobject living in a loaded scene with the appropriately named monobehaviour / method.
static func sendMessageToUnity(message: String) {
        let gameObjectName = "SwiftBridge" 
        let methodName = "ReceiveMessageFromNative"
        let unityFrameworkInstance = UnityFramework.getInstance()  // Get the UnityFramework instance

        unityFrameworkInstance?.sendMessageToGO(
            withName: gameObjectName,
            functionName: methodName,
            message: message
        )
    }

I get this buildtime error trying to run this. Undefined symbol: _OBJC_CLASS_$_UnityFramework

Is it not possible to call methods on UnityFramework from the Unity-VisionOS target? I am a bit naive when it comes to XCode project configuration

  1. The second option, which I will likely need to use for more complicated two-way communication is to use Native Plugins and call into objective-c from c# land.

I have an ultra simple class-pair that looks like this:
obj-c header:

#import <Foundation/Foundation.h>

@interface SwiftBridge : NSObject
+ (void)sendMessageToNative:(NSString *)message;
@end

obj-c class:

#import "SwiftBridge.h"

@implementation SwiftBridge
+ (void)sendMessageToNative:(NSString *)message {
    NSLog(@"%@", message);
}
@end

c# wrapper:

using System.Runtime.InteropServices;

public static class SwiftBridge
{
    [DllImport("__Internal")]
    public static extern void sendMessageToNative(string message);
}

super simple monobehaviour calling said wrapper that lives in a scene.

using UnityEngine;

public class SwiftBridgeBehaviour : MonoBehaviour
{
    private void Awake()
    {
        SwiftBridge.sendMessageToNative("Hello from Unity, Native Land!!!!");
    }
}

This crashes immediately with this runtime error:
dyld[65510]: missing symbol called

Am I missing something with respect to setting up native plugins with VisionOS? It definitely makes it’s way into the build (see screenshot – it is in an ios directory, but I have tried Plugins/VisionOS directory etc…)

image
image1423×694 116 KB

The third, painful, option seems to be using UAAL. I run into issues even going through the template’s readme for iOS w.r.t #importing UnityFramework/UnityFramework.h even after setting everything up to work with the simulator.

Even better would be if there was something pre-built that I can use as part of the PS Swift Wrapper to send messages back and forth. That is not the case, no?

2

@AdamSanche
Note that there has been a discussion about uaal here which was targeting exactly the same thing:

(there is also a way to upvote uaal support in here: https://portal.productboard.com/unity/77-unity-visionos-roadmap/tabs/272-mr-immersive-features)

1 Like
3

So I got the native plugin to work, and saw the log I sent across. I was just missing a method export via extern c.

So the .m is now a .mm (c++) and looks like this

#import "SwiftBridge.h"

@implementation SwiftBridge
+ (void)sendMessageToNative:(NSString *)message {
    NSLog(@"%@", message);
}
@end

extern "C" {
    void SwiftBridge_SendMessageToNative(const char* message) {
        NSString* nsMessage = [NSString stringWithUTF8String:message];
        [SwiftBridge sendMessageToNative:nsMessage];
    }
}

And the C# wrapper updates to use the exported method name SwiftBridge_SendMessageToNative

I of course still need to solve the problem of bridging Unity-VisionOS → UnityFramework

4

Hello, @AdamSanche!
Could you please explain how you inject your WindowGroup with native UI elements?
Is there any better way of doing this except changing the UnityMain.swift code directly?

5

Yeah, we basically have a window group that we are injecting in a post process build step via code editing. We then have a separate framework getting included at build time that is exposed to both the UnityFramework and Unity-VisionOS targets that the SwiftUI stuff talks to to bridge messages.

I haven’t found a better way, with the current implementation other than editing the UnityMainApplication.swift file. If you add a Window Group to the main scene, you will see it appear in tandem with whatever unity content you have loaded. We actually fully replace some of the Unity Swift files in the MainApp folder now given the complexity of our edits, given you have to manage background state etc… Perhaps a bit brittle as we have to monitor changes with each PS update, so I am not entirely satisfied with it as a permanent long term solution. It’s kinda similar to how you have to manage native code on android.

1 Like
6

Do you have an example of what code you’re injecting where? Our team is running into a few issues getting this to work.
Thanks!!

7

There are so many nice native things that Polyspatial does not yet support!
This needs to be a standardized example maintained with full support by Unity.

1 Like
8

Just wanted to follow up as I’m curious about your solution.

We ended up making an interop framework which is working well.

Currently we inject our window at the end of var windowConfigurations: some Scene {....
But upon closing the window we get a nil error from swift runtime.

How are you managing state in your solution?

9

i’m kind of surprised no one has started asset store plugins for all the missing things that reality kit has access to that polyspatial doesn’t …

vision framework for example

10

Hey @AdamSanche did you make it to work as MR Unity App with SwiftUI?
If yes can you share the code and changes you did for the same?