iOS and MultiDisplay - does it work at all?

Unity Engine
, ,
1

Hello,

I am evaluating Unity as we would like to replace our SceneKit/WebGL engines with a single Unity driven engine.
An external display is essential to the iOS product.

I notice the Multi Display iOS generated code is old looking (UIScreen vs UIScene).

All I get (after some tweaking) is a black screen. Trying the Unity-iPhone app on its own yields the same result (without my tweaks - the tweaks were necessary as I am embedding Unity).

I have been unable to find a sample iOS project that displays a separate camera on a second display. Is there such a thing? I would really like to know if this is possible by running a sample app that was known to be good.

Thanks,
-Ed

3

Thank you for your reply Robert,

I’m a bit confused. The documentation lists iOS and Android as supported. And there is code generated that is intended for multiple displays.
Airplay is the proper way to extend, but the default is mirroring which is not good enough (contains other UI which is not needed for a presentation like external display), but the solution it to add a custom view (which the Unity generated code is setting up)

I did already search Github - this seems like a niche that most people don’t go into, no luck there.

I’m finding it a bit strange that the code is in there, and I can see the various renderers and screens for the external display in the debugger. It just feels like broken code, not code that wasn’t intended for this purpose (and also as I mentioned the docs claiming support)

  • Ed
1 Like
4

Replying to myself.

It works. Though I am not sure it changes things, it seems to be a high risk for this working in the future.

This thread had pointers: URP + Multiple Displays

Camera.targetDisplay needs to be set to display 2 for the external screen camera.
This doc Unity - Scripting API: Camera.targetDisplay claims it has no affect on iOS and Android, but that is not the case.

Third step is iOS has moved to a UIScene based architecture, and the old way of setting a UIScreen to a UIWindow was deprecated and now broken (or now broken when targeting iOS 16).

I hacked -[DisplayManager createView:showRightAway:] to use the UIWindowScene:

- (void)createView:(BOOL)useForRendering showRightAway:(BOOL)showRightAway;
{
#if !PLATFORM_VISIONOS
    NSAssert(_screen != [UIScreen mainScreen], @"DisplayConnection for mainScreen should be created with createWithWindow:andView:");
#endif
    if (_view == nil)
    {
#if !PLATFORM_VISIONOS
        UIWindowScene *externalScene;
        NSSet<UIScene *> *connectedScenes = [[UIApplication sharedApplication] connectedScenes];
        for (UIScene *scene in connectedScenes)
        {
            if ([scene isKindOfClass:[UIWindowScene class]])
            {
                if (((UIWindowScene *)scene).screen == _screen)
                {
                    externalScene = (UIWindowScene *)scene;
                    break;
                }
            }
        }


//        UIWindow* window = [[UIWindow alloc] initWithFrame: _screen.bounds];
        UIWindow* window = [[UIWindow alloc] initWithWindowScene:externalScene];

    #pragma clang diagnostic push
    #pragma clang diagnostic ignored "-Wdeprecated-declarations"
        // [UIWindow setScreen:] is deprecated in favor of [UIWindow setWindowScene:], but we are not yet scenes based
        // this API works perfectly fine for now, so we use it until we rewrite/modernize trampoline to be Scene-based
//        window.screen = _screen;
    #pragma clang diagnostic pop

        UIView* view = [(useForRendering ? [UnityRenderingView alloc] : [UIView alloc]) initWithFrame: _screen.bounds];
        view.contentScaleFactor = UnityScreenScaleFactor(_screen);
#else
        UIWindow* window = [[UIWindow alloc] init];
        UIView* view = [(useForRendering ? [UnityRenderingView alloc] : [UIView alloc]) init];
#endif

        [self createWithWindow: window andView: view];

        if (showRightAway)
        {
            [window addSubview: view];
            window.hidden = NO;
        }
    }
}

Perhaps this helps someone.

Hey Unity - I was really excited to be possibly using Unity, but I traversed this issue from posts since 2019. It’s quite a downer, since now my team will need to weigh the risks instead, and I was so looking forward to diving in porting our rather complicated and large visualization platform to Unity and enjoying building upon it using your nice editor.
If I can hack this, surely someone on your team could have done so since 2019?

Not trying to be mean, my use case might be niche - but just take it out of your docs and remove the code. Aside from wasting 2 days on what is normally trivial (external screens on iOS), it leaves an unprofessional feeling about your product, and also one of not being able to have faith that what I got working might simply break again in the future.

Cheers,
-Ed

5

which version of unity did you use for this solution?

6

This doc Unity - Scripting API: Camera.targetDisplay claims

omg this is indeed wrong - targetDisplay works perfectly fine on iOS.

Third step is iOS has moved to a UIScene based architecture, and the old way of setting a UIScreen to a UIWindow was deprecated and now broken (or now broken when targeting iOS 16).

this is not true. Or rather, unity by itself still does not use scenes (we are working on it though), so the code in trampoline is written for “no scenes are used”. When you are using scenes (as i understand you embed unity in your app that uses them) then yes, things might break exactly as you say because you mix two non-compatible technologies

I hacked -[DisplayManager createView:showRightAway:] to use the UIWindowScene:

yeah the code make sense, you can keep it. Alas i dont have a better solution to apply this automatically than in some build postprocess script (well, you can make it prettier by extracting function and tweaking EnsureDisplayIsInited to use your function instead of createView: