Multi display isn't recognize on AOSP Multi display scene

Unity Engine
, ,
1

I making some Device that configured 2 display output on AOSP platform. First display goes out HDMI(primary display) and other output is MiraCast (Android Wifi Direct mirroing)

So I made Simple sample app with 2 cameras to send screen each target display (HDMI, Miracast)

8998762--1239550--upload_2023-5-8_16-48-41.png
8998762--1239550--upload_2023-5-8_16-48-41.png420×630 38.2 KB

8998762--1239553--upload_2023-5-8_16-48-54.png
8998762--1239553--upload_2023-5-8_16-48-54.png417×658 37 KB

I tested Typical Android device (Samsung S21) with Smartview(Android mirroing), it works.

And Enable Virtual Display through Developer option → Simulate secondary displays , it works fine.
(HDMI output + Virtual simulated display)
Simuldate overlay virtual display info dump

DisplayDeviceInfo{“Overlay #1”: uniqueId=“overlay:1”, 1280 x 720, modeId 3, defaultModeId 3, supportedModes [

{id=3, width=1280, height=720, fps=60.000004}
], colorMode 0,
supportedColorModes [0], HdrCapabilities null,
density 213, 213.0 x 213.0 dpi, appVsyncOff 0, presDeadline 33333332, touch VIRTUAL, rotation 0,
type OVERLAY, state ON,
FLAG_PRESENTATION}

But with HDMI/Miracst scene, Unity cannot find secondary display device.
I add script and Logs on Camera gameobject to How many connected displays Unity can find.

public class ActivateDisplay : MonoBehaviour
{
    void Start ()
    {
        Debug.Log ("displays connected: " + Display.displays.Length);
            // Display.displays[0] is the primary, default display and is always ON, so start at index 1.
            // Check if additional displays are available and activate each.
        for (int i = 1; i < Display.displays.Length; i++)
            {
                Display.displays[i].Activate();
            }
    }
}

In Android Logs

  1. Start app :
    04-27 08:43:38.528 1114 1810 I ActivityTaskManager: START u0

{act=android.intent.action.MAIN cat=[android.intent.category.LAUNCHER] flg=0x10200000 cmp=com.UnityTechnologies.com.unity.template.urpblank/com.unity3d.player.UnityPlayerActivity bnds=[3059,548][3819,870]}
from uid 10100

  1. Unity get only Single display, not two

04-27 08:43:39.495 3927 3951 D Unity : ANativeWindow: (3840/2160) RequestedResolution: (0/0) RenderingResolution: (0/0) EGLSurface: (3840/2160)
04-27 08:43:41.539 3927 3951 D Unity : UnloadTime: 4.528698 ms
04-27 08:43:41.991 3927 3951 D Unity : Choreographer available: Enabling VSYNC timing
04-27 08:43:42.036 3927 3951 I Unity : displays connected: 1

and dump on Andorid frameworks, HDMI display and Miracast display are connected successfully.

partial result to adb shell dumpsys display

Display info dump below

Display Devices: size=2
DisplayDeviceInfo{“Built-in Screen”: uniqueId=“local:19260688233415425”, 3840 x 2160, modeId 1, defaultModeId 1, supportedModes [

{id=1, width=3840, height=2160, fps=60.240963}
], colorMode 0, supportedColorModes [0], HdrCapabilities android.view.Display$HdrCapabilities@40f16308, density 420, 300.0 x 300.0 dpi, appVsyncOff 1000000, presDeadline 16600000, touch INTERNAL, rotation 0, type BUILT_IN, address {port=1, model=0x446d7e704843}, state ON, FLAG_DEFAULT_DISPLAY, FLAG_ROTATES_WITH_CONTENT, FLAG_SECURE, FLAG_SUPPORTS_PROTECTED_BUFFERS}
mAdapter=LocalDisplayAdapter
mUniqueId=local:19260688233415425
mDisplayToken=android.os.BinderProxy@6e2bc72
mCurrentLayerStack=0
mCurrentOrientation=0
mCurrentLayerStackRect=Rect(0, 0 - 3840, 2160)
mCurrentDisplayRect=Rect(0, 0 - 3840, 2160)
mCurrentSurface=null
mPhysicalDisplayId=19260688233415425
mAllowedPhysIndexes=[0]
mAllowedModeIds=[1]
mAllowedModeIdsInvalid=false
mActivePhysIndex=0
mActiveModeId=1
mActiveColorMode=0
mDefaultModeId=1
mState=ON
mBrightness=102
mBacklight=com.android.server.lights.LightsService$LightImpl@62128c3
mDisplayInfos=
PhysicalDisplayInfo{3840 x 2160, 60.240963 fps, density 2.625, 300.0 x 300.0 dpi, secure true, appVsyncOffset 1000000, bufferDeadline 16600000}
mSupportedModes=
DisplayModeRecord{mMode={id=1, width=3840, height=2160, fps=60.240963}}
mSupportedColorModes=[0]
DisplayDeviceInfo{“NEXT340-202b64”: uniqueId=“wifi:0e:cf:89:47:1a:0e”, 1920 x 1080, modeId 2, defaultModeId 2, supportedModes [

{id=2, width=1920, height=1080, fps=60.0}
], colorMode 0, supportedColorModes [0], HdrCapabilities null, density 320, 320.0 x 320.0 dpi, appVsyncOff 0, presDeadline 16666666, touch EXTERNAL, rotation 0, type WIFI, address 0e:cf:89:47:1a:0e, state ON, FLAG_PRESENTATION}
mAdapter=WifiDisplayAdapter
mUniqueId=wifi:0e:cf:89:47:1a:0e
mDisplayToken=android.os.BinderProxy@5bf540
mCurrentLayerStack=0
mCurrentOrientation=0
mCurrentLayerStackRect=Rect(0, 0 - 3840, 2160)
mCurrentDisplayRect=Rect(0, 0 - 1920, 1080)
mCurrentSurface=Surface(name=null)/@0x14aa57d

And in SurfaceFlinger, it connected 2 displays also
__SurfaceFligner Display info dump __SurfaceFligner Display info dump

Displays (2 entries)

  • DisplayDevice{72056494526300160, virtual, “NEXT340-202b64”}
    powerMode=2, activeConfig=0, numLayers=1
    Composition Display State: [“NEXT340-202b64”]
    isVirtual=true hwcId=72056494526300160
    isEnabled=true isSecure=false layerStack=0 layerStackInternal=false
    transform 0x00000000 (ROT_0 ) 0x04 (SCALE )
    0.5000 0.0000 0.0000
    0.0000 0.5000 0.0000
    0.0000 0.0000 1.0000

frame=[0 0 1920 1080] viewport=[0 0 3840 2160] scissor=[0 0 1920 1080] needsFiltering=true
colorMode=NATIVE (0) renderIntent=COLORIMETRIC (0) dataspace=UNKNOWN (0) colorTransform=0
Composition Display Color State:
HWC Support: wideColorGamut=false hdr10plus=false hdr10=false hlg=false dv=false metadata=3
Composition RenderSurface State:
size=[1920 1080] ANativeWindow=0x73a5416010 (format 1) flips=758

1 Layers - Output Layer 0x73a1e23600 (Composition layer 0x742f45fc98) (SurfaceView - com.UnityTechnologies.com.unity.template.urpblank/com.unity3d.player.UnityPlayerActivity#0)
Region visibleRegion (this=0x73a1e23628, count=1)
[ 0, 0, 1920, 1080]
forceClientComposition=false clearClientTarget=true displayFrame=[0 0 1920 1080] sourceCrop=[0.000000 0.000000 3840.000000 2160.000000] bufferTransform=0 (0) z-index=0
hwc: layer=0x0836 composition=DEVICE (2)

  • DisplayDevice{19260688233415425, primary, “Null Display”}
    powerMode=2, activeConfig=0, numLayers=1
    Composition Display State: [“Null Display”]
    isVirtual=false hwcId=19260688233415425
    isEnabled=true isSecure=true layerStack=0 layerStackInternal=true
    transform 0x00000000 (ROT_0 ) 0x00 (IDENTITY )
    1.0000 0.0000 -0.0000
    0.0000 1.0000 -0.0000
    0.0000 0.0000 1.0000

frame=[0 0 3840 2160] viewport=[0 0 3840 2160] scissor=[0 0 3840 2160] needsFiltering=false
colorMode=NATIVE (0) renderIntent=COLORIMETRIC (0) dataspace=UNKNOWN (0) colorTransform=0
Composition Display Color State:
HWC Support: wideColorGamut=false hdr10plus=false hdr10=false hlg=false dv=false metadata=3
Composition RenderSurface State:
size=[3840 2160] ANativeWindow=0x7433849010 (format 1) flips=3121
FramebufferSurface: dataspace: Default(0)
mAbandoned=0

  • BufferQueue mMaxAcquiredBufferCount=2 mMaxDequeuedBufferCount=1
    mDequeueBufferCannotBlock=0 mAsyncMode=0
    mQueueBufferCanDrop=0 mLegacyBufferDrop=1
    default-size=[3840x2160] default-format=1 transform-hint=00 frame-counter=2317

I cannot go futher how to unity find multi display info on Android.
If unity get info through Android frameworks, it have to get 2 display correctly.
If get info through Android native framework(Surfaceflinger etc.), it have to get 2 display

but it isn’t.

if it have to fix our side(AOSP or drivers), please give me entry point to analyze this issue

Thanks

2

Unity is using DisplayManager.DisplayListener  |  Android Developers under the hood, try using this listener in UnityPlayerActivity.java, and see if your second display is correctly caught by this listener.

3

Yes but as as I know, DisplayListener called back when secondary display is added or changed size case only.
It also called from framework DisplayManagerService,

I tested under Both Device(HDMI main display, Miracast second display) are connected
and here is dump info, it can find both display are added.
Logical Displays: size=2
Display 0:
mDisplayId=0
mLayerStack=0
mHasContent=true
mAllowedDisplayModes=[1]
mRequestedColorMode=0
mDisplayOffset=(0, 0)
mDisplayScalingDisabled=false
mPrimaryDisplayDevice=Built-in Screen
mBaseDisplayInfo=DisplayInfo{“Built-in Screen, displayId 0”, uniqueId “local:19260688233415425”, app 3840 x 2160, real 3840 x 2160, largest app 3840 x 2160, smallest app 3840 x 2160, mode 1, defaultMode 1, modes [{id=1, width=3840, height=2160, fps=60.240963}], colorMode 0, supportedColorModes [0], hdrCapabilities android.view.Display$HdrCapabilities@40f16308, rotation 0, density 420 (300.0 x 300.0) dpi, layerStack 0, appVsyncOff 1000000, presDeadline 16600000, type BUILT_IN, address {port=1, model=0x446d7e704843}, state ON, FLAG_SECURE, FLAG_SUPPORTS_PROTECTED_BUFFERS, removeMode 0}
mOverrideDisplayInfo=DisplayInfo{“Built-in Screen, displayId 0”, uniqueId “local:19260688233415425”, app 3840 x 2034, real 3840 x 2160, largest app 3840 x 3651, smallest app 2160 x 1971, mode 1, defaultMode 1, modes [{id=1, width=3840, height=2160, fps=60.240963}], colorMode 0, supportedColorModes [0], hdrCapabilities android.view.Display$HdrCapabilities@40f16308, rotation 0, density 420 (300.0 x 300.0) dpi, layerStack 0, appVsyncOff 1000000, presDeadline 16600000, type BUILT_IN, address {port=1, model=0x446d7e704843}, state ON, FLAG_SECURE, FLAG_SUPPORTS_PROTECTED_BUFFERS, removeMode 0}
Display 1:
mDisplayId=1
mLayerStack=1
mHasContent=false
mAllowedDisplayModes=[2]
mRequestedColorMode=0
mDisplayOffset=(0, 0)
mDisplayScalingDisabled=false
mPrimaryDisplayDevice=NEXT340-202b64
mBaseDisplayInfo=DisplayInfo{“NEXT340-202b64, displayId 1”, uniqueId “wifi:0e:cf:89:47:1a:0e”, app 1920 x 1080, real 1920 x 1080, largest app 1920 x 1080, smallest app 1920 x 1080, mode 2, defaultMode 2, modes [{id=2, width=1920, height=1080, fps=60.0}], colorMode 0, supportedColorModes [0], hdrCapabilities null, rotation 0, density 320 (320.0 x 320.0) dpi, layerStack 1, appVsyncOff 0, presDeadline 16666666, type WIFI, address 0e:cf:89:47:1a:0e, state ON, FLAG_PRESENTATION, removeMode 0}
mOverrideDisplayInfo=DisplayInfo{“NEXT340-202b64, displayId 1”, uniqueId “wifi:0e:cf:89:47:1a:0e”, app 1920 x 1080, real 1920 x 1080, largest app 1920 x 1920, smallest app 1080 x 1080, mode 2, defaultMode 2, modes [{id=2, width=1920, height=1080, fps=60.0}], colorMode 0, supportedColorModes [0], hdrCapabilities null, rotation 0, density 320 (320.0 x 320.0) dpi, layerStack 1, appVsyncOff 0, presDeadline 16666666, type WIFI, address 0e:cf:89:47:1a:0e, state ON, FLAG_PRESENTATION, removeMode 0}

If unity get info from DisplaymanagerService, it should be 2 connected devices when Unity app started.
Is there any info to fix this?

4

I might be missing some knowledge here, maybe @florianpenzkofer can help.