UnityPlayer.displayChanged() method to add a Surface as an additional display

Unity Engine
,
1

Has anyone successfully used the displayChanged() method in the UnityPlayer Java class to add a Surface as an additional display that can be rendered to? I’m trying to use this with a Surface created for input to a MediaCodec encoder. The only reference I can find is a single forum post that references a single dot point in Unity 5.x release notes, but this looks like the perfect feature I need to get a rendered image from Unity into the MediaCodec encoder.

When the app starts, Display.displays returns one display (the phone screen). I call UnityPlayer.displayChanged(1, encoderSurface), and a short time later Display.displays will return two displays. The Display.onDisplaysUpdated event never got triggered however, so I check each update for a change to the length of the Display.displays array. Then I enable a camera and set its targetDisplay to the index of the new display. I don’t appear to be getting anything though.

I’m still working on it, but if anyone has used this successfully it would be great to know I’m not running up a dead-end alleyway. And Unity, please document this stuff. Please.

1 Like
2

Odd behaviour found. The input surface created by the MediaCodec encoder is 720x1280, as expected because that is the size we requested for the encoder. When the surface is added to Unity’s displays list, the Display object returns a rendering and system size of 1080x2057. The main display has a rendering size of 1080x2204 and system size of 1080x2340, but this second display should be 720x1280, not the oddball size it is returning.

3

‘targetDisplay’ is only supported on Android since Unity 2020.2. On older Unity versions you would have to use ‘Camera.SetTargetBuffers’

4

Thanks for that. For reference, we are using 2019.4.11f1. I changed the code to use Camera.SetTargetBuffers, but I haven’t been able to verify it’s working yet. The MediaCodec encoder that created the surface still doesn’t appear to be producing any output yet.

The odd rendering and system sizes returned for the new display are still there.

5

The MediaCodec encoder does not produce any output when the surface has not been rendered to, and that is what appears to be the case here. Is UnityPlayer.displayChanged() verified to work and able to be rendered to?

6

I decided to try the project in Unity 2020.2 to see if it worked better. No luck since it breaks on deterministic compilation (had to turn that off) and a very unhelpful error for our custom Gradle build template.
“mainTemplate.gradle file is using the old aaptOptions noCompress property definition which does not include types defined by unityStreamingAssets constant.”
but nowhere does the editor or the docs say what the fix is or how to fix it. Every avenue I take I find a dead-end.

3 Likes
7

Hi.
Unity 2020.2
We cant build on Android too with the same error: “mainTemplate.gradle file is using the old aaptOptions noCompress property definition which does not include types defined by unityStreamingAssets constant.” Could someone from Unity help us?

1 Like
8

With regards to the mainTemplate.gradle error. I had the same thing, so I moved the file and let Unity3D regenerate the default one.

It added a new noCompress line to the aaptOptions section:

aaptOptions {
        noCompress = ['.ress', '.resource', '.obb'] + unityStreamingAssets.tokenize(', ')
        ignoreAssetsPattern = "!.svn:!.git:!.ds_store:!*.scc:.*:!CVS:!thumbs.db:!picasa.ini:!*~"
    }**PACKAGING_OPTIONS**

I copied the noCompress line to my original mainTemplate.gradle file and I no longer had this error (I did however get other errors).

3 Likes
9

I did the same, but in the console you can see that the error regarding unityStreamingAssets, so not sure that this could help =( I think we need to wait for answer from someone from Unity.

10

Can you show your gradle files?

11

Old (generated from a 2019.4.11f1 project)
In the aaptOptions section, the noCompress line is either missing or contains

noCompress = ['.unity3d', '.ress', '.resource', '.obb'**STREAMING_ASSETS**]

New (freshly generated from a new 2020.2.0f1 project)
The aaptOptions section contains

noCompress = ['.ress', '.resource', '.obb'] + unityStreamingAssets.tokenize(', ')

Notice the change to how the Streaming Assets extensions are added?

The error that 2020.2.0f1 gives when confronted with the earlier aaptOptions section (missing the noCompress line or the older format) is

6649711--759553--upload_2020-12-23_1-13-46.png

No hint as to how to fix the problem or any link to documentation describing what is expected, not even in the Console error output. Perhaps it could take a hint from a later different error, also related to Gradle files, that describes what to do and offers to fix the problem.

6649711--759556--upload_2020-12-23_1-14-50.png

It then repeats the earlier aaptOptions error for the launcherTemplate.gradle file.

That should be all of the information you need. I do not know what else to give.

5 Likes
12

Same here. Would be nice to have some documentation or instructions on what to do.

13

Back to the original issue (adding an Android surface as an additional display), after making the change to the gradle files to get the project to build in 2020.2.0f1, the rendering to the extra display appears to work. The MediaCodec encoder is seeing updates to the input surface and producing encoded h264 output.

The summary for those just catching up and for future developers searching for these errors:

  1. The feature to add an Android surface as an additional display appears to require Unity 2020.2.0f1. It does not appear to work correctly in Unity 2019.4.11f1 (I tried for days to get it working), even though the feature was added back in the Unity 5.x series.

  2. If upgrading to Unity 2020.2.0f1 from an earlier Unity project that has a custom Android gradle file, change the existing noCompress line in the aaptOptions section (or add it if it is missing) to the following

noCompress = ['.ress', '.resource', '.obb'] + unityStreamingAssets.tokenize(', ')
5 Likes
14

Thanks, that’s exactly what I needed!

1 Like
15

Hi!

So, have you found the way to resolve the issue after adding this string? Could you share it please? )

1 Like
16

Thanks for your help!

17

same error .“mainTemplate.gradle file is using the old aaptOptions noCompress property definition which does not include types defined by unityStreamingAssets constant.” Is anyone found a solution.

18

when trying to build an apk . Android gradle file changes to its previous version.

19

If you have custom gradle files, you need to make the change listed above in both launcherTemplate.gradle and mainTemplate.gradle.

20

yeah i have custom gradle files and i changed both launcherTemplate.gradle and mainTemplate.gradle but when i tried to make a build then the same error occurred and mainTemplate.gradle changed to their previous version.
i dont know the exact reason but this issue starts when i ugraded my unity editor to latest