Unity ARKit support issue with iOS 16

Unity Engine
, ,
1

As part of validating Unity against the upcoming iOS 16 release, we have discovered a timing issue in Unity’s implementation of ARKit. The issue causes camera frames to arrive out of order, leading to an unstable camera feed. This issue affects all Unity titles using ARKit. We have a fix, which will be available in Unity ARKit package versions 4.1.11, 4.2.5 and 5.0.1, as soon as they pass QA validation. Affected developers will need to update to a fixed version and resubmit your applications.

We know this is an inconvenience, and a possible impact to your business, as many of you have applications that will break when iOS 16 is launched next week. While issues like this are not uncommon with large OS updates like this, we should have identified the issue sooner, and reacted faster. We can do better and will be evaluating steps we can take to ensure this doesn’t happen again in the future. In the mean time, we are working with Apple to help you get titles updated and through certification as quickly as possible.

3 Likes
2

I first submitted a bug for this with iOS 16 Beta 1. Back in June! When running on iOS 16 the camera feed is unstable (shaky and jittery) when multi-threaded rendering is enabled · Issue #988 · Unity-Technologies/arfoundation-samples · GitHub.

I had hoped this would have already been on your radar as it was an obvious issue when running even arfoundations samples repo on iOS 16. Why did your own QA/Testing not pick this issue up? Do you have any processes around testing/validating new versions of iOS/Android?

I was also hoping that my ticket might be picked up a bit faster as it was a major issue with an upcoming system release. Is there a way to get tickets expedited or mark them critical?

1 Like
3

In my project, upgrading the ARF version did not solve the problem.
After isolating the problem, I found that the problem occurs when using ARCoreExtensions and using the arcore geospatial API.
I reported it on the ARCoreExtensions side, but I thought it might be related to this issue reported on ARF, so I am reporting it here as well.

Below is the Issue to ARCoreExtensions.

4

Just updated to iOS version 16.3.1. Getting Black Screen in the background. :frowning:

5

@kittphi some troubleshooting steps:

  • Update to latest patch of AR Foundation and ARKit packages (specifically important that you use 4.2.7 or newer for 4.2, or 5.0.3 or newer for 5.0)
  • Make sure your scene contains AR Session, XR Origin, ARCameraBackground, and ARCameraManager as described here: Scene setup | AR Foundation | 5.0.7
  • If using URP, make sure you update to the latest patch of your Unity Engine. (Specifically there is a known black-screen bug in URP on Unity 2022.2.0f4 and below). Also make sure you add the AR Background Renderer feature as described here: Universal Render Pipeline | AR Foundation | 5.0.7

Hope that helps!

6

I am not sure, but I’ll clarify anyway.

Does ARKit package within AR Foundation use WebCamTexture inside to provide Camera Feed on the Background of scene?

If it uses, then the bug was not solved for WebCamTexture itself (without AR) for Unity 2021 LTS, 2022 LTS, 2023.2a17.

If it’s not, then there is another same bug with WebCamTexture itself (without AR).

More noticeable for the front camera, and after the several device orientations changing. iPhone XS Max.

7

No. It uses the custom material that you pass to the AR Camera Background component: Camera components | AR Foundation | 5.0.7, or the provider plug-in’s default shader if you don’t override it.

Our default shaders for ARCore and ARKit camera background are open source and included in the respective packages.

2 Likes
8

I don’t know if it was about ios 16. But iphone 11 with ios 16.6 cannot get AR for user facing camera

Try using the chooser result in these

No Camera Floor Offset GameObject specified for XR Origin, using attached GameObject.
Camera "AR Camera" does not use a Tracked Pose Driver (Input System), so its transform will not be updated by an XR device.  In order for this to be updated, please add a Tracked Pose Driver (Input System) with bindings for position and rotation of the center eye.
Configuration Descriptor 0x2089d4770 (rank 2): Rotation and Orientation, Plane Tracking, Light Estimation (Ambient Intensity), Light Estimation (Ambient Color), Raycast
Configuration Descriptor 0x2089d5b20 (rank 0): World Facing Camera, Rotation and Orientation, Plane Tracking, Image Tracking, Object Tracking, Environment Probes, 2D Body Tracking, Human Occlusion Stencil, Human Occlusion Depth, Collaboration, Auto-Focus, Light Estimation (Ambient Intensity), Light Estimation (Ambient Color), Raycast
Configuration Descriptor 0x2089d6d90 (rank 1): World Facing Camera, Rotation Only, 2D Body Tracking, Human Occlusion Stencil, Human Occlusion Depth, Auto-Focus, Light Estimation (Ambient Intensity), Light Estimation (Ambient Color)
Configuration Descriptor 0x2089d4540 (rank 1): World Facing Camera, Rotation Only, Image Tracking, 2D Body Tracking, Human Occlusion Stencil, Human Occlusion Depth, Auto-Focus, Light Estimation (Ambient Intensity), Light Estimation (Ambient Color)
Configuration Descriptor 0x2089d40e0 (rank -1): World Facing Camera, Rotation and Orientation, Plane Tracking, Image Tracking, Environment Probes, 2D Body Tracking, 3D Body Tracking, 3D Body Scale Estimation, Auto-Focus, Light Estimation (Ambient Intensity), Light Estimation (Ambient Color), Raycast
    Requested Features: User Facing Camera, Auto-Focus
features not satisfied: User Facing Camera

As you can see, it only return World Facing Camera. It seem like the AR system did not get selfie camera from the start

9

@BrothersPicuresGame The most common reason for this issue is that your project configuration does not enable ARKit Face Tracking. Here’s a link to our docs that explains this: Project configuration | Apple ARKit XR Plugin | 5.0.7