AR Foundation 5 - poor stability and drift even in samples

Updated / EDIT: I’ve since had a response from Andy about this being seen and reported previously in this thread : AR Foundation v5.0

The examples provided match my experiences. Seems related to more recent version of ARCore being used rather than a change in AR Foundation itself.

Mods, feel free to close or remove this post.


Original post :

This is a slight cross-post from the git repo for the samples, but thought it might be worth posting here as well as I assume things are slightly more active on the forum.

First off- some hardware/software info…

AR Foundation 5.0
Unity 2021.3 (LTS)
Android 12 device with AR Core support.

We have an AR app that we’re looking at updating, and at the same time building our AR framework for future projects and cross-promotional applications. The previous app is built on AR Foundation 4.2.3 and in Unity 2021.3, so it’s not far off what is the current active releases.

I’ve been evaluation AR Foundation 5, which I’ve been doing in Unity 2021.3 (URP 12.1.8). I initially tested in 2022, but had the usual black screen issue so wen’t back to 2021 and an older URP version to solve this - we also generally prefer being on LTS versions of Unity anyway. as far as I’m aware, ARFoundation 5 should be supported on 2021.3. It gives us some excellent workflow improvements with the simulator, and we also want to make sure we’re up to date with latest versions as there are changes to libraries that we’d rather deal with now rather than later.

A blocker currently is the quality of the tracking and the stability of the AR objects. This applies to any AR object, whether it’s an AR plane, a tracked image or an anchored object in the world. It’s just not performing anywhere near as good as the product we made in AR Foundation 4.2.3.

General tracking is ‘okay’, things appear where you think they should be, but any particular rotation of the device shows very very noticeable ‘drift’, particularly when you rotate the device across the horizontal (ie. rotating the device towards the sky or the floor). This doesn’t happen in our 4.2.3 app, and doesn’t happen in other AR apps we’ve tested on the device (Android AR Elements, Angry Birds etc) - so I don’t think there is a problem with this device. I can’t show any of our custom anchored objects due to licensing, but I have linked to videos that just shows basic plane tracking. You can see that the planes seem to ‘bounce’ when rotating the device. It’s worth noting that I’m only moving the device every so slightly, the tiniest of amounts, angling them up and down. A matter of 1 or 2 degrees.

Throughout all the below videos, make a note of the edges of the detected planes and how much they move up and down.

This is a video from our test project : https://www.youtube.com/shorts/zpMzZavq2pM

These are from the ar foundation sample project (plane detection and object placement)

Plane Detection : https://www.youtube.com/shorts/WD3HONHiSm0

object placement : https://www.youtube.com/shorts/k4OEhZVYngE

And finally this is from the AR Essentials app :

The final one, I know, isn’t an ARfoundation app, but will be using the same version of ARCore installed on the device - I’ve mainly included it as an example of the device handling AR completely fine.

I’ve checked through all the settings and met the requirements, tried multiple locations, and even confirmed the conditions are acceptable - they aren’t causing an issue with other apps / our ARFoundation 4.2.3 applications. I’ve even made sure to try the same markers in the same locations across both the apps and AR Foundation versions on the same phone - it’s still solid on 4.2.3 and poor on 5.0

There are various posts about tracking and drift of objects, but some are from old versions or are to do with particular implementation, whereas here I’m seeing it not only in our simple tests, but also in the samples from this git repo.

So my questions right now would be whether what I’m seeing in my project and from the samples expected? Should I expect to see better results in the sample projects? To reiterate, the movements I’m making to the phone are very very minor, as little as I possibly could make. I’d expect a degree of movement with quick, large adjustments, but from very small movements there is a lot of drift - To the point when characters are placed in the AR scene, it looks like they are bouncing or on a trampoline. Any pointers or guidance here would be welcomed. I’m at the point now where I need to make a decision over what version of AR Foundation to build the rest of our modules on, and I’d love to use AR Foundation 5, but currently I can’t commit to it.

There was some camera input drift occuring in AR Foundation 5.X when using Input System’s TrackedPoseDriver. This is fixed starting from the following Unity versions:

  • 2021.3.29f1
  • 2022.3.6f1
  • 2023.1.5f1
  • 2023.2.0a23

This should greatly improve stability for tracked objects.

1 Like

Do you mean 2023.2.0b1?
0a23 does not exist and the beta is out

@DevDunk nice catch! Our internal build tooling gave us the incorrect version number because the automated system was not aware that there would be no alpha 22 at the time that we merged this PR.

I checked the changelog for 2023.2.0b1 and confirmed that your suspicion is correct. The changelog entry for this is:

  • XR: Fixed an issue with XR Input → Input System that caused Input System’s TrackedPoseDriver rotation to drift between Update / BeforeRender calls.
1 Like

Only just seeing this -but excellent news! Starting a new AR project, so hopefully we can migrate to AR Foundation 5! Thanks @andyb-unity @thomas_key

3 Likes

Hallo,
i’m wondering is there a best practise to position and rotate objects properly with ar foundation and ArCore. Im doing Image Tracking and the positioning is really bad.

@levoxtrip AR Foundation does not change anything about how GameObjects and Transforms work in your scene. You’ll likely want to subscribe to the trackedImagesChanged event, access the transform component of each tracked image as it is added, instantiate your GameObject(s) as children of those transforms, and set their local position and rotation to 0. This would should render your content at the center of the image, then you can apply any offsets you need for your app from there.

1 Like

One thing to note that has come up a few times on the forums: Your lighting conditions matter greatly for any kind of tracking, especially on ARCore (as Android devices currently don’t have access to LIDAR like ARKit tracking does). So make sure the area you are working / testing in is well-lit or the images may have trouble being recognized.

You may also want to look at using the arcoreimg to validate how recognizable your images are.

A good reference for the ARCore image tracking specific quirks is the ARCore Augmented Images docs.

1 Like