ARFoundation + Post Processing Stack = Tracking Drift

I have been trying to introduce Post Processing Stack effect depending on the user’s actions. I import the Post Process package (2.12) via Package manager, add components to the AR Camera and when I build the tracking is not stable anymore. Please find screencapture with slight Chromatic Aberration.
NiceUnfitFly
If I disable PP Layer and Volume components on the camera, tracking comes back to normal.
I’m using Unity 2018.2.18f1 and ARFoundation preview 20.
Has anyone encountered this issue? Thanks!

The video looks like you have quite a low framerate; that could impact tracking quality. Is that the case, or is it just how it was captured?

Was this problem solved, as I have the same issue.
I did manage to determine that this drift only occurs when the device is in Landscape with any form of post-processing enabled. This drift did not occur when any of the devices were in portrait with post-processing enabled.

I have replicated this issue on Android and iOS using the following unity versions and mobile devices.
Unity versions: 2018.2.13f1 and 2018.3.0f2
Mobile devices: Samsung S7 edge, IPhone 8, IPhone X
Plugin versions Unity 2018.2.13f1: Post-Processing v2.1.2, ARFoundation 1.0.0-preview.17, ARCore XR Plugin 1.0.0-preview.18, ARKit XR Plugin 1.0.0-preview.14
Plugin versions Unity 2018.3.0f2: Post-Processing v2.1.2, ARFoundation 1.0.0-preview.20, ARCore XR Plugin 1.0.0-preview.23, ARKit XR Plugin 1.0.0-preview.17

Also note that three builds were done for each Unity and mobile pairing:
first with auto rotation enable, second with the build locked to landscape, and third with the build locked to Portrait.

I manage to find the following temporary solution.
Set AR camera to depth 0.
Add a second camera any where to the AR scene.
4107961--359986--SceneLayout.png

For the second camera set Clear Flags to Don’t Clear, Culling Mask to Nothing, and set Depth to 1 assuming your AR camera’s depth is set to 0. Finally add the Post process layer to the second camera and remove the Post process layer from the AR camera
4107961--359989--CameraSetup.png

2 Likes

I hit the same problem and got a (dirty) fix.

While Shaun workaround above kinda worked, I’m doing something a bit more exotic using an alpha mask to not affect the AR background with post process.

Drift was very clear on this setup as i could see my post process effect float around my meshes and bleed on the real world pixels around.

Drift looked very similar to transformation matrix issues I had in the past using multiple cameras in ARFoundation…
Turns out the post process layer script reset the transformation matrix multiple times while executed.
It seems to be related to some temporal anti aliasing stuff.
I just commented out all calls that affect the transform matrix in the script and no more drift with post process on !

3 Likes

Was this also related at all to the device struggling with post processing? Thus far my experience with mobile devices and post processing is to keep it minimal or none at all(!)

I recently ran into this problem. Has anyone found any other solutions because the ones posted above do not seem to work.

I guess this problem might be related to missmatch between expected field of view of the render camera and the actual field of view of the device camera. I encountered this issue before in arcore and fixed it by assigning fov values instead of changing camera projection matrix. Try changing
ARCameraBackground.cs line 331:

//camera.projectionMatrix = eventArgs.projectionMatrix.Value;
camera.fieldOfView = Mathf.Atan(1 / eventArgs.projectionMatrix.Value[5]) * 2f * Mathf.Rad2Deg;
5 Likes

Thanks @shaun_fuzzy_unity , the additional camera trick helped a lot (ARF 3.0.0 preview, 2019.2.2f1).

Well dnnkeeper’s solution worked just fine for me. In my case I had a Camera displaying a UI over an AR Camera Render Target, which is also displaying some UI elements.I needed the post processing stack attached to the AR camera, thus the tracking drift occurred, plus, the UI canvas was scaled up like crazy, editing the ARCameraBackground.cs as dnnkeeper suggested fixed both the issues, Thank you.

2 Likes

What version are you using? Becuase I cant seem the find it in the ARCameraBackground.cs (in the package) but it sounds like i have the same issue with my worldspace UI.

edit: I updated to 3 and tried that setting without any effect :frowning:

I’m using ARFoundation 2.0.2 and Unity 2019.2.11f1
Actually the UI was scaled before I add the the postprocessing, as a workaround I was using Canvas Scaler! But when I edited the ARCameraBackground The AR UI scale was fixed!

Did anyone manage to disable PPv3 on URP for background texture only? The solution @shaun_fuzzy_unity provided no longer works

1 Like

Thanks, @dnnkeeper , the solution still works. (Unity 2019.3.12f). But sometimes
ARCameraBackground script is reverting back to his base look. And the line in code, which we changed, goes back as it was: camera.projectionMatrix = eventArgs.projectionMatrix.Value;
Can I somehow cancel this reverting?

All assets inside Packages folder are reverted to original values every Unity launch and every update of Package Manager. I ended up automatically applying needed changes from script.

1 Like

Thanks! Can you provide some example, please?

@Andrew_Zim I don’t have a code for your case, but you can get the general idea from this script and modify it for your needs.

6081399–659925–ARFaceEditorMemorySafetyErrorFixer.cs (3.15 KB)

1 Like

Thank you!

@tdmowrer Any chance this gets fixed properly in ARFoundation?

1 Like

Still seems like this is an issue 2 years later. Would be nice to see this addressed.