Can't send "ARPoseDriver.cs" position/rotation to separate GameObject

Infinite_Showrooms · December 29, 2021, 10:53pm

Hey all. Long time reader, first time posting!

I’m trying to modify AR Foundation’s “ARPoseDriver.cs” to send the resulting AR device position/rotation to a separate game object and I can’t understand why it’s not working.

Ultimately I want to just disable position/rotation on the “AR Camera” and have it be managed by the separate object (more on that below), but for now I’m not even able to send the resulting values to a dummy Vector3 and Quaternion for testing.

I’ve declared 3 values for testing in lines 17-19:

using System;
using System.Collections;
using System.Collections.Generic;
using System.Runtime.CompilerServices;
using UnityEngine;
using UnityEngine.XR;

namespace UnityEngine.XR.ARFoundation
{
    /// <summary>
    /// The ARPoseDriver component applies the current Pose value of an AR device to the transform of the GameObject.
    /// </summary>

    [HelpURL(HelpUrls.ApiWithNamespace + nameof(ARPoseDriver) + ".html")]
    public class ARPoseDriver : MonoBehaviour
    {
        public Transform DigitalPlayer;
        public Vector3 CameraPosition;
        public Quaternion CameraRotation;

===
Here’s an image of how that shows in Unity Inspector (look at the public variables now displaying under "AR Pose Driver script):

And later starting at “PerformUpdate()” (which I believe overrides the regular “Update()” I’m trying to update the position value to that dummy “CameraPosition” Vector3 I’ve got set up to at least visualize if it’s receiving values (line 15 and the 2 comments are what I added):

#endif // UNITY_UNITY_2020_1_OR_NEWER
        }
        void Update() => PerformUpdate();
        void OnBeforeRender() => PerformUpdate();
        void PerformUpdate()
        {
            if (!enabled)
                return;
            //This line also does not affect pose data
            var updatedPose = GetPoseData();
            //Even commenting out the below two lines have no effect on the camera tracking so line 70-74 has no effect on the camera location
            if (updatedPose.position.HasValue)
            {
                transform.localPosition = updatedPose.position.Value;
                CameraPosition = updatedPose.position.Value;
            }
            if (updatedPose.rotation.HasValue)
                transform.localRotation = updatedPose.rotation.Value * updatedPose.rotation.Value;
        }

====
The rest of the file is just what’s in AR Foundation 4.1.7.

But no matter what I try (no errors in the code) I can’t get that “CameraPosition” value to show anything other than 0,0,0 when I run the editor.

I tried a different route of “breaking” the code to see where exactly the position/rotation gets called, but even when I comment out the line “var updatePose = GetPoseData();” the camera STILL gets all of the position/rotation data from ARkit.

I can’t understand where in this “ARPoseDriver.cs” code that the actual location/rotation gets returned so I can send the values to a separate GameObject.

Anyone able to help?

For context, I’ve had this method running in Unity 2017 for the last few years without a problem, but since I want to take advantage of the newer ARkit features I’m trying to get this working now in Unity 2021.1.22.
I’m doing this all for location-based games with “QR code” tracked images on the walls, so in my Unity 2017 version rather than use “QR codes” to move the entire scene to the tracked image I had it reverse-engineered so that when a “QR code” on the floor gets seen in the physical room it’s the PLAYER that gets moved around the 3D environment (rather than the 3D environment getting moved around the player). This way their 3D position always corresponds to the world position in the actual room, which makes it easy to quickly set up the game for multiplayer by just dropping a QR code on the ground and also allows me to use NavMeshes (since the 3D room never actually moves).

Sending the iPhone’s Camera location to a separate object and having the “AR Camera” location calculated separately is the first step to making this all work, which has worked great in my Unity 2017 build.

AR Foundation has added a ton of functionality that I’ve really loved, but I can’t find a way to “break” it to send the camera position/rotation data to a separate GameObject.

PS, I’m also using the “AR Foundation Remote 2.0” plugin and it’s been a dream for AR prototyping: AR Foundation Remote 2.0 | Utilities Tools | Unity Asset Store

Infinite_Showrooms · December 30, 2021, 2:06pm

Shoutout to @KyryloKuzyk for “AR Foundation Remote 2.0”, ps: https://assetstore.unity.com/packages/tools/utilities/ar-foundation-remote-2-0-201106

Just bought it yesterday and even getting to this point of failure/asking for help probably would have taken me an extra week of testing if I had to run iOS builds each time and send the Debug.Log to the iPhone display like I used to.
Anyone working with AR Foundation should definitely buy this!

KyryloKuzyk · December 30, 2021, 4:18pm

This is one of the limitations of the AR Foundation Remote plugin: the plugin doesn’t implement the native XRInputSystem. Instead, it finds the ARPoseDrive component sitting on the ASSessionOrigin.camera and applies position and rotation directly to the camera’s game object.

METHOD 1
There is a simple way to write a custom ARPoseDriver that will work in the Editor for AR Foundation <= 4.1.7 and Unity < 2020.1:

Make a copy of the existing ARPoseDriver.cs script and rename the class inside the copied script.
Add this line on top of your copied script:
using InputTracking = ARFoundationRemote.Runtime.InputTracking;

METHOD 2
There is another way to get the camera pose that will work regardless of AR Foundation or Unity version. I tested it with Unity 2021.2.6 and AR Foundation 4.2.1, but there is a chance that this code will stop working in future Unity releases because AR Foundation is currently making a transition to New Input System.
You can also wrap the using InputTracking... line with a [version define]( AR Foundation Remote | Test and debug your AR project in the Editor page-7#post-6561910) to decouple your project from the plugin.

using System.Collections.Generic;
using UnityEngine;
using UnityEngine.XR;
using InputTracking = ARFoundationRemote.Runtime.InputTracking;


public static class ARFoundationRemoteUtils {
    public static Pose? GetCameraPose() {
        var states = new List<XRNodeState>();
        InputTracking.GetNodeStates(states);
        foreach (var state in states) {
            if (state.nodeType == XRNode.CenterEye) {
                if (!state.TryGetPosition(out var position)) {
                    return null;
                }

                if (!state.TryGetRotation(out var rotation)) {
                    return null;
                }

                return new Pose(position, rotation);     
            }
        }

        return null;
    }
}

blamejane · December 30, 2021, 6:52pm

Thanks for this link, just purchased!

Infinite_Showrooms · December 30, 2021, 7:30pm

Ah, thank you. I was wondering if it had something to do with your plugin. I can’t expect Unity Desktop to perfectly reproduce iOS, nor do I expect your remote plugin to work for all fringe use cases like mine. Thanks for your support and I’m glad you just got another sale from this thread via @blamejane !

So you’re saying I need to downgrade to 2020.1 for this simple method you posted to work? Would a further iteration like 2020.1.17 work just 2020.1.0, exactly? Or are you saying by “<” that I need to use the last version of 2019?

I might also just try upgrading to Unity 2021.2.6 and AR Foundation 4.2.1, since you said that worked.
I’d cut the “ARPoseDriver.cs” and put this modded “CameraPoseTest.cs” code on the same “AR Camera” GameObject with the stock “ARCameraManager.cs”, right?

KyryloKuzyk:

This is one of the limitations of the AR Foundation Remote plugin: the plugin doesn’t implement the native XRInputSystem. Instead, it finds the ARPoseDrive component sitting on the ASSessionOrigin.camera and applies position and rotation directly to the camera’s game object.

There is a simple way to write a custom ARPoseDriver that will work in the Editor for AR Foundation <= 4.1.7 and Unity < 2020.1:

Make a copy of the existing ARPoseDriver.cs script and rename the class inside the copied script.

Add this line on top of your copied script:
using InputTracking = ARFoundationRemote.Runtime.InputTracking;

Unfortunately, for AR Foundation > 4.1.7 and Unity >= 2021, there is no such simple modification available. You can grab the camera pose with the following not-so-pretty piece of code. It will work the same both in the Editor and in real builds. I tested it with Unity 2021.2.6 and AR Foundation 4.2.1, but there is no guarantee that it will work in future versions of Unity and AR Foundaiton.
using System.Collections.Generic;
using UnityEngine;
using UnityEngine.XR;
using InputTracking = ARFoundationRemote.Runtime.InputTracking;


public class CameraPoseTest : MonoBehaviour {
    void Update() {
        print($"camera pose: {GetCameraPose()}");
    }

    public static Pose? GetCameraPose() {
        var states = new List<XRNodeState>();
        InputTracking.GetNodeStates(states);
        foreach (var state in states) {
            if (state.nodeType == XRNode.CenterEye) {
                if (!state.TryGetPosition(out var position)) {
                    return null;
                }

                if (!state.TryGetRotation(out var rotation)) {
                    return null;
                }

                return new Pose(position, rotation);    
            }
        }

        return null;
    }
}

KyryloKuzyk · December 31, 2021, 9:17am

Infinite_Showrooms:

Ah, thank you. I was wondering if it had something to do with your plugin. I can’t expect Unity Desktop to perfectly reproduce iOS, nor do I expect your remote plugin to work for all fringe use cases like mine. Thanks for your support and I’m glad you just got another sale from this thread via @blamejane !

So you’re saying I need to downgrade to 2020.1 for this simple method you posted to work? Would a further iteration like 2020.1.17 work just 2020.1.0, exactly? Or are you saying by “<” that I need to use the last version of 2019?

I might also just try upgrading to Unity 2021.2.6 and AR Foundation 4.2.1, since you said that worked.
I’d cut the “ARPoseDriver.cs” and put this modded “CameraPoseTest.cs” code on the same “AR Camera” GameObject with the stock “ARCameraManager.cs”, right?

I’m sorry for the small confusion caused by my answer. Please see the edited response above.
Please use your current Unity and AR Foundation. The main philosophy of the plugin is that it should not influence your existing project in any way.

If you use AR Foundation <= 4.1.7 AND Unity < 2020.1, then use METHOD 1.
Else, use METHOD 2.

So if your Unity is 2020.1.17 (which is > than 2020.1), please use METHOD 2.

It depends on what you’re trying to achieve. If you just want to manipulate some GameObject’s position, you can call the code above at any point to get the camera pose and apply it to the game object. For example:

using UnityEngine;


public class ApplyCameraPoseToThisGameObject : MonoBehaviour {
    void Update() {
        var cameraPose = ARFoundationRemoteUtils.GetCameraPose();
        if (cameraPose.HasValue) {
            transform.position = cameraPose.Value.position;
            transform.rotation = cameraPose.Value.rotation;
        }
    }
}

Infinite_Showrooms · January 3, 2022, 5:52pm

@KyryloKuzyk Thanks so much! I’ll dig into this over the next 1-2 weeks and see how it goes!

KyryloKuzyk · April 7, 2022, 8:26am

UPDATE:
The latest version of the AR Foundation Remote plugin now supports the native XRInputSystem in this setup: Unity >= 2020.1, AR Foundation >= 4.0. This means that the ARPoseDriver.cs now works exactly the same way it works on real devices and the aforementioned workarounds are no longer needed.

Infinite_Showrooms · June 7, 2022, 5:58am

Hey @KyryloKuzyk , sorry for my delayed reply! I actually thought I wrote you already!

Well I HAD been busy the last few months shifting to some other priorities, but now I’m writing you at your Gmail address about a few possible paid projects involving ARkit, including this one.

I’ll send you the email shortly. Hope you’ve been well!

Infinite_Showrooms · June 7, 2022, 7:29am

Hey @KyryloKuzyk ,

Just sent you an email to the gmail address on your GitHub repo.

You could also just schedule a 15-30min meeting with me if that’s faster than thinking through a reply (sometimes I find it’s faster to talk through heavy spec-related projects like this than to try to email back and forth to clarify points, but up to you).

Hope you’re having a good day!

-J

Topic		Replies	Views
ARFoundation need help! Unity Engine XR , AR-Foundation	3	3598	September 3, 2019
Can only use Pose to update 1 object Unity Engine XR	1	893	May 23, 2019
coding help Unity Engine Scripting	2	677	October 31, 2019
Change AR Camera Postion Unity Engine XR	4	2538	August 31, 2022
Exclude GameObject transform from being controlled by AR Unity Engine XR , AR-Foundation , Question	1	446	September 28, 2023

Can't send "ARPoseDriver.cs" position/rotation to separate GameObject

I’ve declared 3 values for testing in lines 17-19:

And later starting at “PerformUpdate()” (which I believe overrides the regular “Update()” I’m trying to update the position value to that dummy “CameraPosition” Vector3 I’ve got set up to at least visualize if it’s receiving values (line 15 and the 2 comments are what I added):

Related topics