Adding tracking AR maker images at runtime support for Vision Pro

Unity Engine
, , ,
1

Hi,
I’d like to add images to XR Reference Image Library at runtime on Vision Pro.
I tried with the document of AR Foundation, and got the following error when I call mutableLibrary.ScheduleAddImageWithValidationJob.

  NotImplementedException: The method or operation is not implemented.
  at UnityEngine.XR.VisionOS.VisionOSImageDatabase.ScheduleAddImageJobImpl (Unity.Collections.NativeSlice`1[T] imageBytes, UnityEngine.Vector2Int sizeInPixels, UnityEngine.TextureFormat format, UnityEngine.XR.ARSubsystems.XRReferenceImage referenceImage, Unity.Jobs.JobHandle inputDeps)

It seems that the feature is not implemented on visionOS yet.

Unity 2022.3.21f1
AR Foundation 5.1.2
Apple visionOS XR Plugin 1.1.6
PolySpatial visionOS 1.1.6

Do you have a plan to implement the feature soon for visionOS?

2

The format of the Texture 2D should be changed to RGB24bit etc from Automatic otherwise you’ll get the following error, by the way

InvalidOperationException: The texture format ASTC_6x6 is not supported by the current image tracking subsystem.
  at UnityEngine.XR.ARSubsystems.MutableRuntimeReferenceImageLibrary.ValidateAndThrow
3

Same problem was posted and answered.

4

Hello, do we have any news regarding this feature? I just submitted it on the portal.

5

Yes. I added support to dynamically load an image for tracking in 2.0.0-pre.9.

Here is an example of how you can do that. Note I cut/pasted that from one of our internal tests for this and had to modify it to make sense here, I’m not sure if the Awake bit if enabled should be true or false, if one doesn’t work try the other.

    public class ImageTrackingTestManager : MonoBehaviour
    {
        [SerializeField]
        ARTrackedImageManager m_ImageManager;

        [SerializeField]
        XRReferenceImageLibrary m_ImageLibraryOne;

        [SerializeField]
        Texture2D m_TextureOne;

        bool libraryUpdateInProgress;
        JobHandle libraryUpdateJob;

        public void Awake()
        {
            m_ImageManager.referenceLibrary = m_ImageLibraryOne;
            m_ImageManager.enabled = false; //Might need to be true, not sure
        }

        public void DynamicOneButtonPressed()
        {
            if (libraryUpdateInProgress)
            {
                Debug.Log(k_LogMarker + "Image Add still in progress aborting.");
                return;
            }

            if (m_ImageManager.referenceLibrary is MutableRuntimeReferenceImageLibrary mutableLibrary)
            {
                var scheduleAddImageJob = mutableLibrary.ScheduleAddImageWithValidationJob(
                    m_TextureOne,
                    "one",
                    .2f);

                libraryUpdateInProgress = true;
                libraryUpdateJob = scheduleAddImageJob.jobHandle;
            }
        }

        void Update()
        {
            if (libraryUpdateInProgress)
            {
                if (libraryUpdateJob.IsCompleted)
                {
                    m_ImageManager.enabled = true;
                    libraryUpdateInProgress = false;
                    Debug.Log(k_LogMarker + "Finished adding an image.");
                }
            }
        }
    }
6

Thanks a lot for that update and for this examle. This really helps. Can the library be also created at runtime?

7

Yeah, check out this API: CreateRuntimeLibrary.