World anchors on the HL2 with the new XR plugin

Unity Engine
,
1

I’m trying to get world anchors to work using the new XR plugin. The implementation using WorldAnchorStore and its related calls have been marked as deprecated. When I try to run the code on the HL2 anyway using the WMR XR plugin, nothing happens. I’m assuming there is a working replacement for them.

In the manual for the WMR XR plugin, this is what is mentioned for the “reference points” (which have been renamed to Anchors in 3.0+ versions of the AR subsystems, but for the time being I’m on AR Subsystems 2.1.3 since that’s the one used in the documentation’s code sample below)

The highlighted sentence makes me worried - has the old system been deprecated without there actually being a working replacement?

This is the code sample:

TrackableChanges<XRReferencePoint>? currentRps = RPSubsystem?.GetChanges(Allocator.Temp) ?? null;

if (currentRps != null)
{
   foreach(var rp in currentRps?.added)
   {
#if ENABLE_WINMD_SUPPORT // Necessary since the Windows Types are only valid through WinMD projection.
       AnchorData data = Marshal.PtrToStructure<AnchorData>(rp.nativePtr);
       SpatialAnchor anchor = data.spatialAnchor as SpatialAnchor;
       if (anchor != null)
       {
           // Do something with the anchor here.
       }
#endif
   }
}

Unfortunately, AnchorData and SpatialAnchor are unknown classes, even in the C# player project (i.e. where ENABLE_WINMD_SUPPORT is defined). I can’t find any reference to them online. The sample also does not tell me how to save and restore anchors - the API also seems a lot more complicated than the deprecated one. Is it possible to get some information on how anchors work with the XR plugin? Thanks!

2

Add this:

#if ENABLE_WINMD_SUPPORT
using Windows.Perception.Spatial;
#endif

[StructLayout(LayoutKind.Sequential)]
struct AnchorData
{
    public int version;

    [MarshalAs(UnmanagedType.IUnknown)]
    public System.Object spatialAnchor;
}
3

If that works, can you file a bug about incorrect sample in documentation and post the case ID here?

4

I was pointed in the direction of the XRAnchorStore by Edd Smith on the HoloDevelopers Slack channel, which allowed me to figure out how the anchors are supposed to be used with the new XR plugin. Here’s some sample code:

AnchorableObject

using System;
using System.ComponentModel;
using Unity.Collections;
using UnityEngine;
using UnityEngine.Events;
using UnityEngine.XR.ARSubsystems;

public class AnchorableObject : MonoBehaviour
{
    public bool LoadAnchorOnStart = true;

    [SerializeField, Tooltip("World anchors must have a unique name. Enter a descriptive name if you like.")]
    protected string _worldAnchorName;
    [SerializeField]
    protected TMPro.TMP_Text _debugLabel;

    [Space(10)]
    public UnityEvent OnAnchorLoaded;

    private TrackableId _xrAnchorId;
    private AnchorStoreManager _anchorStoreManager;

    private void OnValidate()
    {
        if (string.IsNullOrEmpty(_worldAnchorName))
        {
            _worldAnchorName = Guid.NewGuid().ToString();
        }
    }

    private void Start()
    {
        _anchorStoreManager = FindObjectOfType<AnchorStoreManager>();
        if (_anchorStoreManager)
        {
            _anchorStoreManager.PropertyChanged += AnchorStore_PropertyChanged;

            if (LoadAnchorOnStart && _anchorStoreManager.AnchorStore != null)
            {
                LoadAnchor();
            }
        }
        else
        {
            LogDebugMessage($"No {nameof(AnchorStoreManager)} present in scene.", true);
        }
    }

    private void LateUpdate()
    {
        UpdateAnchorPose();
    }

    public bool SaveAnchor(bool overwrite = true)
    {
        if (_anchorStoreManager?.AnchorPointsSubsystem == null || _anchorStoreManager?.AnchorStore == null)
        {
            LogDebugMessage($"Can't save anchor {_worldAnchorName}: reference point subsystem or anchor store have not been initialized.", true);
            return false;
        }

        XRReferencePoint anchor;

        if (overwrite)
        {
            // Delete the current anchor in the store, so we can persist the new position.
            _anchorStoreManager.AnchorStore.UnpersistAnchor(_worldAnchorName);
        }

        // Attempt to save the anchor.
        if (_anchorStoreManager.AnchorPointsSubsystem.TryAddReferencePoint(new Pose(transform.position, transform.rotation), out anchor))
        {
            if (_anchorStoreManager.AnchorStore.TryPersistAnchor(anchor.trackableId, _worldAnchorName))
            {
                _xrAnchorId = anchor.trackableId;
                LogDebugMessage($"Successfully saved anchor {_worldAnchorName}.");
                PositionFromAnchor(anchor);
                return true;
            }
            else
            {
                LogDebugMessage($"Failed to save anchor {_worldAnchorName}.", true);
            }
        }
        else
        {
            LogDebugMessage($"Failed to add reference point for anchor {_worldAnchorName}.", true);
        }

        return false;
    }

    public bool LoadAnchor()
    {
        if (_anchorStoreManager?.AnchorPointsSubsystem == null || _anchorStoreManager?.AnchorStore == null)
        {
            LogDebugMessage($"Can't load anchor {_worldAnchorName}: reference point subsystem or anchor store have not been initialized.", true);
            return false;
        }

        // Retrieve the trackable id from the anchor store.
        TrackableId trackableId = _anchorStoreManager.AnchorStore.LoadAnchor(_worldAnchorName);

        // Look for the matching anchor in the anchor point subsystem.
        TrackableChanges<XRReferencePoint> referencePointChanges = _anchorStoreManager.AnchorPointsSubsystem.GetChanges(Allocator.Temp);
        foreach (XRReferencePoint anchor in referencePointChanges.added)
        {
            if (anchor.trackableId == trackableId)
            {
                _xrAnchorId = anchor.trackableId;
                PositionFromAnchor(anchor);
                OnAnchorLoaded.Invoke();
                LogDebugMessage($"Found anchor {_worldAnchorName} in added reference points.");
                return true;
            }
        }

        LogDebugMessage($"Did not find anchor {_worldAnchorName} in reference points subsystem after XRAnchorStore load.", true);
        return false;
    }

    public void ClearAnchor()
    {
        _xrAnchorId = default;

        if (_anchorStoreManager.AnchorStore != null)
        {
            _anchorStoreManager.AnchorStore.UnpersistAnchor(_worldAnchorName);
        }
    }

    private void UpdateAnchorPose()
    {
        if (_xrAnchorId == default || _anchorStoreManager?.AnchorPointsSubsystem == null)
        {
            return;
        }

        TrackableChanges<XRReferencePoint> anchorChanges = _anchorStoreManager.AnchorPointsSubsystem.GetChanges(Allocator.Temp);

        foreach (XRReferencePoint anchor in anchorChanges.updated)
        {
            if (anchor.trackableId == _xrAnchorId)
            {
                PositionFromAnchor(anchor);
                break;
            }
        }
    }

    private void PositionFromAnchor(XRReferencePoint anchor)
    {
        if (_debugLabel)
        {
            _debugLabel.text = $"{_worldAnchorName} tracking: {anchor.trackingState}\r\n{anchor.pose.position}\r\n{anchor.pose.rotation}";
        }

        transform.position = anchor.pose.position;
        transform.rotation = anchor.pose.rotation;
    }

    private void AnchorStore_PropertyChanged(object sender, PropertyChangedEventArgs e)
    {
        if (LoadAnchorOnStart && e.PropertyName == nameof(AnchorStoreManager.AnchorStore) && _anchorStoreManager.AnchorStore != null)
        {
            LoadAnchor();
        }
    }

    private void LogDebugMessage(string message, bool isWarning = false)
    {
        if (_debugLabel)
        {
            _debugLabel.text = message;
        }

        if (isWarning)
        {
            Debug.LogWarning($"[{GetType()}] {message}");
        }
        else
        {
            Debug.Log($"[{GetType()}] {message}");
        }
    }
}

AnchorStoreManager

using System.Collections.Generic;
using System.ComponentModel;
using UnityEngine;
using UnityEngine.XR.ARSubsystems;
using UnityEngine.XR.WindowsMR;

public class AnchorStoreManager : MonoBehaviour
{
    public event PropertyChangedEventHandler PropertyChanged;

    private XRReferencePointSubsystem _anchorPointSubsystem;
    public XRReferencePointSubsystem AnchorPointsSubsystem
    {
        get { return _anchorPointSubsystem; }
        private set
        {
            _anchorPointSubsystem = value;
            PropertyChanged?.Invoke(this, new PropertyChangedEventArgs(nameof(AnchorPointsSubsystem)));
        }
    }

    private XRAnchorStore _anchorStore;
    public XRAnchorStore AnchorStore
    {
        get { return _anchorStore; }
        private set
        {
            _anchorStore = value;

            if (AnchorStore != null)
            {
                Debug.Log($"[{GetType()}] Anchor store initialized.");
            }

            PropertyChanged?.Invoke(this, new PropertyChangedEventArgs(nameof(AnchorStore)));
        }
    }

    private async void Start()
    {
        AnchorPointsSubsystem = CreateReferencePointSubSystem();
        AnchorStore = await _anchorPointSubsystem.TryGetAnchorStoreAsync();
    }

    private void OnDestroy()
    {
        if (AnchorPointsSubsystem != null)
        {
            AnchorPointsSubsystem.Stop();
        }
        if (AnchorStore != null)
        {
            AnchorStore.Dispose();
        }
    }

    private XRReferencePointSubsystem CreateReferencePointSubSystem()
    {
        List<XRReferencePointSubsystemDescriptor> rpSubSystemsDescriptors = new List<XRReferencePointSubsystemDescriptor>();
        SubsystemManager.GetSubsystemDescriptors(rpSubSystemsDescriptors);

        string descriptors = "";
        foreach (var descriptor in rpSubSystemsDescriptors)
        {
            descriptors += $"{descriptor.id} {descriptor.subsystemImplementationType}\r\n";
        }

        Debug.Log($"[{GetType()}] {rpSubSystemsDescriptors.Count} reference point subsystem descriptors:\r\n{descriptors}");

        XRReferencePointSubsystem rpSubSystem = null;
        if (rpSubSystemsDescriptors.Count > 0)
        {
            rpSubSystem = rpSubSystemsDescriptors[0].Create();
            rpSubSystem.Start();
        }

        return rpSubSystem;
    }
}

Be aware that when you are on AR Subsystems 3.0+, ReferencePoint has been renamed to Anchor.

5

Thank you @Thomas-Mountainborn for sharing your code. I am trying to add persistent anchors using the new XR SDK on Hololens 1 and your code has been really helpful but I cannot get it to work. I am using Unity 20.2, MRTK 2.5.3, Windows XR Plugin 4.2.1, XR Plugin Management 4.0.0 and AR Foundation 4.0.9.
Could you provide a minimal working example? I am confused on how to combine the UnityEngine.XR.ARSubsystems with the MRTK.

6

What exactly is it that won’t work? ARSubsystems and MRTK should be able to exist next to each other without much issue.

7

Hi, @Thomas-Mountainborn . Have you used local anchor transfers? I don’t know how to use it with the new XR plugin. Any advice or tips would be greatly appreciated.:slight_smile:
(Local anchor transfers enable one HoloLens device to export an anchor to be imported by a second HoloLens device. Local anchor transfers in Unity - Mixed Reality | Microsoft Learn)

8

I have not, sorry.

9

You’ll want to use this to get the anchor store: Class XRAnchorSubsystemExtensions | Windows XR Plugin | 5.3.0 (unity3d.com)

Then you can use this: Class XRAnchorStore | Windows XR Plugin | 5.3.0 (unity3d.com) to persist anchors to the SpatialAnchorStore if you haven’t already.

Then you can use the SpatialAnchorStore Class (Windows.Perception.Spatial) - Windows UWP applications | Microsoft Docs API to GetAllSavedAnchors and serialized those to a stream that you can then share/send.

From there you deserialize the stream on the target device, and use XRAnchorStore to add the anchors.

Alternatively you can use Quickstart: Create a HoloLens app with Unity - Azure Spatial Anchors | Microsoft Docs

10

What is an alternative for WorldAnchor, WorldAnchorStore and WorldAnchorTransferBatch(local anchor sharing) for WMR. Is there an minimal example implemented for this somewhere?

11

The pose right above you goes into detail about this. For the best experience and future portability I’d look at the last link.

12

Hi,
I am using this code but it shows error in the line
AnchorStore = await _anchorPointSubsystem.TryGetAnchorStoreAsync();
Because I am using OpenXR. Could you suggest me the new API reference?

13

There’s a bunch of links in joejo’s post, check those.

14

Hi, we are trying to solve a problem of sending an anchor from one HoloLens to another. Azure Spatial Anchors require using Azure and the service will be retired November 20, 2024 with no announced replacement. Using the method you described would solve our problem, however, are you sure it works ? GetAllSavedAnchors returns a set of “SpatialAnchors” objects. As far as I know, SpatialAnchor class doesn’t expose any way to get the actual anchor data and thus it cannot be serialized. And I didn’t find any way to extract Spatial Anchors from HoloLens. Or is there something I am missing ?
Thank you

15

Hi @richard_quaternar , this is a question for Microsoft. Note our FAQ :

I’m going to lock this thread accordingly. I understand that this is a concerning issue, and I wish that Unity could help you here, but it’s not our question to answer.