ARFoundation 2 image tracking with many ref images and many objects

Hi,

I'm looking at the the ARFoundation examples -> https://github.com/Unity-Technologies/arfoundation-samples/tree/master/Assets/Scenes/ImageTracking

What's the best approach for having one object created for each tracked image? Using ARKit I created a GenerateImageAnchor for each Reference Image and Prefab To Generate, but I don't see a similar approach for ARFoundation Image tracking.

Thanks!

2 Likes

I am having the same problem here. Any help would be much appreciated.

1 Like

I'm really confused, why they haven't included that feature in the first place; image tracking is kinda useless if you cant generate different prefabs for each reference image. Maybe I'm just too spoiled from the ARKit...

Anyway, I followed the little guide by @Skeketor23 ( https://discussions.unity.com/t/706350 page-3#post-4556173 ), but couldn't make it work.

I simply put my prefabs in the ressources folder and named theme accordingly to their respective image in the referenceImageLibrary.
Working with the trackableID might be better, but I'm not entirely sure how to use them in that case (are they known a priori? Meaning before building my project?). That would not solve the issue though, I would still face the same problem:
I can't find a way to assign the prefabs to their respective image. The documentation isn't really helping either in that regard.

(It's hard to tell what's actually going on because of the lack of a remote for the editor.)

2 Likes

I use the reference image name and instantiate a prefab

private void OnTrackedImagesChanged(ARTrackedImagesChangedEventArgs eventArgs)
        {
            for (int i = 0; i < eventArgs.added.Count; i++)
            {
                eventArgs.added[i].referenceImage.name;
// use that to get the gname
}
}

Here's an example of using the tracked image event -> https://github.com/Unity-Technologies/arfoundation-samples/blob/master/Assets/Scenes/ImageTracking/TrackedImageInfoManager.cs#L97

2 Likes

[quote=“SpiderJones”, post:4, topic: 743828]
I use the reference image name and instantiate a prefab

private void OnTrackedImagesChanged(ARTrackedImagesChangedEventArgs eventArgs)
        {
            for (int i = 0; i < eventArgs.added.Count; i++)
            {
                eventArgs.added[i].referenceImage.name;
// use that to get the gname
}
}

Here’s an example of using the tracked image event → https://github.com/Unity-Technologies/arfoundation-samples/blob/master/Assets/Scenes/ImageTracking/TrackedImageInfoManager.cs#L97
[/quote] I too am trying to add this functionality.
So Spider did you end up making your own or amending the Tracked Image Manager? Would you mind posting the whole class? Thanks!

You should never add code to a framework, you should create classes that encapsulate and adapt the framework. Otherwise you make it impossible to easily update the framework. So yes, I created my own class. My class has a lot of code that is specific to my project so I'm not going to share it. I just added a listener to the event, just like here -> https://github.com/Unity-Technologi.../ImageTracking/TrackedImageInfoManager.cs#L97

Then, iterated over the other Lists like I said.

Another tip, when I instantiate my GameObject, I am adding the ARTrackedImage as the parent.

[quote=“SpiderJones”, post:6, topic: 743828]
You should never add code to a framework…
[/quote]

Thanks, Spider. I am fairly new to C# and OOP as I’m more of a tech artist being tasked with implementing AR in Unity. I’m concurrently going through these tuts: http://rbwhitaker.wikidot.com/c-sharp-tutorials as I try to dev this app. Wish me luck! :smile:

Maybe this will work for you ->

private void OnTrackedImagesChanged(ARTrackedImagesChangedEventArgs eventArgs)
        {
            ARTrackedImage trackedImage = null;

            for (int i = 0; i < eventArgs.added.Count; i++)
            {
                trackedImage = eventArgs.added[i];
                // instantiate AR object, set trackedImage.transform
                // use a Dictionary, the key could be the trackedImage, or the name of the reference image -> trackedImage.referenceImage.name
                // the value of the Dictionary is the AR object you instantiate.
            }

            for (int i = 0; i < eventArgs.updated.Count; i++)
            {
                trackedImage = eventArgs.updated[i];
                if (trackedImage.trackingState == TrackingState.Tracking)
                //if (trackedImage.trackingState != TrackingState.None)
                {
                    // set AR object to active, use Dictionary to get AR object based on trackedImage
                    // you can also include TrackingState.Limited by checking for None
                }
                else
                {
                    // set active to false
                }
            }

            for (int i = 0; i < eventArgs.removed.Count; i++)
            {
                // destroy AR object, or set active to false. Use Dictionary.
            }
        }
4 Likes

Sorry to dig this thread up over and over, but I think it might be helpful for other people working with ImageTracking in the future:

trackedImage.trackingState (in your example): Does it ever change from "tracking" to "none" for you guys? Although the camera loses the image, the TrackingState remains "tracking" (which is really weird to me).

I managed to instantiate different prefabs for different images quite easily after having the name avialable (referenceImage.name), load them from the resources folder and assign them their respective parent, but they remain active, even though the imageAnchor is nowhere to be found.

Do I misunderstand the concept of the TrackingStates or is it simply buggy?

2 Likes

use this to deactivate or hide you AR Objects:

for (int i = 0; i < eventArgs.removed.Count; i++)
            {
                // destroy AR object, or set active to false. Use Dictionary.
            }
1 Like

TrackingState has three states:
TrackingState.Limited
TrackingState.None
TrackingState.Tracking

1 Like

[quote=“SpiderJones”, post:10, topic: 743828]
use this to deactivate or hide you AR Objects:
[/quote]
Thanks for the quick reply! I am aware of the three TrackingStates. Even with your attempt, it doesn’t want to work for me (my guess is that it’s never changing to the trackingState none (or limited)):

for (int i = 0; i < eventArgs.removed.Count; i++)
        {
            trackedImage = eventArgs.removed[i];
            NamePrefab = trackedImage.referenceImage.name;
            TempPrefab = trackedImage.transform.Find(NamePrefab).gameObject;
            Destroy(TempPrefab);
        }

The code you shared is not using the "updated" List. You are using the "removed" List. So the "none" state not working is irrelevant, right? I have not tested the "none" state, but I do know that for me the "limited" state does work, and that is why I'm ignoring it, because my ARObjects would do strange things if they were visible and the TrackingState was "limited". Also, using Find is cpu intensive. You should store a local reference to your AR GameObject. Use a Dictionary with the referenceImage.name as the key, and the AR GameObject as the value. and maybe there is an issue with how you are trying to find the AR GameObject, are you using the referenceImage.name as the name for your AR GameObject? When you instantiate a Prefab it adds clone to the end of the name. Regardless, I recommend that you do not use this approach, use a Dictionary.

You can see my app that uses ARFoundation image tracking here -> http://www.superherorobot.com/projects/crispr-3d/

[quote=“SpiderJones”, post:13, topic: 743828]
[…]but I do know that for me the “limited” state does work, and that is why I’m ignoring it, because my ARObjects would do strange things if they were visible and the TrackingState was “limited”.
[/quote]
First of all: Thanks for the idea with the dictionary. For me, this only solved my issue to an extend, because it will only switch to trackingState.limited, if you lay down the device on a table and wait for ~2 seconds (found it out through pure chance with some debug logs). The problem is: only then the prefab.setactive is set to be false. The ARKit directly changes the state directly after losing its ImageAnchor, 2 seconds seem to be quite long (besides the annoying laying down part, which is extremely weird). I assume you didn’t have the same experience?

I created a video to further show my problem:
https://drive.google.com/open?id=1XP4dkKymXubgTILautDcOvTu1eNBtijf

The debug log shows the following:
“we are in the updateCycle with trackingstate Tracking”
or
“we are in the updateCycle with trackingstate!= Tracking”

2nd one only appears after laying down the device. Holding it still without image in the camera view doesnt result in a trackingState change to limited… -.-

Your app looks very clean, good job!

Thank you for your compliment. I'm sorry, but I don't fully understand your issue. In my app, when users show the camera the image marker (playing card), the AR object shows, when the card is hidden, the object doesn't show. There are 13 AR objects in my app that show for each type of card. My app works like Vuforia (on iOS only because ARKit tracks moving images while ARCore does not). I am using the approach I shared, and I'm ignoring the "Limited" state. When it's "Tracking" I show the ARObject, when it's "None" or "Limited" I hide it. And I use the "remove" list to hide AR Objects. And it works perfectly. I do not know what else I can do to help you. Good luck!

1 Like
using System.Collections;
using System.Collections.Generic;
using UnityEngine;
using System;
using UnityEngine.XR.ARSubsystems;
using UnityEngine.XR.ARFoundation;

public class ARPerImage : MonoBehaviour
{

            Dictionary<string, GameObject> myPrefabsDictionary =
            new Dictionary<string, GameObject>();

            List<GameObject> o = new List<GameObject>();

    // Start is called before the first frame update
    void Start()
    {
        myPrefabsDictionary.Add("Rafflesia", Resources.Load("Rafflesia") as GameObject);
        myPrefabsDictionary.Add("Logo", Resources.Load("Logo") as GameObject);
        myPrefabsDictionary.Add("QRCode", Resources.Load("QRCode") as GameObject);
    }

    // Update is called once per frame
    void Update()
    {

    }

        private void Awake()
    {
        var arTrackedImageManager = gameObject.GetComponent<ARTrackedImageManager>();
        arTrackedImageManager.trackedImagesChanged += OnTrackedImagesChanged;
    }

    private void OnTrackedImagesChanged(ARTrackedImagesChangedEventArgs eventArgs)
    {
        ARTrackedImage trackedImage = null;
        // Check the new tracked images
        for (int i = 0; i < eventArgs.added.Count; i++)
        {
            trackedImage = eventArgs.added[i];
            string imgName = trackedImage.referenceImage.name; // this is the name in the library
            GameObject prefab = myPrefabsDictionary[imgName];
            o.Add( Instantiate(prefab, trackedImage.transform));
        }

        for (int i = 0; i < eventArgs.updated.Count; i++)
        {
            trackedImage = eventArgs.updated[i];
            if (trackedImage.trackingState == TrackingState.Tracking)
                o[i].active=true;
            else
                o[i].active=false;

        }

        for (int i = 0; i < eventArgs.removed.Count; i++)
        {
                o[i].active=false;

        }
    }

}

can anyone help me figure this out? the object is spawning alright, but it keeps active event if i point to another target.....

1 Like

@brunno159 seems to be an Android specific problem: https://github.com/Unity-Technologies/arfoundation-samples/issues/157

[quote=“brunno159”, post:17, topic: 743828]

using System.Collections;
using System.Collections.Generic;
using UnityEngine;
using System;
using UnityEngine.XR.ARSubsystems;
using UnityEngine.XR.ARFoundation;

public class ARPerImage : MonoBehaviour
{
  
            Dictionary<string, GameObject> myPrefabsDictionary =
            new Dictionary<string, GameObject>();
          
            List<GameObject> o = new List<GameObject>();
          
    // Start is called before the first frame update
    void Start()
    {
        myPrefabsDictionary.Add("Rafflesia", Resources.Load("Rafflesia") as GameObject);
        myPrefabsDictionary.Add("Logo", Resources.Load("Logo") as GameObject);
        myPrefabsDictionary.Add("QRCode", Resources.Load("QRCode") as GameObject);
    }

    // Update is called once per frame
    void Update()
    {
      
    }
  
        private void Awake()
    {
        var arTrackedImageManager = gameObject.GetComponent<ARTrackedImageManager>();
        arTrackedImageManager.trackedImagesChanged += OnTrackedImagesChanged;
    }
   
    private void OnTrackedImagesChanged(ARTrackedImagesChangedEventArgs eventArgs)
    {
        ARTrackedImage trackedImage = null;
        // Check the new tracked images
        for (int i = 0; i < eventArgs.added.Count; i++)
        {
            trackedImage = eventArgs.added[i];
            string imgName = trackedImage.referenceImage.name; // this is the name in the library
            GameObject prefab = myPrefabsDictionary[imgName];
            o.Add( Instantiate(prefab, trackedImage.transform));
        }
      
        for (int i = 0; i < eventArgs.updated.Count; i++)
        {
            trackedImage = eventArgs.updated[i];
            if (trackedImage.trackingState == TrackingState.Tracking)
                o[i].active=true;
            else
                o[i].active=false;
              
        }
      
        for (int i = 0; i < eventArgs.removed.Count; i++)
        {
                o[i].active=false;
              
        }
    }

}

can anyone help me figure this out? the object is spawning alright, but it keeps active event if i point to another target…
[/quote]

Hi Bruno,

I tried running your code on my project (changing the Gameobject names in the Dictionary and such to match my own) but I wasn’t able to even spawn any objects at all with my images.

I have all the Objects I want to spawn inside a folder named Resources (to be able to use the Resources.Load, but I wasn’t able to call the Gameobjects and I can’t seem to get a console log when running it on my iOS to check for errors…

I imagine you are using Android but could you give some light to this?

Thank you!

I've been struggling with this, but managed to stitch something together (for iOS) using the different topics on this forum.
I've taken a slightly different approach, using a serialized array to store the connections between image targets and prefabs via the Inspector. I'm actually using GameObjects already in the scene instead of instantiating prefabs from Project, because I want changes in the objects (e.g. animations, user interactions) to stay in tact between tracking sessions, so to speak.

using System.Collections;
using System.Collections.Generic;
using UnityEngine;
using UnityEngine.UI;
using UnityEngine.XR.ARSubsystems;
using UnityEngine.XR.ARFoundation;

[System.Serializable]
public class MarkerPrefabs
{
    public string marker;
    public GameObject targetPrefab;
}

public class SwitchPrefab : MonoBehaviour
{
    /* Insepctor array */
    public MarkerPrefabs[] markerPrefabCombos;
    ARTrackedImageManager m_TrackedImageManager;

    void Awake()
    {
        m_TrackedImageManager = GetComponent<ARTrackedImageManager>();
    }

    void OnEnable()
    {
        m_TrackedImageManager.trackedImagesChanged += OnTrackedImagesChanged;
    }

    void OnDisable()
    {
        m_TrackedImageManager.trackedImagesChanged -= OnTrackedImagesChanged;
    }

    private void OnTrackedImagesChanged(ARTrackedImagesChangedEventArgs eventArgs)
    {
        foreach (var trackedImage in eventArgs.updated)
        {
        /* If an image is properly tracked */
        if (trackedImage.trackingState == TrackingState.Tracking) {

                /* Loop through image/prefab-combo array */
                for(int i = 0; i < markerPrefabCombos.Length; i++)
                {
                    /* If trackedImage matches an image in the array */
                    if (markerPrefabCombos[i].marker==trackedImage.referenceImage.name) {

                        /* Set the corresponding prefab to active at the center of the tracked image */                     
                        markerPrefabCombos[i].targetPrefab.SetActive(true);
                        markerPrefabCombos[i].targetPrefab.transform.position = trackedImage.transform.position;
                    }
                }            
            /* If not properly tracked */
            } else {

                /* Deactivate all prefabs */
                for(int i = 0; i < markerPrefabCombos.Length; i++) {
                    markerPrefabCombos[i].targetPrefab.SetActive(false);
                }
            }
        }
    }

}
2 Likes