Trying to put a cube on AR Face (AR Foundation)

Questions & Answers
1

I’m trying to make an AR Flappy Bird game where the flappy bird is attached to the nose tip of the face tracked by ARFoundation’s Face Tracker. For the prototype, I’m using a simple 3D cube and trying to set it’s position to the nose tip of the AR Face when a face is detected. When I build it and test it on my android phone, the cube is visible initially when I haven’t pointed the camera to my face (as it should). As soon as I bring my face in front of it, it detects my face correctly (face mask renders correctly) but the cube disappears. Either the cube is being positioned somewhere outside the view of the camera or it’s too small to be visible or there is some other error. It might have something to do with localPosition and position since the cube is a child of AR Session Origin. This is the script that I have attached to AR Session Origin to set the cube’s position:


    [SerializeField] ARFaceManager faceManager;
    [SerializeField] Transform flappyBird;

    void Start()
    {
        faceManager = GetComponent<ARFaceManager>();
        faceManager.facesChanged += OnFacesChanged;
    }

    void OnFacesChanged(ARFacesChangedEventArgs args)
    {
        foreach (ARFace face in args.updated)
        {
            if (face.trackingState == TrackingState.Tracking)
            {
                Vector3 noseTipPosition = face.vertices[9]; // the index for the nose tip vertex may vary depending on the device and face mesh configuration
                flappyBird.localPosition = noseTipPosition;
            }
        }
    }

In case you’re wondering, yes, I have referenced faceManager and flappyBird correctly. I’m unsure about “face.vertices[9]” (it was something chatgpt suggested).
I also tried using this instead of the entire OnFacesChanged logic:


    void Update()
    {
        foreach (ARFace face in faceManager.trackables)
        {
            Vector3 noseTipPos = face.vertices[9] + new Vector3(0, 0, 1f); //offset in case its being rendered at the cam's position
            flappyBird.position = noseTipPos;
        }
    }

What am I doing wrong? It must be the code but I can’t seem to figure out why. I’ve tried changing position to localPosition, face.vertices[9] to face.leftEye.position (in case that works) but nothing seems to work. Is there a completely different approach I should be taking to making this game?

2

@aditdesai - It looks like you are on the right track, but there might be a couple of issues causing the cube to disappear. Here’s what you can try:

First, make sure you’re using the correct index for the nose tip vertex. The index can vary depending on the device and face mesh configuration. To find the nose tip vertex index, you can print all vertices positions in the console and manually identify the correct index.

Next, you should set the flappyBird’s position in world space, not local space. Try using flappyBird.position instead of flappyBird.localPosition. You can also apply an offset to make sure the cube is not rendered inside the face mesh.

Something like this:

void OnFacesChanged(ARFacesChangedEventArgs args)
{
    foreach (ARFace face in args.updated)
    {
        if (face.trackingState == TrackingState.Tracking)
        {
            Vector3 noseTipPosition = face.transform.TransformPoint(face.vertices[9]); // Transform local vertex position to world space
            Vector3 offset = new Vector3(0, 0, 0.1f); // Apply an offset to avoid rendering inside the face mesh
            flappyBird.position = noseTipPosition + offset;
        }
    }
}

Alternatively, you can use the Update method as you suggested, like this:

void Update()
{
    foreach (ARFace face in faceManager.trackables)
    {
        if (face.trackingState == TrackingState.Tracking)
        {
            Vector3 noseTipPos = face.transform.TransformPoint(face.vertices[9]) + new Vector3(0, 0, 0.1f); // Transform local vertex position to world space and apply an offset
            flappyBird.position = noseTipPos;
        }
    }
}

Make sure to find the correct nose tip vertex index, and try these updated methods to see if they fix the issue. If you still experience problems, please provide more details and I can try take another look for you.