how to build AR application for necklace try-on using face detector AR foundation

Unity Engine
, ,
1

Need help with placing the necklace object at the appropriate location but unable to do so.
I’ve been following the AR Unity learning course that builds an application for glasses.
Any suggestion as to how should I proceed?

2

This is probably going to be a somewhat challenging app to design with respect to the capabilities of today’s mobile AR platforms.

You would probably proceed in one of these two ways (or try both and see which works better for you):

  1. Use face tracking in the user-facing camera to get a face position, and place the necklace on the user by calculating an offset from the face position.

  2. Use body tracking in the rear-facing camera to get a body position, and place the necklace on the user by calculating an offset from a known body joint.

We have a samples app that you can use as a getting started point that includes demos of both of these features: GitHub - Unity-Technologies/arfoundation-samples: Example content for Unity projects based on AR Foundation

3

Well I am trying the first approach you’ve mentioned, using the AR face manager, if I place my necklace prefab in the face prefab field, it appears on the face of the user instead of the neck.
i did write this script below

using System.Collections;
using System.Collections.Generic;
using UnityEngine;
using UnityEngine.XR.ARFoundation;
using UnityEngine.XR.ARSubsystems;

public class NecklacePlacement : MonoBehaviour
{
[SerializeField]
private ARFaceManager arFaceManager;

[SerializeField]
private ARAnchorManager arAnchorManager;

[SerializeField]
private GameObject necklacePrefab; // Reference to your necklace prefab

private void Start()
{
if (arFaceManager == null || arAnchorManager == null || necklacePrefab == null)
{
Debug.LogError(“Missing required components or references!”);
return;
}

// Subscribe to the AR Face Manager events
arFaceManager.facesChanged += OnFacesChanged;
}

private void OnFacesChanged(ARFacesChangedEventArgs args)
{
foreach (var face in args.added)
{
// Create an anchor at the detected face’s position
CreateNecklaceAnchor(face.transform.position, face.transform.rotation);
}
}

private void CreateNecklaceAnchor(Vector3 position, Quaternion rotation)
{
// Adjust the offset to move the necklace down from the face to the neck
Vector3 offset = new Vector3(0f, -1.5f, 4f);
position += offset;
// Create an anchor at the detected neck position using AddComponent()
// ARAnchor anchor = arAnchorManager.AddAnchor(new Pose(position, rotation));
ARAnchor anchor = arAnchorManager.gameObject.AddComponent();
anchor.transform.position = position;
anchor.transform.rotation = rotation;
Debug.Log("Creating necklace anchor at position: " + position + ", rotation: " + rotation);

// Instantiate and attach the necklace prefab to the anchor
GameObject necklaceInstance = Instantiate(necklacePrefab, anchor.transform.position, anchor.transform.rotation);
necklaceInstance.transform.SetParent(anchor.transform);
}
}

But the necklace prefab doesn’t appear on the face.
Is there an approach where using the AR face manager, i try to add some ar default face in face prefab field and then position these necklaces?
what should i do?
Kindly help.

4

You would have to use either the face position or the vertices of the face mesh to see if there is way to calculate an offset that works for you.

5

I’m done with placing the necklaces by using an ar default face and then just facing the necklace model.
In my UI canvas, I have buttons that represent each necklace, so I want to render that particular necklace only when it clicked on, how can I achieve that?

6

Unity employees may not be able to help you with general app development questions in the forums. UI and buttons work the same in AR as they do in non-AR projects, so I recommend that you seek more information in places such as Unity Learn or YouTube to understand how to build UI in Unity.

7

Okay thanks

1 Like
8

My English is not well enough to explain perfectly but I try my best
Unity version 2022.3.19f1
using 3D(URP) Core

@andyb-unity When I am using a face mesh to a put 3D necklace on a neck in unity I test on a necklace in my Android phone so the necklace will not be perfectly stable on the neck. Necklace automatically move means high and width will automatically change.
How to fix these issues

one more question for you
without a mesh I can do it 3D necklace on face perfectly

9

Unity employees cannot give you development advice outside the domain of Unity API’s. I’m happy to help if you have questions about AR Foundation. It is your responsibility to design and build solutions for your use case.