Custom mesh for AR Face Visualizer

Unity Engine
1

Hello, I want to make AR Face tracking for Android with AR Foundation. AR Face Mesh Visualizer works great, but I can not find the way to use my own custom mesh.

2

Unfortunately, you can’t use a custom face mesh with ARCore/ARKit. This mesh is generated by ARCore/ARKit at runtime.

What you can do - is to place your mesh on top of a tracked face. You can use ARCore face regions to place different parts of your mesh on top of left/right forehead and nose tip.

3

Thanks for the answer. I tried “ARCoreFaceRegions” sample from AR Foundation Sample. But it detect just 3 points on my face: 2 eyebrow and nose. It looks that mouth tracking is not detected.

4

@Cenda yes, the mouse position is not included to face regions. I made a test and found ARFace indices that correspond to the main mouse points.
1: top lip, 14: bottom lip, 78: left mouse corner, 292: right mouse corner

Here is a quick example, you can find other indices with it. Add this script to your face prefab. Also, add ParticleSystem to visualize mesh points. This is a dirty hack and there is no guarantee that face indices will not be changed in future versions of ARCore.

using System.Linq;
using UnityEngine;
using UnityEngine.XR.ARFoundation;


public class FaceMeshTest : MonoBehaviour {
    [SerializeField] new ParticleSystem particleSystem = null;
    [SerializeField] ARFace face = null;
    [SerializeField] int skipNumOfIndices = 1; // 1: top lip, 14: bottom lip, 78: left mouse corner, 292: right mouse corner
    [SerializeField] int visualizeNumOfIndices = 1;


    void Awake() {
        face.updated += delegate {
            var mainModule = particleSystem.main;
            var particles = face.vertices
                .Skip(skipNumOfIndices)
                .Take(visualizeNumOfIndices)
                .Select(pos => new ParticleSystem.Particle {
                    startColor = mainModule.startColor.color,
                    startSize = mainModule.startSize.constant,
                    position = pos,
                    remainingLifetime = 1
                })
                .ToArray();
             
            particleSystem.SetParticles(particles, particles.Length);
        };
    }
}
2 Likes
5

What a great tip! Thank you :slight_smile: I can get position of these components and move with my bones on the face :slight_smile:

6

How to use this? this display nothing !! i thought this will display particle on the detected face but nothing happens!!

7

Please make sure that you’ve populated properties in Inspector and set up ParticleSystem correctly.

8

I had followed all steps :

AR Foundation setup :
Add face Manager
Create new AR Default face and add your script and assign the same object as Face and debug particle
create a prefab of Face and assign in face manager.
Run in device

Face prefab displayed but nothing else !! even I tried transparent material on face object

If possible can you provide a demo GitHub project for the test ?

9

I replaced ParticleSystem with spheres and uploaded an example to github:

3 Likes
10

Great. It works, Thanks dude for helping.