ARCore User face tracking and mainLightDirection estimation

Unity Engine
, ,
1

So, our AR app is using face recognition to put content on the users face.

As long as I set the ‘Light Estimation’ to only do ‘Ambient Intensity’ and or ‘Ambient Color’ it’s fine, but as soon as I flip on ‘mainLightDirection’ the scene goes to World camera AR mode, without face detection.

In our SceneKit/ArKit/iOS app light direction estimation works great while user facing.

Is it possible that ARFoundaton/ARCore Android does not support face recognition as well as light direction estimation at the same time? What are the options here?

PS: ARFoundation installed is 4.0.9, Unity 2020.2.

6654364--760360--Screen Shot 2020-12-23 at 22.17.38 .png
6654364--760360--Screen Shot 2020-12-23 at 22.17.38 .png704×270 23.3 KB

6654364--760363--Screen Shot 2020-12-23 at 22.17.42 .png

2

Does Environment Probe Estimation work with User facing AR? I added the same AREnvironmentProbeManager that is also used in the ARFoundation samples app to my scenes AR origin, yet nothing seems to happen (and my scene’s metal test ball does not receive any extra lighting).

3

PS: Delving further into this, it also appears that off the two values:

args.lightEstimation.averageColorTemperature
args.lightEstimation.colorCorrection

only the colorCorrection will ever have a value. averageColorTemperature is always null, which is kind of annoying, since this is the main value I use for getting the lighting correct on iPhones.

4

There’s no way to do this as ARCore only supports mainLightDirection in world-facing mode, and ARKit only supports it in user-facing mode.

This video describes how to check supported configurations per device:

1 Like