Humanoid avatar using Mediapipe BlazePose

Unity Engine
,
1

I am making an exercise game using Mediapipe’s BlazePose in Unity through BlazePoseBarracuda (GitHub - creativeIKEP/BlazePoseBarracuda: BlazePoseBarracuda is a human 2D/3D pose estimation neural network that runs the Mediapipe Pose (BlazePose) pipeline on the Unity Barracuda with GPU.). I am using a HMD, but not the hand-held controllers. I am trying to map the output of the pose landmarks from BlazePose to the joints of an avatar that I want to use to represent the player. However, I am struggling as all the examples I find use the position of the controllers to get the hand positions. Any suggestions on how I can do this without the use of controllers? I am taking inspiration from this project: GitHub - maceq687/FullBodyPoseEstimation: Full body pose estimation to be used with HMD (Quest2) built in Unity

Thanks in advance!

2

I’m working in the same topic. An Humanoid avatar using Mediapipe, but with a slightly different approach, I am adapting the Pose landmark detection data towards that required by Unity’s Avatar structure. For now I have significant progress but there are others to be resolved.