How can I animate a humanoid avatar using only a CSV file? S.O.S.

Unity Engine
1

Hello everyone,

I’m currently trying to animate my humanoid avatar with a CSV (comma separated values) file that contains the desired x,y,z coordinates of each joint, frame after frame (no rotation information). I want to accomplish this myself without the use of any 3rd party software.

Current Solution: I set my avatar’s rig to humanoid and wrote a script to attach every joint on the avatar to the desired Vector3 coordinate in the csv file. I’m also using Unity’s IK system to connect the avatar’s wrists and ankles to the red “skeleton” that I constructed out of the CSV points and DrawLine() functions. The problem is that only the wrists and ankles connect, but none of the other joints are connecting! When I disable the IK, my avatar completely breaks! (Depicted in the image) I would really appreciate any insight, suggestions, or tutorials. Thanks!
.

3159244--240342--mocapIssue4.PNG
3159244--240342--mocapIssue4.PNG889×572 254 KB

3153244--239706--mocapIssue1.PNG
3153244--239706--mocapIssue1.PNG712×528 180 KB

3159244--240340--mocapIssue2.PNG
3159244--240340--mocapIssue2.PNG733×535 181 KB

3159244--240341--mocapIssue3.PNG
3159244--240341--mocapIssue3.PNG876×569 254 KB

Retargeting a human pose from motion tracking data that includes joint positions
2

Hi,

The only transform on a simple humanoid skeleton you should script translate is root (unless you want to make some strange behavior).

If you do not have access to joint rotation data then you should derive them from the mocap translations (recursing the heirachy), then per frame animate your model using joint rotations.

Stick to Quaternions.

1 Like
3

If Ragdoll kinda works then
Solution 2 : Make all Rigidbody of RagDoll to kinematic so that they are not influenced by physics. That maybe your best solution.

Please post the link to the project too. :slight_smile:

4

Is the animator on the character playing something ?

Does the CSV have local or global positions ? Maybe post a sample of the CSV

Extra question:
Why CSV ? :frowning:

5

Trying to puppeteer a ragdoll with translations is a hack that will not yield good results.

When you translate a joint in the real world… if you move it to a position that does not coincide with joints natural loci… it will dislocate the joint or break the bone, which in unity hinge joints is a “spaz out” or break.

Optimal solution will derive joint rotations.

Start from the root, try and calculate the rotation of that joint using just the two points that describe its ends. once you got that, recurse the whole heirachy, transforming each joints worldspace rotation into localspace of its parent joint. You may also have to apply rotation offset to these derived rotations before applying to avatar, depending on how your avatar is rigged.

As I warned before… stick to quaternions, lest your compass fails and you sail into monsters.

NOTE: to be a good general solution it will have to take into account differences (bone lengths) between actors skeleton and avatars skeleton if there is one, but that comes later.

1 Like
6

Hi Silicon Droid,
By the way, amazing work on Galaxy VR! Your game really helps keep the bar high for mobile VR.

I really like your idea of deriving the joint rotations, but I have another issue before I can get to that stage. I included a video of my project. What I did so far is to use the standard Unity IK system to connect the wrists and ankles of my 3D Avatar to the red “skeleton” that I made out of the CSV positional data. The red “bones” are created with simply ‘DrawLine(joint1, joint2, Red)’. Using IK is great at keeping my Avatar connected at the wrists and ankles, but the rest of the body won’t connect to the joints of the skeleton. Any advice on this? In my code, I’m assigning the position of the avatar’s joints to the corresponding Vector3 positions from the skeleton, but they refuse to connect. I think it has to do with the idle animation that I gave the avatar (which Unity requires if I want IK functionality). When I take away the animation and the IK, the joints all connect, but it completely destroys the avatar’s limbs. (Illustrated in my original images). Furthermore, SetIKHintPosition (with a weight of 1) isn’t doing a very good job at keeping the avatar’s knees and elbows connected to the skeleton.

private void Update(){

  head = animator.GetBoneTransform (HumanBodyBones.Head);
  head.position = neckPositionVector;
}

I’m very lost at this point. Do you have any insight on how to keep the IK and also connect the rest of the avatar’s joints to the skeleton’s?

1 Like
7

Hi guillermoi,

Yes, the Avatar’s animator is playing an idle animation, since at least one animation is required to use Unity IK. The CSV has all global Vector3 positions of the joints. Each row is a frame. It looks something like this:

-32.691,-500.68,5130.7,99.624,-494.62,5068.7,102.06,-47.594,5192.8,132.95,340.4,5348,-141.21,-506.74,5192.8,-117.59,-104.65,5348,-91.833,319.84,5472.1,-32.295,-731.89,5068.7,-5.839,-983.33,5006.6,-31.307,-1067.3,4913.5,-5.8028,-1132.5,4975.6,150.53,-957.29,5006.6,406.04,-893.98,4944.6,564.13,-860.32,4758.4,-165.22,-975.09,5099.7,-389.07,-950.09,5254.9,
-32.649,-500.04,5124.2,99.495,-493.98,5062.1,101.93,-47.533,5186.2,133.56,341.96,5372.4,-141.03,-506.09,5186.2,-117.44,-104.52,5341.4,-92.243,292.68,5496.5,-32.253,-730.94,5062.1,-5.8313,-1008,5000,-31.265,-1065.8,4906.9,-5.7951,-1131,4969,150.33,-956.03,5000,402.95,-887.18,4906.9,563.35,-859.13,4751.8,-165.01,-973.83,5093.1,-388.58,-948.9,5248.3,
-8.0585,-497.63,5054.4,95.715,-491.52,4992.3,98.094,-51.352,5116.4,128.44,330.95,5271.6,-140.97,-503.74,5116.4,-117.88,-107.75,5271.6,-93.174,312.46,5426.8,-7.9596,-725.21,4992.3,-7.8606,-998.27,4930.3,-32.825,-1054.9,4837.2,-33.246,-1119.4,4899.2,145.72,-946.98,4930.3,394.1,-903.94,4837.2,547.95,-844.96,4651,-138.41,-964.86,5023.4,-386.95,-946.44,5209.5,

My current problem is keeping the avatar’s joints connected to the corresponding positions from the csv file. I was able to give the avatar’s body correct rotation by creating rectangle game objects that I then connected to the skeleton’s left and right hip joints. I have these rectangles LookAt() each other every frame. I am then able to match the avatar’s LookRotation() to match the rotation of these rectangles.

8

@Virtual_Velociraptor :

Try applying in LateUpdate (it may not work but worth a shot):

        private void LateUpdate()
        {

            head = animator.GetBoneTransform(HumanBodyBones.Head);
            head.position = neckPositionVector;
        }
9

Don’t know if it’s still relevant for you, but I used this approach the other day:
https://github.com/mariusrubo/Unity-Humanoid-Mocap-CSV

10

Hi dude,
it’s a long time for this project you did and hope you solve the problem.

i wonder if you can help me, i had a task to create and animate a skeleton bones using csv file data so i hope you can figure out my way and indicate any resources to assist me to do my task, thank you.

11

Hello,

I hope this message reaches you, I am also having same issue I am using local position and local rotation and I am not figure out the issue with this I am having similar results from yours. I hope and wish if you good could help me of how you solved the issue.

Thank you in advance.

12

Hi guys, I think you get some resources for this. I also having same problem please give some resources.

13

Hey, could you please give me the resource/code if you got it? @shyam_123 @memadeddin