Balancing biped

Unity Engine
,
1

Howdy guys

I want to balance a biped on a shifting platform ( think a ship at sea ). I’ve created the project but Im worried that my policy for the ml agent wont be feasible.

My attempt here is to have the zmp and compoint laying almost ontop of eachother, which I reward the
agent for. Please correct me if this the wrong way of doing this.

I have some concerns while looking at the training.

  • velocity should probably be apart of calculation of center of mass point.
  • I’ve come across “inverted pendelum” which I think in principle is reach the zero momentum
    for balance ? but from what I saw that project required very high framerate for it to be accurate.

All in all I just wanted to spruce up a ragdoll state but I would be cool if I could get it
rebalance - but Im not holding my breath.

Im thankful for any input on this or experiance !

        Vector3 zmpPoint = PointOnDirectionPercent(rFoot.joint.transform.position, lFoot.joint.transform.position, 0.5f);

        if (rFootContact.HasContact() && !lFootContact.HasContact())
        {
            zmpPoint = rFoot.joint.transform.position;
        }
        else if (!rFootContact.HasContact() && lFootContact.HasContact())
        {
            zmpPoint = lFoot.joint.transform.position;
        }

        Vector3 zmpLoc = root.transform.InverseTransformPoint(zmpPoint);
        zmpLoc.y = 0;
        Vector3 comLoc = root.transform.InverseTransformPoint(CoMPoint);
        comLoc.y = 0;

        // Todo calculate total centermass of all character

        Vector3 comZmpDelta = zmpLoc - comLoc;
        float CoMScore = Mathf.Clamp(1 - comZmpDelta.magnitude, 0, 1);
        //Debug.Log("CoMScore : " + CoMScore);

        float headHeightOverFeetReward = (head.position.y - lFoot.joint.transform.position.y) + (head.position.y - rFoot.joint.transform.position.y);

        AddReward(
            +0.01f * (CoMScore * headHeightOverFeetReward)
        );

Here is a snippet.

The CoMpoint is the center of mass of all the articulated joints - calculated like this.

        Vector3 CoM = Vector3.zero;
        float c = 0f;

        foreach (ArticulationBody part in assembly)
        {
            CoM += part.worldCenterOfMass * part.mass;
            c += part.mass;
        }


        return CoM /= c;
2

You can try using a body root object (hips, torso etc) and reward the agent for minimizing its inclination.
Something like

float normalized_inclination = Mathf.Clamp01(Vector3.Angle(Vector3.up, body_root.up) / 90f);
float reward = Mathf.Pow(1 - normalized_inclination, exponent) * reward_factor;

Or maybe calculate the average up angle of a couple of body parts.

3

Hello mbaske - seen your stuff on youtube !

I’ve replaced the balancing mess I did with this - and its looking better. However the agent starts sliding its feet
rather then taking steps to rebalance. I’ve come up with a solution - training as we speak. Once a foot is grounded
it sets a vector3 pos - I then reward the agent for velocity towards that target with the foot.

I will report back with my findings to the thread.

Again if anyone else dealt with the sliding feet and reluctance to take steps please reply :slight_smile:

Edit

Btw about the sliding feet - Im not talking about friction - kinda like vibrating its feet to move to a more balance pose - typical agent non humanlike behaviour such is life.