ML Agents and New Input System

Unity Engine
,
1

I have 6 actions mapped to 6 inputs buttons using the new input system. However, when I add another action along with its button, the agents keep repeating the same first actions forever! I noticed that since I am using a heuristic brain. I use a “Decision Requester” component.

I suspect it has something to do with RequestAction() running underneath although I never call this function in my code.

Any suggestions to prevent repetitive behavior are welcome.

2

Can anybody advise why the RequestAction() is being called by itself? What is the purpose of such behavior? and how can I stop it?

3

Are you retraining the agent with the 7 actions or are you using the heuristic function from the previous version of the action space? Is the behavior type set to default or heuristic in the behavior parameters script?

4

Thank you andrewcoh for replying.

I am not in the training phase at all. I only use the heuristic function for now. The behavior type is set to “Heuristic”.

Here is my heuristic function:-


public override void Heuristic(float[ ] actionsOut)
{
controls.Enable();

// Control animation states and speed
///ControlAnimation();

actionsOut[0] = controls.Player.Move.ReadValue<Vector2>()[0];
actionsOut[1] = controls.Player.Move.ReadValue<Vector2>()[1];

///Listen to Sprint Button
if (controls.Player.Sprint.ReadValue<float>() == 1)
{
actionsOut[2] = 5f;
}

if (possess_the_ball)
{

if (controls.Player.Pass.ReadValue<float>() == 1)
{
anim.Play("kick_near");
actionsOut[2] = 1f;

} else if (controls.Player.LongBall.ReadValue<float>() == 1)
{
actionsOut[2] = 2f;

} else if (controls.Player.ThroughBall.ReadValue<float>() == 1)
{
actionsOut[2] = 3f;

} else if (controls.Player.ShootsAtGoal.ReadValue<float>() == 1)
{
actionsOut[2] = 4f;
anim.Play("kick_far");
}

}
else if (possess_the_ball == false && controls.Player.Pass.ReadValue<float>() == 1)
{
actionsOut[2] = 6;
}

}

Notice the final else. When I comment it and reduce the number of actions by 1, the actions keep repeating. and when I uncomment it and increase the number of actions by 1, the actions repeat occasionally.

Also, when I replace the last action with (and add the binding)

else if (possess_the_ball == false && controls.Player.Tackle.ReadValue<float>() == 1) {

instead of

else if (possess_the_ball == false && controls.Player.Pass.ReadValue<float>() == 1)```
it keeps repeating forever.

I tried to remove the parent if statement to test, but the problem persists"
if (possess_the_ball)
{
"
I tried to clear the GI cache, but with no success.

It seems to me as a Cache problem as the code does not want to be changed!
1 Like
5

So, when you say ‘6 actions’ do you mean you have 6 action branches i.e. the array actionsOut should contain an entry for actionsOut[0…5] or that you have fewer action branches but with 6 possible values i.e. actionsOut[0] = [1…6]. Can you screenshot your behavior parameters script?

6

I attached a screenshot of the behavior parameters. I use this component on two opposing teams differentiating them with team ID 1 and 2.

This is my OnActionReceived() function

  public override void OnActionReceived(float[] vectorAction)
    {

        // continously monitor last touch
        last_touch_player_team();


        if (IsEngagementInPassIOrShootDone())
        {
            isEngagedInPassShoot = false;
        }
        AttachBallToPlayer();
        //Defend();
        ToggleDefenderWinsToFalse();

        MoveThePlayer(new Vector2(vectorAction[0], vectorAction[1]));


        switch (vectorAction[2])
        {

            case 0: break;
            case 1:
                try
                {
                    Pass(targetPlayer.GetSiblingIndex());
                }
                catch (System.Exception ex)
                {
                    print("ERROR:  " + ex);
                }
                break;
            case 2:
                LongBall(targetPlayer.GetSiblingIndex());
                break;
            case 3:
                ThoroughBall(targetPlayer.GetSiblingIndex());
                break;
            case 4:
                Shoot(lookingDirection);
                //KickBall();
                break;
            case 5:
                Sprint(new Vector2(vectorAction[0], vectorAction[1]));
                break;
            case 6:
                Defend();
                break;
        }
}

6078858--659559--Screen Shot 2020-07-11 at 2.09.35 PM.png
6078858--659559--Screen Shot 2020-07-11 at 2.09.35 PM.png656×716 55.4 KB

7

Ah, I see. Can you turn the decision frequency on the decision requester down to 1 and try playing your agent?

8

I turned the decision requester to 1 with “Take Actions Between Decisions” to false, and also to true, but the problem persists.

9

Problem solved! I added an else statement for action # 0 in the heuristic method. I did not know that was necessary. Thanks for helping anyway.