No action in OnActionReceived with Instantiate Prefab Agent

Unity Engine
, ,
1

Hi. I have a model that has an agent which I want to take continues steps in the x/z direction on a plane. I have trained other models with this same code in different projects with no problem. The difference here is that I am instantiating the agent in the environment script from prefab that has the behavior script attached. Is it not possible to instantiate the agent automatically from a separate script?

The agent is instantiated and shows up in my training area, but upon training w/SAC or using heuristic, it does not move.

If I print ControlSignal in line 51, the value is 0. Also I tried setting the behavior type to default for training and heuristic when I was using the keyboard. Neither produce any action.

Here is my code for the agent.

using System.Collections.Generic;
using UnityEngine;
using Unity.MLAgents;
using Unity.MLAgents.Sensors;

public class bunAgent : Agent
{

    private Rigidbody rBody;
    private bool isDead;
    public float bunSpeed = 10;

    // Start is called before the first frame update
    void Start()
    {
        rBody = GetComponent<Rigidbody>();
    }

    public override void OnEpisodeBegin() //how to "birth" the agent and target
    {
        if (this.transform.localPosition.y < -5f)
        {
            // If the Agent fell, zero its momentum
            this.rBody.angularVelocity = Vector3.zero;
            this.rBody.velocity = Vector3.zero;
            this.transform.localPosition = new Vector3(0, 1f, 0);
        }
    }



    public override void CollectObservations(VectorSensor sensor)
    {
        // // Target and Agent positions
        // sensor.AddObservation(Target.localPosition);
        // sensor.AddObservation(this.transform.localPosition);
        // // Agent velocity
        // sensor.AddObservation(rBody.velocity.x);
        // sensor.AddObservation(rBody.velocity.z);
    }




    public override void OnActionReceived(float[] vectorAction)
    {
        // Actions, size = 2
        Vector3 controlSignal = Vector3.zero;
        controlSignal.x = vectorAction[0];
        controlSignal.z = vectorAction[1];
        rBody.AddForce(controlSignal * bunSpeed);

        // Rewards
        SetReward(.01f);

        // Fell off platform
        if (this.transform.localPosition.y < -5)
        {
            SetReward(-1f);
            EndEpisode();
        }
    }

Here is my code that is instantiating the agent (a bit simplified to keep it short).

using Unity.MLAgents;
using TMPro;
using System.Collections;
using System.Collections.Generic;
using UnityEngine;
using System;

public class forestEnv : MonoBehaviour
{

    [Header("Prefabs")]
    public GameObject Wild_rabbit_prefab;

    [Header("Hyperparameters")]
    public int numBunnies;
    public float spawnRange;

    private List<GameObject> spwndBunnies;
    private List<Tuple<Vector3, float>> occupiedPositions;


    // Start is called before the first frame update
    void Start()
    {
      ResetArea();
    }

    void ResetArea()
    {
      occupiedPositions = new List<Tuple<Vector3, float>>();
      ResetBunnies();
    }

    private void ResetBunnies()
    {
      if (spwndBunnies != null)
      {
        foreach (GameObject spwndBun in spwndBunnies.ToArray())
        {
          Destroy(spwndBun);
        }
      }
      spwndBunnies = new List<GameObject>();
      for (int i=0; i < numBunnies; i++)
      {
        GameObject bunInstance = Instantiate(Wild_rabbit_prefab, transform);
        RandomlyPlaceObject(bunInstance, spawnRange, 5);
        spwndBunnies.Add(bunInstance);

      }
    }
2

It turns out you need kinematic in the Rigidbody to be off.