Hello!
I’m learning to use the ML-Agents tool and I’m facing some problem…
I wrote a class called “MoveToGoalAgent” in which the Agent (Cube) moves to the goal (Capsule). The “Heuristic” class tracks button clicks (arrows) and passes values to “ActionBuffers”. In theory, my Agent should move by pressing keys (arrows) if I launch the project even without training the model, but this does not happen.
My character only spawn (Rigidbody are on), fall to the panel and stand. Even if I start training model (in the hope that something will change): change on “Behavior type: Default” and running console “mlagents-learn --force --run-id=test1” - still nothing moves.
Сode “MoveToGoalAgent”:
using System;
using System.Collections;
using System.Collections.Generic;
using System.Numerics;
using Unity.MLAgents;
using Unity.VisualScripting;
using Unity.MLAgents.Sensors;
using UnityEngine;
using static UnityEngine.GraphicsBuffer;
using UnityEngine.UIElements;
using Unity.MLAgents.Actuators;
public class MoveToGoalAgent : Agent
{
[SerializeField] private Transform targetTransform;
public override void OnEpisodeBegin()
{
transform.position = new UnityEngine.Vector3(0f,1f,0f);
//Y-axis = 1f to prevent the Agent from falling through the platform
}
public override void CollectObservations(VectorSensor sensor)
{
sensor.AddObservation(transform.position);
sensor.AddObservation(targetTransform.position);
}
public override void OnActionReceived(ActionBuffers actions)
{
float moveX = actions.ContinuousActions[0];
float moveZ = actions.ContinuousActions[1];
float moveSpeed = 1f;
transform.position += new UnityEngine.Vector3(moveX, 0, moveZ) * Time.deltaTime * moveSpeed;
Debug.Log(moveX);
Debug.Log(moveZ);
}
private void OnTriggerEnter(Collider other)
{
if (other.TryGetComponent<Wall>(out Wall wall))
{
SetReward(-1f);
Debug.Log("-1 reward");
}
if (other.TryGetComponent<Goal>(out Goal goal))
{
SetReward(+1f);
Debug.Log("+1 reward");
EndEpisode();
}
}
public override void Heuristic(in ActionBuffers actionsOut)
{
Debug.Log("Heuristic method work...");
ActionSegment<float> continuousActions = actionsOut.ContinuousActions;
continuousActions[0] = Input.GetAxisRaw("Horizontal");
continuousActions[1] = Input.GetAxisRaw("Vertical");
}
}
Objects on scene:
Inspector of Agent (Cobe):
I’m trying to solve this problem in every possible way: I changed the weight of the object and its speed, but still the Agent does not move. I suspect that the Heuristic method does not work or that the ActionBuffers variables do not work correctly.
Versions:
ml-agents: 1.0.0,
ml-agents-envs: 1.0.0,
Communicator API: 1.5.0,
PyTorch: 2.2.2+cu121
CUDA: 12.4
Thank you in advance!