Hi community. Have a beginner question on this tutorial (very happy with documentation btw). I set up everything closely to the guide, I can control ball agent using Heuristics, but when I run mlagents-learn following message is recieved:
[stats.py:118] RollerBall. Step: 10000. Time Elapsed: 105.998 s. No episode was completed since last summary. Training.
And yes, ball is standing still. Ball has decision requester script assigned. What else should I check?
Edit: btw, I successfully run demo scenes/environments from original repository; I use new folder for new environment, though.
PS
I’m sorry if it has been discussed already thousands time. I was not able to narrow down search to ML-Agents sub-forum so got lot’s of unrelated results and gave up.
Hi.
The fact that you see the message [stats.py:118] RollerBall. Step: 10000. Time Elapsed: 105.998 s. means that the RollerBall is communicating properly with Python. The issue is that No episode was completed since last summary. It is likely that there is nothing in the Game code that terminates the episode of the Agent. This means that the Agent would keep playing the game without ever ending the task (either by dying or by succeeding). I would advise setting up a maximum number of steps for your Agent through the “max step” property of the Agent or by manually ending the Agent’s episode (through the “EndEpisode()” method on the Agent). For example, you should end the episode if the Agent falls off the platform or when it successfully picks an item.
Hi Vincent! Thank you for the reply. I have EndEpisode() conditions as per the tutorial: this.transform.localPosition.y < 0 and distanceToTarget < 0.9f. And it works perfectly when I run it in Heuristics. However, I do not see ball moving at all when I start learning, it sits still at same place.
Have you tried “controlling” the ball during training? I wonder if the Heuristic is somehow overriding the Python actions.
It could also be due to the action of moving to be very small and un-perceptible, maybe increasing the decision period of the decision requester or increasing the speed change of the actions of the Agent would help diagnose it.
Can you send your code so I can try this on my end? I do not know what is wrong and I cannot reproduce this issue without further information.
Attaching Assets folder (code, materials, scene). Really appreciate your effort, Vincent!
6423141–718008–Assets.zip (10.8 KB)
Hi,
I looked at the assets you sent. First of all, the BehaviorType is set to Heuristic only, I think you changed it to Default when training but I just wanted to make sure. The issue is that in your Behavior Parameters, your VectorAction SpaceType is set to discrete. It should be continuous given the implementation you have of OnActionReceived.
If you have 2 discrete actions with branch size of 1, it means there are 2 actions but only one option for each (option 0) hence the action picked was always the only one available : (0, 0)
1 Like
It worked! You’re my savior, Vincent! Thank you a lot!
