I recently have been getting into MLAgents and I was curious if there is a way to punish the agent if it does nothing. For example I have a simple scene with a small room and a goal, if the agent hits the walls it gets -1 points and if it hits the goal it gets +1. If an agent doesn’t get lucky and touch the goal in the first few episodes it eventually determines “if I do nothing my score will be 0 rather than -1” and it will just sit there in the center of the room.
Is there a way to punish the agent for this behavior in order to avoid this scenario? I have combed through the documentation but can’t find anything that would allow for such a thing.
It’s common practice in a lot of the ml-agents example projects to utilize a small penalty (-1/maxSteps) to encourage the AI to do anything to accumulate more reward.
–edit: it’s also important to think about these RL Agents similar to training animals. You don’t necessarily want to penalize them with negative reward often. Think about it like a positive reward with points deducted off for not completing the task in the exact way you intended.
For example: a navigate towards a point task could have a reward that is +1 for facing directly at point during locomotion, then have points deducted from the whole +1 whenever the angle deviates from “facing directly at point”. This way the agent is still accumulating some positive reward, but there’s room for improvement.
Ah didn’t realize it was something so simple as placing it in the OnActionReceived() section. Thought you had to have it in a special area or some kinda check otherwise.
As for the 2nd part would that be done by using a vector3 distance check or just relying on random chance?