I’m using ML-Agents v0.10.1. I don’t know if it has been change on newer version or not.
But, I’m counting on the success rate of the car parking model so I modified my code in the inference mode to count the time that the car is parking successfully by increment the counter by 1 before calling Done(), but the increment is 3-4 times before the AgentReset() is called. Which make my success rate 200 - 400% (lol). Now, I use boolean flag to solve this and it woking.
But, I’m curious that is this how it should behaved?
You’re correct, the agent doesn’t actually reset until the next decision happens after the Done(). But this behavior will be changed to immediate reset in this month’s release (and is already the case on master), so stay tuned!
Okey, I understand. BTW, why do that in the first place? It make no sense at all to me. And is it effect the training process in some way?
It was done this way to keep communication of the Done() flag in sync with actions and rewards - this is the way most reinforcement learning APIs are set up, to make training easier. Training shouldn’t be affected with the new code.