when creating an agent with ml agents for the first 9 million steps everything went well, but after that the agent became even worse than before. How can I go back to a certain step. My chart looks like this :
This is what checkpoints are for. Every N steps (configured in your training configuration file) ML-Agents will create a checkpoint model.
There should be another setting named keep_checkpoints which tells ML-Agents how many checkpoints from the past it should keep.
In other words: Depending on your settings you can start the training anew from an old checkpoint. If there is no such checkpoint then you will have to start over.