The effect during training is very good, but the effect after importing the model into use is very poor, what is the reason?
ml-agents: 0.24.0,
ml-agents-envs: 0.24.0,
Communicator API: 1.4.0,
PyTorch: 1.7.1+cpu
One reason I can see is that there are issues with time scale. One way to see this is to resume the experiment but pass the command line argument --time-scale=1. This will resume the training but with a time scale of 1.
If the agent behaves poorly, it means that something in your code is preventing the simulation from running the same way at low and high time scales. Solutions include, reducing “race conditions” between the fixed update and the update loop. Other sources of issues can be the use of coroutines or use of Time.realtimeSinceStartup. If nothing works, you might have to do the entire training with a lower time scale.
If the Agent behaves fine, then there might be something in your code that makes the training simulation different from the non-training simulation. I would look for places in which Academy.IsCommunicatorOn is used or #if UNITY_EDITOR