I was training my agents and suddenly during the training this error popped out, I do not find a reason why can this happen, there has to be a number in that place.
That is why my training was stable until this happen:
2020-06-24 16:59:39 INFO [stats.py:100] PruebaBAT7_CubeLearning: Step: 3900000. Time Elapsed: 10656.156 s Mean Reward: -6.788. Std of Reward: 9.317. Training.
2020-06-24 17:08:36 INFO [stats.py:100] PruebaBAT7_CubeLearning: Step: 4200000. Time Elapsed: 11193.373 s Mean Reward: -6.011. Std of Reward: 9.916. Training.
2020-06-24 17:19:44 INFO [stats.py:100] PruebaBAT7_CubeLearning: Step: 4500000. Time Elapsed: 11861.222 s Mean Reward: -290.725. Std of Reward: 695.414. Training