Which optimizer ML-Agents uses in general?

Hello!

I read in an old same topic’s question that ML-Agents uses Adam as optimizer. I would like to discover more information about that:

  • Is adam used for all the supported main trainers? (ppo, sac and poca)
  • Can someone point me to the file in the repo where the optimizer is being declared and initialized?

The first question is related to a message that appears when you resume the training of a network using the same configuration file but with different trainer type: for example, if we change from ppo to poca, it outputs that it can’t find the adam optimizer status, while viceversa it says that can’t find the value optimer status, so adam seems not there in poca, or hidden/wrapped in something else.

The second question is for an eventual idea of changing the optimizer toward sgd with nesterov, that in some cases performs better than adam.

Thank you very much!

Have you already found out which optimizer is used for poca?

Yes, I went throught the source code on github and I found all the trainers with their relative optimizer. PPO SAC and POCA all use Adam as main optimizer. SAC has 3 internal optimizer (for its nature) all based on Adam.
If you would like to explore the code, here is the link ml-agents/ml-agents/mlagents/trainers at develop · Unity-Technologies/ml-agents · GitHub
:smile:

2 Likes

Thank you!