What is the best way to learn configuration setup

Unity Engine
,
1

I am trying to understand configuration yaml’s, but it’s a lot of moving pieces. Does anyone know a good resource with a deep breakdown of most important combinations?

I’ve looked at documentation for release 19 ml-agents/docs/Training-Configuration-File.md at release_19_docs · Unity-Technologies/ml-agents · GitHub

Looked at different sample files ml-agents/config at release_19_docs · Unity-Technologies/ml-agents · GitHub

But it’s still fairly confusing. When do i use trainer type ppo, when sac or poca?

Setting “buffer_size”, Typical range: PPO: 2048 - 409600; SAC: 50000 - 1000000. When do i use the upper range, and when do i use the lower range?

Can i train an agent, achieve some reward, increase complexity in the code and continue with the same .onnx file?

How do i spot a bad config quickly? What is a reasonable training time for a simple task on a regular computer? A million steps with no major drops?

8326317--1093683--upload_2022-7-31_23-59-23.png
8326317--1093683--upload_2022-7-31_23-59-23.png895×224 8.14 KB

8326317--1093683--upload_2022-7-31_23-59-23.png
8326317--1093683--upload_2022-7-31_23-59-23.png895×224 8.14 KB

2

Yes it is really a lot of Black Art. It comes down to having some awareness of the Reinforcement Learning algorithms. I would suggest Groking Deep Reinforcement Learning, by Morales. Its pretty heavy stuff, but it slowly brings you up to speed with the core Algorithms and the purpose of the RL parameters. I pretty much stick with PPO for my base, and then only use POCA and Group Rewards for collaborative scenarios.
Alternatively you need to study and review the different configuration files between the different example projects provided, and choose those whish are similar to your own scenarios.
Star off very simple, get the basics right, observe some Training, and grow from there. It can take weeks, and many many iterations to get the yaml configuration combination right. And can be Very frustrating at times

3

Totally agree. I hardly ever stray too far from the example configs, as they seem to work pretty well for lots of use cases.

There’s an app for that :wink: Well, at least it helps automating dumb parameter grid searches.
https://github.com/mbaske/ml-agents-hyperparams