if i can add my 2 cents: i feel there is still too much āold stuffā aroundā¦ see teh docs and tutorials online.
ML-agents has so much changed in latest years.
i would consolidate everything onto Release 2, or even more release 3, since i see there are some big changes in master branch about configs and all
There is one main problem Iām encountering when trying to teach ML-Agents to new people, most specifically students. Itās way too hard to find information on how to actually implement your own algorithms. The documentation on the Python API exists, yes, but in my experience itās not thorough enough to get something up and running. So I have to run through the process with them every single time.
As an example - imagine the task was to āuse a DQN with ML-Agentsā. This task is given to someone who has only used your PPO algorithm so far, since thatās the default. How is that person supposed to get the necessary information to implement this algorithm? In my experience itās scattered throughout the documentation, and even then itās not clear.
Whatās missing is a tutorial on how to set up a very simple ML algorithm on your own. Similar to the ābuild your own environmentā page. I had to go through a lot of trial and error and I donāt understand why this information is so hard to come by. You want people to use ML agents for research, right? Why not provide an example on how to actually use other algorithms not implemented by Unity?
Hey we would like to have one 2D example environment. All example environments are in 3D. If it is possible one very basic 2D environment can be really helpful for beginners as below:
Iād say that the OpenAI gym wrapper would be the best option for something like this. https://github.com/Unity-Technologies/ml-agents/tree/master/gym-unity
I donāt think ML-agents is intended to be a platform to develop novel RL algorithms, rather as a way of easily creating new environments and either using the prebuild tools like PPO, or use something that can work with gym. There are a few other projects as well, but gym is probably still most popular.
Thanks Jeffrey! The additional environments is what Iāve been waiting for - sounds great! Could you explain more what the āindustrial use casesā are or would be? The biggest problem Iām currently having is that all the example environments seem pretty trivial and simple and Iāve not been able to train any AI in a slightly more complex (but still relatively easy) environment. A big part of the problem is that MLAgents / Unity seems CPU bound so adding a good GPU doesnāt seem to speed up things at all. It takes me about 12 hours for 1 million steps, so 1 to 3 million steps has been a practical upper bound for me and that limits the AI to only very simple behaviors.
I think multi agent support for gym wrapper should be added, having to run multiple envs instead of multi instances within an env adds significant overhead
