For context, I did some AI systems research back in college, about 10-ish years ago. Mostly with classifiers and recommenders; Deep Belief Networks in particular. I had to get out of the field for mental health. I finally got back into it with ML-Agents a couple weeks ago. I had to update myself on a lot of things, and I’m no expert with it yet, but I have noticed a few things which could be improved.
The biggest improvement would be to change how the current documentation is set up. At the moment, the docs.unity3d page is exceedingly bare. Sure, it redirects to the github docs, but the landing page is more of a quick and dirty overview, with a sparse quick-link bar on the right. Instead, you have to look for the link to the more comprehensive documentation page, but even that is incomplete, and serves more as a primer for what ML-Agents and RL in general is about. You really have to dig through it to find hyperlinks to more in-depth documentation; links which aren’t even in the Table of Contents panel. A good example of this is the page for Training Parameters used in the .YAML config file.
What users need is a single place to go to for their documentation needs, with a comprehensive and descriptive table of contents. Likely something which supports nesting, to make things easy to locate. As it is, it’s an unpleasant treasure hunt to get to the information one requires.
Another thing I’ve noticed with the documentation is that it’s first and foremost geared towards those with an AI background. Mainly professionals, researchers, and students studying AI. There’s been a great effort made to make it more digestible by advanced hobbyists, and it’s quite commendable. My above recommended change to the documentation layout would help, but I think a good addition would be a glossary page, linked to when more advanced terminology is used. Little things like that would help make the learning process streamlined. Otherwise, they’re expected to either keep multiple tabs open, or navigate around to refresh their memory.
When it comes to the Examples page, you make sure to note key values, such as observation size, reward shaping, and actions. What would really help would be to include the .YAML parameters used, and importantly, why those values were chosen. I’ve seen many people dialing things in by trial and error, using the value ranges listed in the Training Config File documentation. This leads to a lot of wasted time and effort, and much frustration.
Similarly, expanding the TensorBoard page to include how to use the various graphs to troubleshoot and dial in those .YAML parameters would be a godsend.
Another thing that would be nice is a page which gets into multi-environment training. Differences between, and the pros and cons, training multiple environments in-scene, vs multiple environments via the –num-envs CLI argument. Example use cases would be welcome, to drive home when and why a given method is best.
Building on that, a similar comparison for running pytorch on CPU vs GPU would be fantastic (with subsequent analysis for multi-env training on said device). This would be a time where nitty gritty details are welcome, weighing the pros and cons of hardware, as well as the Behavior Parameters of the Agent, if the Agent is utilizing threading (via .Net threading, or Jobs), etc.
For instance: Let’s say you have two machines. One has an Intel i-5 CPU and an RTX 5090 GPU, whereas the other has a Ryzen 32-core CPU, but an RTX 3070 GPU. Naturally, the GPU and the System Memory is an important consideration here, but that’s the general gist of what I’m suggesting.
This also leads into another documentation suggestion; including pytorch+cuda as an installation option, rather than just the cpu version of pytorch.
I’m sure there are other things I can suggest, but those are the main sticking points off the top of my head.
