I’m wondering, if transfer learning in ML-Agents is possible now? Or how can we modify the scripts to make it possible? For example, in the official example of 2v2 soccer game. What if now I want to make a 3v3 soccer game, using the already trained model of 2v2 to make it train faster?
For now, my idea is to remove the first layer of trained model, and add a new layer of different number of inputs . And use the weights of trained model (except first layer) to initialize new weights for training. But I don’t know how to do it in ML-Agents. Any instructions? Thanks a lot!
We don’t currently have any support for modifying the layers or changing the observation or action sizes, but it’s something we’re doing research on.
However, once you’ve trained a 2v2 model, you can pass the --initialize-from argument to mlagents-learn to start training a new model using the weights of previous one as a starting point.
Thanks. Do we have to make sure that the observation and action sizes are same as before, when we use --initialize-from? Or it can adapt to it automatically?
Would planning for this and preloading input and output nodes ahead of time work as a workaround at all? It would probably affect performance and itteration times but would it work? So for E.g: Lets say im making an agent that needs say 4 observations for flight, 4 more for the first behviour i want and 5 observations for the second behvaviour i want. Can i start with the 4 observations but pass empty (or static -1) values for the other 9 until add those observations and actions when i make the environment more complex? Same with Actions
you might want to start a new training run but initialize it using an already-trained model. You may want to do this, for instance, if your environment changed and you want a new model, but the old behavior is still better than random. You can do this by specifying --initialize-from=, where is the old run ID.
For Example: