Academy and imitation training

Unity Engine
, ,
1

Hi All,

I am new to ML-agents and Unity. I have been searching for this topic for a while and would appreciate if anyone can help me with this.

I learned that I can train ML-Agents with imitation learning from a player object with human input. They are two players side by side. One is a human player and the other is machine player learning from the player’s behavior.

From a video I watched, the setup requires academy and imitation learning. But I cannot find sufficient reference document and samples about the topic. Also I even got confused after further search. During the search, it seems that “Academy” is a thing of past (not sure if this is correct)?

I would appreciate very much if someone help with the good document/reference material etc.

2

Hi
GAIL (Generative Adversarial Imitation Learning) / Behavior Cloning:

Recording Demonstrations:

In the examples which come with ML Agents the following have GAIl / BC versions as well

  • Crawler - Both GAIL and BC
  • Hallway - GAIL
  • PushBlock - Both GAIL and BC
  • Pyramids - Both GAIL and BC

Re: “it seems that “Academy” is a thing of past (not sure if this is correct)?”
Not sure if you mean something else by “Academy”. But no not at all. The academy is the central process coordination system on the Unity side.

3

Hi ChillX,

Thanks so much for your reply, really appreciate it.

I am able to do demo recording and do “offline behavioral coloning” etc. But I am not able to do “real time training” with human and machine player side by side in an Academy. This feature, I believe, is called “online behavioral coloning”? I would very appreciate if you can point me to correct direction or share some material about this, please?

4

As far as I am aware. And I might be wrong. There is no support for “online behavioral cloning” as you described it in Unity ML Agents.
In theory you could modify it to be able to do exactly what you want but that would be a significant amount of work.

Just my two cents on this. Given the functionality of the GAIL feature, I don’t really see the point for what you describe as “online behavioral cloning”. Meaning it would achieve a lot less than what GAIL already provides.

5

Thanks for your reply.