Curriculum Learning Logic

Unity Engine
,
1

Hello,

I’m trying to use curriculum learning. Unity has wall jump example and explained in this doc . As I know curriculum learning is about teaching environment to your agent step by step. Curriculum part of yaml config file has configs for every step and value field in it. Wall Jump agents chooses random environment on every episode and uses different NNmodel for each one. I think environment should be selected or created using curriculum value field and there should be one NNmodel. Am I wrong ? Is it possible to get that value ? How can I do that ?

Thank you all for your responses.

2

Hi. You are right that by default the WallJump does not require curriculum and does not use it either. We do provide a curriculum yaml, as you point out. If you use that, the GetWithDefault calls will use the value sent from the trainer: ml-agents/Project/Assets/ML-Agents/Examples/WallJump/Scripts/WallJumpAgent.cs at beef5871ce85a09e463b5809cb7f43f145a96493 · Unity-Technologies/ml-agents · GitHub.

3

hello awjuliani thank you for your answer.As I understand GetWithDefault(“small_wall_height”) takes value field of that curriculum lesson… What i wonder is which specific lesson am I in current episode ? So we need a field like name to identify current curriculum step.

4

given a simple config example:

Lesson_number:
    curriculum:
      - name: Lesson0 # The '-' is important as this is a list
        completion_criteria:
          measure: reward
          behavior: My_B
          signal_smoothing: true
          min_lesson_length: 100
          threshold: 2500
        value: 0
      - name: Lesson1 # This is the start of the second lesson
        completion_criteria:
          measure: reward
          behavior: My_B
          signal_smoothing: true
          min_lesson_length: 100
          threshold: 2500
        value: 1

you can get the lesson num with:

EnvironmentParameters m_ResetParams;

float lessonNum = m_ResetParams.GetWithDefault("Lesson_number", 0);

that way, after training, you can also manually set the lesson you want to test.

5

hello, sorry for joining the question party late.
what does

  • measure: progress
  • threshold: 0.1

or

  • measure: reward
  • threshold: 2500

mean ? does it mean that after 10% of max_steps it will go to the next lesson? and that after getting a reward of 2500 or higher on the last 100 lessons it will then go to the next lesson?

also on the WallJumpAgent.cs there are three variables that are then assigned on ModelOverride, how do these work? are these just different onnx that will be saved with different names or do we have to provide something else / a network trained somewhere else (?).

public NNModel noWallBrain;
public NNModel smallWallBrain;
public NNModel bigWallBrain;
string m_NoWallBehaviorName = “SmallWallJump”;
string m_SmallWallBehaviorName = “SmallWallJump”;
string m_BigWallBehaviorName = “BigWallJump”;