ML-Agents Release 1 (April 30, 2020) - Discussion Thread

Unity Engine
, ,
1

Hi everyone,

We are very excited to have released ML-Agents Release 1 on April 30, 2020. This is a huge release in so many ways, specifically that we are moving the C# and communicator to v1.0!

In the following days, we will be publishing a blog post on all the changes on ML-Agents Release 1 and what it means.

In the meantime, please refer to the release notes for ML-Agents Release 1 here:

As usual, please follow our guidelines on where to open issues, bugs, discussion. All feedback welcome and thank you again for your support of ML-Agents.

Jeff

2

Come on, work hard

3

Hi,
I am working with ML Agents for some time now. I like this project, it improved much since early versions. But I always end up modifying package’s C# source code, throught not as much as I did before.
For example, now I’m working on multi-goal reinforcement learning (goals are fixed actually, so there is just multiple rewards, one for each goal). I need to send multiple rewards to python side, so I’ve made a subclass of SideChannel. But there is a problem: I can’t accurately synchronize these side rewards with main one without modifying C# package source. To solve this I modified Agent.SendInfoToBrain to call another function which generates and sends auxiliary rewards to python code.
Before that, there was another problem, when I needed to disable ending of episode when agent’s GameObject is deactivated. Also haven’t found a way besides modifying C# source.
Maybe these are just my specific problems and workaround isn’t that bad, but I still decided to share.
Keep up good work.

4

Congratulations! Thank you for your hard work, each release has attacked major pain points, and it’s great to see the project reaching a stable point.

1 Like
5

Hey, I am having issues after the update to get Curriculum training to work…

from terminal:
mlagents-learn ~/ml-agents/config/ppo/trainer_config.yaml --run-id=some_id-curriculum

I tried following the migration page on github but it still does not work:

Added this to trianer_config.yaml, instead of having it in seperate file:
curriculum:
measure: reward
thresholds: [0.1, 0.3, 0.5, 0.5, 0.5]
min_lesson_length: 100
signal_smoothing: true
parameters:
add_obstacle: [0, 1, 2, 3, 4, 5]

Previously, my thresholds were at 0,9 for all of them and would reach the first after around 300k steps. Now it does not even seem to reach 0.1…

I use:

 stage = (int)Academy.Instance.EnvironmentParameters.GetWithDefault("add_obstacle", 0);

to retrieve the values, and later have a switch statements that initiate different obstacle GameOcjects.

It worked perfectly fine before the update, but now it does not work at all.

6

Can you open up a Github issue with the above?

7

Since removal of notebooks ( which in my opinion sucks for beginners) I have updated all of my syntax for Rainbow DQN and made a repo about it.

If any starter looking to just jump to it & use as reference here’s it. No PG /AC algorithm yet but I will add as soon as possible.

https://github.com/ugurkanates/MLAgents-Google-Collab

8

Installation instructions don’t seem to work
after flowing the directions from “Installation.md” in the Release1 branch Docs folder

  • created a separate conda env
  • cloned Release1 branch
  • “pip3 install mlagents”

the command “mlagents-learn --help” reports “not recognized as an internal or external command”

DId I miss something?

9

humm… worked second time. But, the config file has been changed. no PPO directory. looks like you have to specify “trainer_config.yaml” instead. Guess the user docs haven’t caught up to release

10

Hi, currently I am working on a Space Shooter game and I created an agent for it. But whenever I try to build it gives this error:
UnityEditor.BuildPlayerWindow+BuildMethodException: 4 errors
at UnityEditor.BuildPlayerWindow+DefaultBuildMethods.BuildPlayer (UnityEditor.BuildPlayerOptions options) [0x002bb] in <17b72532ee2c4da1b6f632d3f1705fe0>:0
at UnityEditor.BuildPlayerWindow.CallBuildMethods (System.Boolean askForBuildLocation, UnityEditor.BuildOptions defaultBuildOptions) [0x00080] in <17b72532ee2c4da1b6f632d3f1705fe0>:0

here is my code


```csharp
using System.Collections;
using System.Collections.Generic;
using UnityEngine;
using Unity.MLAgents;

public class SpaceshipAI : Agent
{

    public static SpaceshipAI Instance;
    SpaceshipMovement movement;

    public void OnEpisodeBeginFunction()
    {
        OnEpisodeBegin();
    }

    private void Awake()
    {
        Instance = this;
    }

    private void Start()
    {
        movement = GetComponent<SpaceshipMovement>();
    }


    public override void OnActionReceived(float[] act)
    {
        Vector2 temp = new Vector2(act[0], act[1]);
        movement.movement = temp;
       
    }

    public void AddRewardFunction(float reward)
    {
        AddReward(reward);
    }

    public void EndEpisodeFunction()
    {
        EndEpisode();
    }

}
11

I did not see the docs explaining the tensorflowsharp… does it mean it was not supported ?

12

Correct, we no longer support TF# but support Barracuda inference instead.

13

I see thanks!

14

Thank you for great product!
Can’t catch whats happening with online_bc/offline_bc trainers. There is no any mention in migration logs nor in docs. Are their not supported anymore? Are there any alternatives via gail or behavioral_cloning?