Python API question for Thesis

Unity Engine
,
1

Hello everyone, i’m working on my thesis and i’m talking about the ML-Agents architecture and how the different components talk to each other. I’m struggling on finding an answer on how the Python API tells the RL algorithms to get an action. According to the docs,

DecisionSteps — contains the data from Agents belonging to the same “Behavior” in the simulation, such as observations and rewards. Only Agents that requested a decision since the last call to env.step() are in the DecisionSteps object.

So this class has only references of the agents that need a decision. My question is: what if i just need and action for the agent/s and not a decision? Which class make the request to the algorithms?

I 99% sure, if i’m not wrong, that the request have to pass via Python API because to get an action you must call the policy and pass observation and reward relative to the current step, indeed exist a method called SetActions which returns the action the algorithm carried out. But i’m not finding any specification on how the request is made.

Thanks for your time.

2

Best option would be for you to install ML Agents examples and actually run one of the examples. Even the 3D ball example will answer your questions.

Action = Decision = same thing
Except where (in the unity side) a previous action is just repeated.

Just roughly the execution flow is (roughly) as follows:

Unity Side:
Agent.RequestDecision();
triggers a request to the agent base code which in turn queues the request in the Academy
When the Academy steps (by default in fixed timestep) these requests and observations will be passed to the Python side via Message Pack objects serialized over tcp sockets.
The Academy step is a blocking call which will block until the Python side evaluates the request and returns the response containing the actions.

3

Ok, much clear now. But last question, if action=decision, why on the decision requester component you have the option “Take Action Between decisions”? If they are the same, what’s the meaning of the setting? Could be the same last action received carried out until the next decision ?