Execution Order of WriteDiscreteActionMask and CollectObservations

airoll · October 13, 2021, 8:30pm

What is the execution order of WriteDiscreteActionMask and CollectObservations? Does WriteDiscreteActionMask execute after CollectObservations? I’m wondering if some of the data I’m processing in CollectObservations can be used in WriteDiscreteActionMask.

miguelalonsojr · October 18, 2021, 8:18pm

Hi,

Have a look at

github.com

Unity-Technologies/ml-agents/blob/05c027551e296291d8110165b23b6ad417fe9a4e/com.unity.ml-agents/Runtime/Agent.cs#L1078


      
          else
          {
              m_Info.CopyActions(m_ActuatorManager.StoredActions);
          }
          
          UpdateSensors();
          using (TimerStack.Instance.Scoped("CollectObservations"))
          {
              using (m_CollectObservationsChecker.Start())
              {
                  CollectObservations(collectObservationsSensor);
              }
          }
          using (TimerStack.Instance.Scoped("WriteActionMask"))
          {
              m_ActuatorManager.WriteActionMask();
          }
          
          m_Info.discreteActionMasks = m_ActuatorManager.DiscreteActionMask?.GetMask();
          m_Info.reward = m_Reward;
          m_Info.groupReward = m_GroupReward;

CollectObservations is called before any calls related to action masking (ml-agents/com.unity.ml-agents/Runtime/Agent.cs at 05c027551e296291d8110165b23b6ad417fe9a4e · Unity-Technologies/ml-agents · GitHub).

If you’d like to use the collected observations for other data processing, you can grab the collected observations using GetObservations():

ml-agents/com.unity.ml-agents/Runtime/Agent.cs at 05c027551e296291d8110165b23b6ad417fe9a4e · Unity-Technologies/ml-agents · GitHub.

Hope this helps.
-M.

Topic		Replies	Views
CollectDiscreteActionMasks Problem Unity Engine ML-Agents , Question , com_unity_ml-agents	2	1215	June 5, 2020
Are there specific examples of using WriteMask() in discrete actions Unity Engine ML-Agents , Question , com_unity_ml-agents	2	739	February 9, 2021
The connection between CollectObservations and OnActionReceived Unity Engine ML-Agents , com_unity_ml-agents	1	797	May 6, 2022
Unity ML-Agents Limited Choice Unity Engine ML-Agents , com_unity_ml-agents	1	1076	May 20, 2020
ML-Agents v0.15 Release (Mar 19, 2020) - Discussion Thread Unity Engine ML-Agents , com_unity_ml-agents , Official	0	1348	March 19, 2020

Execution Order of WriteDiscreteActionMask and CollectObservations

Related topics