DecisionStep vs TerminalStep

Unity Engine
,
1

Hi all,
I’m noticing that during runtime, some Agent ID information is being passed to both the DecisionStep and TerminalStep.
To be more specific, in Python I can query DecisionStep.agent_id and TerminalStep.agent_id, and find that some cases some Agent IDs are on both lists.

Which is the correct data point for the duplicated Agent ID?
Is this a possible bug?

See image below of an example of the 3DBall with 12 agent IDs.

5927234--633719--upload_2020-6-1_12-5-16.png
5927234--633719--upload_2020-6-1_12-5-16.png1206×354 22.1 KB

I notice that the DecisionStep has a 0 reward for Agents 1, 7 and 9 but a different observation vector is seen on both objects. I would assume that the observation on the TerminalStep would be the actual terminal step.

Is my assumption correct?

Thanks!

5927234--633707--upload_2020-6-1_11-55-36.png
5927234--633707--upload_2020-6-1_11-55-36.png852×208 9.72 KB

5927234--633716--upload_2020-6-1_12-3-32.png
5927234--633716--upload_2020-6-1_12-3-32.png918×366 19.1 KB

2

Hi,

If an AgentId is both in DecisionStep and TerminalStep, it means that the Agent reseted in Unity and immediately requested a decision.
In your example, Agents 1, 7 and 9 had their episode terminated, started a new episode and requested a new decision. All in the same call to env.step()

3

Hi vincent, thank you SO much for the clarification!!!