How to get expected reward or action probabilities during inference mode?

Unity Engine
,
1

Hi! I’d like to get expected reward or action probabilities during inference mode.
Is there a place in the code (e.g. a barracuda script) where I can query this information from?
Would be very helpful and interesting to see.

2

Do you mean cumulative reward? Can you share your use case?

You may be able to hack something together in com.unity.ml-agents/Runtime/Inference/ApplierImpl.cs

3

I’d be interested in just visualising in e.g. the UI how the expected reward changes depending on where the agent is in the environment, and also see how certain it is of its actions (by looking at the probability distribution of potential actions that the agent samples from).

ApplierImpl.cs seems at first glance useful for this. I’ll give it a try. Thanks for the idea!

4

Bumping this because I’m also interested in being able to access that information and haven’t found a solution yet.

Also opened request on github: Remove multinomial sample from ONNX model output · Issue #5788 · Unity-Technologies/ml-agents · GitHub