Heuristic() called in FixedUpdate() means user input events get dropped

Unity Engine
, ,
1

I just came across Controlling Agents Using Heuristic Is Not So Responsive? · Issue #3245 · Unity-Technologies/ml-agents · GitHub. Because my Decision Period is 1, that should mean that Heuristic() is called with every FixedUpdate(). But that’s not good enough for fast input event checking, as I’ve read it’s common practice in a normal game to check for input events in Update(), because Update() gets called for every frame.

What are my options to make Heuristic() and OnActionReceived() as responsive to input as in a normal game? On my laptop, it appears that Heuristic() and OnActionReceived() get called about half as frequently as Update() is.

I could cache events that get collected in Update(), but that seems hacky. The other option that occurs to me is to call OnActionReceived() from Update() and bypass Heuristic() entirely. Slightly less hacky. Both options seem like they’re hacks to get around the UMLA architecture. Am I missing something, and there’s a right way to do this?

Thank you in advance.

1 Like
2

I would recommend going with caching the inputs. Another option would be to remove the decision requester and manually call Agent.RequestDecision() every update on the Agent. Be careful to change it back during training or to train with a low timescale.
I would advise against calling OnActionReceived manually as this can mess up the ML-Agents loop (it was not intended to be used like that)

3

Hi Vincent,
When I manually call RequestDecision (on heuristic mode and no decisionrequester component) why does Heuristic method gets called 3 times if I only requested decision once?

4

Generally speaking, making changes to game state in Update is bad. Including “processing” input from users. Note that what Vincent mentions about caching input in Update calls and processing it in fixed update is fine.

Update() depends on the user’s system and happens independently of physics updates. For networked and physics based games all updates to game state should happen in fixedupdate to get a consistent gameplay.

Fun Fact: In many old DOS games from 80s and early 90s, one had to set update interval depending on CPU speed and games would “feel” different when played on different systems!