Best way to collect observations on player move

Unity Engine
, ,
1

I’ve trained model for my turn-based game. Right now I’m wondering what is the best way to connect it to my existing game flow.

In my case AI is giving tips on demand for the player. It means player plays on his own, but when he clicks “help” I’m calling RequestDecision() and providing player with suggested move.
Right now player moves are not registred by the Agent, that means CollectObservations is called only when player ask for help and decision is made by the Agent. I use Stacked Vectors so this has especially bad influence on decision making as model is not fed with observations during player moves.

I’m trying to find out what is the best way to fix it. I want to collect observations even when player is making the moves. I wonder if using Heuristic() is the only way to do it. I want to keep my code loosely coupled so it doesn’t seem to be the best choice to move handling user input to Agent’s code. Is there any better way to force agent to collect observations on player move?

2

It sounds like you would want to run the trained agent for all steps just like the agent is playing the game, except that the output action from the agent is not actually used.

Heuristic() is not the best way here since if you specify behavior type to Heuristic the agent model will not be used.

I’d suggest you to call RequestDecision() in every step the actual player make a move, and that will result in calling CollectObservations() every step. And you can just ignore the output action or do nothing in OnActionReceived, and have you user input handled elsewhere.

1 Like
3

Thanks, that popped into my mind, but I wasn’t sure it’s proper use :wink: