ML-Agents - what data would be most useful?

I’m making a driving AI, where it chooses its path based on goals that are spawning out in the world. I can’t figure out if it would be better to send the absolute world position of the objects into the AI, or the distance to the objects from the AI player?

The first is more information as it effectively contains the distance and direction, but maybe the AI doesn’t understand positions very well and would do better with things (like distance) getting bigger or smaller?

Assuming your agent can move and turn, distance alone would prooobably work best. It should figure out quickly exactly where in the world the goal is purely based on distance after moving around and turning a little bit. Providing absolute position will make position clearer, but may slow training down as you’re saddling your agent with more data input. (In theory - YMMV)

It can depend on how you’re rewarding your agent too. Like if it’s purely “shorter distance = good”, and there are obstacles in your world, your agent may be discouraged from going around an obstacle since it would increase the distance to goal. In practice this wouldn’t likely matter too much so long as your agent learns the BIG reward comes from ultimately reaching the goal.

(sorry for the necro, posting for posterity if it helps anyone else!)

Thanks nyonge, by chance, I’m just getting back to working on this so it is still helpful to hear someone’s opinion on it!

1 Like