Totally new to ML. Thinking about trying out ML Agent for the first time for a 2D fighting game (like Street Fighter, The King of Fighters etc.) where you control a character that can move, fly and do some skills to beat the other side.
The goal is to have an AI that could play against the player. Is ML Agent the way to go, or is it actually more suitable to just write heuristic codes? Also is there any example project/ use case of ML Agent in this type of games?
Avbsolutely possible, but you’ll probably end up with an AI thats not much fun to play against.
Creating a balanced AI will be difficult. You’d have to be careful how you handle your reward system (or stop training early before getting to an optimal strategy) as it will find the “weak spot” in your game and spam it, making it dominate and not much fun to play against!
Depending on how many actions your fighting AI can or needs to take, and the amount of GPU power you have, training may take much longer than actually making a Heuristic AI. ML Agents usually take a fair bit of training before they are any good at anything for even mildly complex situations.
But if you want to have an AI that is more unpredictable and tries interesting things, you can attempt to train an Fighting AI. RL agents often tend to find very creative solutions.
everyone is waiting for this game but never seems to happen … except for that sumo game from 10 years ago
I’m working on a street fighter style game using ML right now. It’s still early but I’m also concerned with the problems that spankment mentioned. We’ll see how it works.
I decided to try ML because my game will have huge variation in abilities availble for each character.
It depends on whether your game is physics based or using animations.
Animations are easier - the agent policy can make discrete decisions in order to choose from a given number of movements. However, a challenge with this is unnatural looking transitions, like foot sliding for example. Sebastian Starke has done a lot of great work on this over the last couple of years:
Physics on the other hand is hard. The problem here is primarily constructing a realistic looking physics rig. AFAIK, no current physics engine can realistically simulate muscles. At least not the commonly used ones, all I could find in this regard are a few research projects. In Unity and other game engines, you’re basically dealing with stick figures. This involves a lot of tweaking collider shapes, rigidbody masses and joint spring settings for the individual body parts.
I’ve been experimenting with self-play fighters on and off for a while now, but the results are a bit underwhelming so far. You can check out my repo GitHub - mbaske/ml-selfplay-fighter: Self-Play Boxing Match made with Unity Machine Learning Agents
There’s no readme page yet I’m afraid, since I’m not that happy with the results at this point. What I did was generating demonstration files from animation clips and then combined imitation learning with PPO/self-play.
Here’s the accompanying video:
My advise: start simple. Try smaller projects first and learn how to tweak the hyper parameters and how to feed the right inputs. E.g. make a physics ball that tries to follow a randomly moving target, etc. Otherwise you will be overwhelmed by the complexity and will just have a very frustrating time when your agents are not learning anything.