Post Your ML-Agents Project

Thread on community created ML-Agents projects. Please share any posts, images, videos or project links.

1 Like

Hi, here are a couple of my project videos. There are source files available for most of them, but I haven't really kept those up to date, some are using old versions of ML-Agents. Also, since I'm still finding my way around Unity, I would approach some things differently now than I did back when I created these projects...
You can find more info in the video descriptions and a few other Unity and ML-Agents related videos on my channel.


https://github.com/mbaske/ml-motorcycles

https://www.youtube.com/watch?v=KzP-8lUQdmU

https://github.com/mbaske/ml-dogfight


https://github.com/mbaske/ml-hover-bike-race

https://github.com/mbaske/robot-ants

18 Likes

https://www.youtube.com/watch?v=9t4YBXZe7hc

https://discussions.unity.com/t/797987
https://github.com/mbaske/ml-selfplay-fighter

EDIT
THANK YOU for featuring the self-play fighter on the Unity blog.
I'm embarrassed to admit I only saw it this week, should really check the blog more often...


https://github.com/mbaske/angry-ai

https://www.youtube.com/watch?v=_H11-7eXIko

https://github.com/mbaske/ml-table-football


https://github.com/mbaske/ml-simple-driver

https://github.com/mbaske/ml-drone-collection

17 Likes

https://www.youtube.com/watch?v=3-4_-FBDr8Q

https://www.youtube.com/watch?v=B_cGPis6eDk

19 Likes

wow @mbaske !

I've put together a project that makes the machine learning process into the game itself! I've set up a twitch stream where you can watch creatures learn to navigate their environment to reach an exit and bet game coins on which ones you think will make it the furthest. It's running at https://twitch.tv/orbward

2 Likes

@mbaske Your project videos are always incredible. Also, that boxing match is hilarious!

Thanks!

Not as impressive as the above projects :), but I wrote a tutorial on self driving car racing.

crazywastefulgiantschnauzer

(already made September 2019, I just updated it now to 1.0.0 ML-Agents).

6680107--765583--upload_2021-1-2_17-50-7.png

9 Likes

Hi everyone! I've been playing with ML agents again over the holidays and trying to see if I can build a small RTS game where units are trained agents.

Duel training with self-play:


Resource gathering:

I'm experimenting with giving player control over high-level policies (e.g. which resource to gather, whether to prioritize attack or survival), curriculum learning (I've set up a system of lessons and scenarios to learn complex behavior step by step), various perception sensors (e.g. basic "range" sensor with OverlapSphere, "smell map" sensor with decaying influence that could give rough pathfinding data).

My overall goal is to build a small prototype/proof of concept to understand how RL can be part of a designer's toolbox and learn best practices along the way.

I started looking into ML for Unity several years ago but had no time to fully flesh out something. I initially tried to set up my own interface to python, then the first version of ML-Agents came out but it was a bit rough around the edges. I'm very impressed with how far it has come and found it quite easy to understand and use. Thanks for making it!

6 Likes

I'm working on getting Unity ML to help train the AI for a tactics game (turn based strategy game). First I did TicTacToe. Code available on my github.

Next I implemented a GridWorld mini-game in the game.

.

I tried to do a more advanced mini-game (one vs. one duel) but it ran way too slowly for the agent to learn anything. I'm working on speeding up the code. I can't duplicate the game board and have many games run in parallel (like TicTacToe and the Unity examples) without a lot of changes to my code. I can get by without having to render anything so I'm trying to do some research to see how to go about creating a 'fast' mode to the game.

3 Likes

RoboLeague - a car soccer environment inspired by Rocket League for deep reinforcement learning experiments in an adversarial self-play setting. The project is open source - https://github.com/roboserg/RoboLeague

udylxv
pq71qt

4 Likes

Hi everyone! I am experimenting with ML-Agents on Unity's Boat Attack environment. I have reduced the environmental objects for a smooth training process.
To project is open source and to get the assets and build versions of this environment, check out the following links:
Github env build repo - https://github.com/Dhyeythumar/BoatAttack-with-ML-Agents-build-versions
Github env assets repo - https://github.com/Dhyeythumar/BoatAttack-with-ML-Agents

Check out the following video of the trained agent:


The following video contains the complete training process streamed from Google Colab to the twitch server:

https://www.youtube.com/watch?v=T0Oxb3aliwQ

4 Likes

Hi, you can now find updated "Angry AI" source files working with the latest (1.7.2-preview) ML-Agents release on Github.
The project uses imitation learning, multi-stage training, hierarchical/tiered agent control and grid sensor observations.

https://github.com/mbaske/angry-ai

https://www.youtube.com/watch?v=7YYwERuy-w4

4 Likes

WWWoooWW!! Great job guys! Here you are another one. An Air Hockey simulator with 12 brains trained using self-play. It is available in the play store.


https://play.google.com/store/apps/details?id=com.OnMyOwnStudio.AIAirHockey

3 Likes

I've been experimenting with agent flocking behaviors:


Demo Link

4 Likes

https://www.youtube.com/watch?v=um7PkvgZj18

Not the most advanced agents, but it works with raytracing.

Download Link

3 Likes

There are some really cool projects in this thread :smile:

@mbaske I'm a big fan of the exploration drone. It reminds me of the mapping drones in Prometheus, which I thought were the coolest thing when I saw it.

@aureliantactics A few of your Medium articles were really good references while I was learning to use the ML toolkit. Thanks for writing them!

Here are a few of the projects I've used ML-Agents for.

This is an agent that controls a physics-based MKV:


Here is an agent that tries to use the MKV to intercept incoming projectiles:

This one is a variation of Boids that tries to maintain a certain distance from a target point:

https://www.youtube.com/watch?v=o6D4PtzJhSI

8 Likes

A happy mile-stone in my un-named project; VR interaction with agents (even during live training!)

limpfilthyborer
aliveequatorialaddax
massivedecimalgrayling

Still massively unfinished, but I think this shows extreme promise.

1 Like

My first Unity project using ML-Agents.

The game is a agility game, where you have to try to get a ball into an arc by moving the board.
This can be very frustrating so I wanted to give it a try with Unity ML-Agents after following the excellent Hummingbirds tutorial by Immersive Limit LLC (see https://learn.unity.com/course/ml-agents-hummingbirds).

The agent moves the board, rotating it either to the left, right, forward or backward and gets a reward when a ball enters the arc.

1 Ball

After several tries I finally succeeded. ML-Agents is able to play the game, with 1 ball, in an average time of less then 6 seconds, amazing. See a video of a game at
https://www.youtube.com/watch?v=gKZBCa-mwug
.

I learned to keep to rewards simpel, I started with a difficult reward schema but the Agent failed to learn.
The key to succes was to use the 'Ray Perception Sensor 3D' component. That made a huge difference for the better.

The training run of 4 million episodes takes about 1 hour on my machine.

2 Balls

Training (4 million steps) took 147 minutes. It takes the agent an average of 73 seconds.
See a video of a game at

(first win is at 00:11)

3 Balls

Training (12 million steps) took 500 minutes. It takes the agent an average of 1054 seconds to win.
See a video of a game at

. The first 00:05:30 are trimmed. The first win is at 00:10 (actual time at 05:40).
Not bad, but it would be nice to improve this. Still I can't win the 3-ball game in less than this time, too frustrating. To improve the ML Agents performance in playing the game it may need a bit of curiosity.

The Unity project is at https://github.com/jpzpj/Unity-ML-Agility-Game

3 Likes