I want to make an AI that uses machine learning and learns how to play the unity3D game I made.
The problem is I don’t know where to start. Should I try to make the AI inside of unity using monobehavior or should I separate the AI and wrote the machine learning part in Python/ C++/ C# and try to integrate it into my Unity project?
If I choose to write the AI outside of the unity3D would it be hard to implement?
Also, this is for learning purpose so I want to code the AI myself (I am open to libraries or frameworks).
all depends on what you want to use and do. Reinforcement learning? Neural Networks? Deep Learning?
Depending on this you should decide what exactly you want to programm.
Also you should perhaps show what your game is because this is basically the foundation to choose the right algorithms.
In general if you do it yourself just write it in unity.
Tbh I’d advise against this since it will most likely only bring you frustration. Unless you really know what you do you will probably not reach the performance of a state of the art algorithm like e.g. the RL ones provided in Unity ML-Agents.
Trying to use tensorflow and tuning this is already a task that can be difficutl enough.