Hello all,
MLagents is amazing and the number 1. reason why haven’t switched to other Game Engines yet (besides easy compute shaders, HDRP, C# etc.).
But tbh I would like to be more flexible, using own DeepLearning implementations etc. We really need either a better documentation for the ML gym or some new system to simply use ML in Unity.
I don’t see it as something difficult to do but would greatly improve the use cases for Unity.
There are C# ML frameworks but I haven’t looked into how to use .NET libraries in Unity3D.
If I am uniformed pelase help me out and tell me if there is already some offical system or good Asset!
Cheers
Hello,
thanks for the answer!
unfortunately not it seems bc : “The Python Scripting package is a feature designed to work only in the Unity Editor. It isn’t available in runtime builds”
1 Like
One of reasons I have never touched python ML, is exactly that, it is designed to only work for in editor ML training.
If you want to release game and let player to edit behaviour and re-train, it is tough luck, as far I am concerned. Therefore, I wrote while ago own machine learning code, using DOTS for performance. This way I don’t need to be stuck to only editor.
Obviously python package has tensorflow and other libraries, which may be useful for other projects. But it is limiting for post production and testing, after game is released. Specially if your game design is data driven.
1 Like
Hey, thanks for the answer!
Writing all from scratch seems to be the standard way in Unity… currently I also need pressure calculations for some sword penetration for my VR fun project (still struggling) and Physix doesn’t support this of course natively.
Editor only, does it includes the playmode? Then you could do some fun stuff with it and just export the model for the build; but the documentation are one-liners, seems a hussle to make it work…
Re ML writing all from scratch, it is a nice exercise indeed. But I would like to use Pytorch for performance reasons (it’s much faster than anything we can implement with Unity (fast crazy optimized C++ code and GPU use)) and bc it has built in RL and ML algorithms, easy to make new ones.
PPO and SAC are fine, but there so much more fun stuff you could do with Vision DL and simple supervised learning.
I will look into the Python Editor stuff; especially the ML.NET C# package from Microsoft seems to be the right way to go
Was ist ML.NET, und wie funktioniert es? - ML.NET | Microsoft Learn
Can’t you use this Sentis overview | Sentis | 2.0.0 for adding your models?
2 Likes
Hello!
Thanks for mentioning this, haven’t heard of Sentis. But it’s only for inference.
But what I was referring to is to have Python and C# Unity synchronized in FixedUpdate to train Reinforcement stuff. Exactly like the MLagents gym. But finding many examples is hard to implement own stuff.
Cheers!
1 Like