Good day, as everyone is aware, once you start training your models your computer becomes sufficiently utilized, therefore, is it possible to set up a centralized server so workstations can take advantage of them?
If so, is there any documentation on this?
Thank you
Something like: ml-agents/docs/Training-on-Amazon-Web-Service.md at main · Unity-Technologies/ml-agents · GitHub
Yes, pretty much! thank you kindly.
It seems that the “exe” and TensorFlow need to be on the same device.
If I wanted to run the EXE locally but have a service backend take the majority of the resource utilization, is that possible or even practical?
Sorry, I don’t know the answer to that. Conceptually I think there is no issue here, as long as the latency between client and server is fast enough. I don’t see any obvious way of doing this though within Unity. Sounds like a good feature request.
Hey thanks a lot for your input… I couldn’t find anything obvious myself. Maybe a code change somewhere im sure.
Cheers!
We don’t currently have any solutions for training across multiple machines. There’s a description of some upcoming work on our 1.0 blog post: https://blogs.unity3d.com/2020/05/12/announcing-ml-agents-unity-package-v1-0/ (see the “ML-Agents Cloud” section).
Thanks for the reply.
What about running the ML learn EXE continually even after the Unity engine stops…
I suppose you can always use something like ‘netcat’ to forward traffic from one machine to the other… I assume the connection is tcp based…
I suppose you can always use something like ‘netcat’ to forward traffic from one machine to the other… I assume the connection is tcp based…