DOTS ECS Integration as Reinforcement Learning Environment

Hey everyone!

I’m currently working on my MSc Thesis and could really use some insights from the Unity community! :slightly_smiling_face:

My goal is to provide a seamless learning environment for training and deploying Deep/Reinforcement Learning agents in ECS-based Unity projects.

Right now, I’m considering two main approaches:

  1. Adapting the existing ML-Agents package (partially, at least) to make it ECS-compatible.
  2. Building a new library from scratch that simplifies using Unity as a learning environment, built on top of external trainers (PyTorch), and agent inference implementation, all within an ECS-based architecture.

My main concern is whether this idea actually has practical value. With tools like Sentis already enabling runtime neural networks deployment in Unity, would an ECS-native learning environment for RL agents be something useful for developers? Would this make sense as a research topic?

I’d love to hear any thoughts, feedback, or even pointers to similar work that I might have missed!

Thanks a lot in advance!