An Agent which just sends visual observations (a CameraSensor component) to my custom trainer?

[unity_95E2298061EFD837527E]

I have a MainCamera object in my Unity simulation, which has a bird’s eye view overlooking my Unity setup. I want to have a stationary ML-Agents agent object with the CameraSensor component whose only job is to send captured frames when my simulation runs to a custom trainer plugin that I have implemented in my ml-agents. How do I set up such an agent object in my Unity? Do I simply add a Camera Component to the Main Camera and attach the required script to it? Kindly help me out with this.

[yuvalstn-ception]

@unity_95E2298061EFD837527E

I have a MainCamera object in my Unity simulation, which has a bird’s eye view overlooking my Unity setup. I want to have a stationary ML-Agents agent object with the CameraSensor component whose only job is to send captured frames when my simulation runs to a custom trainer plugin that I have implemented in my ml-agents. How do I set up such an agent object in my Unity? Do I simply add a Camera Component to the Main Camera and attach the required script to it? Kindly help me out with this.

I faced similar issues, if I’m not mistaken the cameraSensor component must be attached to the agent your using, and in the the cameraSensor component there is a field where you can insert camera to be used.
I would love to hear about how you implemented your own custom trainer I’m having a hard time adding my own implementations of classic RL algorithms such as DDPG, and TD3.

[unity_95E2298061EFD837527E]

Hey there! Thanks a lot for the information! In my case, the custom trainer is not an RL algorithm but something completely unrelated (PredNet architecture, if you’ve heard of it). So, I’m not sure if my experience will be of any help. Additionally, still working on making it work without any errors. This is the tutorial I followed though: Custom Trainer Plugin - Unity ML-Agents Toolkit. I hope this helps!