How to transfer the ML-Agent model trained in simulation to a real robot?

Unity Engine
, ,
1

Hello,
I was trained a robot-car in unity using ml-agent with PPO algorithm.
Now I want to take the final policy and test it on a real robot.
For that, I trying to load the final network to python somehow and from there I will send commands to my robot online.
There is some guidance on how to do that? or some related recommendations?
Until now I tried to load the .pt file with PyTorch, but I think I must have the model definition to create the model instance and then use state_dict().
please help me with it, thanks

2

Same question.Looking forward to replies.

3

I haven’t done this myself, though, it sounds super fun. If you go to the post in the sticky section " post your ml-agents project " - a user has shared this video about training in unity and deploying inference in a physical robot.

If I were going into this project, the primary thing I would consider is how to accurately simulate the physicality of each component of the robot regarding control (stepper motor updates for example) and sensor input (shutter speeds of camera/upload latency, etc) within unity.

Unity has also released a github for robotics and perception which will likely help you in your experimentation.

Lastly, I frequently notice people asking duplicate questions on this forum - don’t hesitate to use the search functionality in Unity Forums and sort for the ml-agents specific posts. There’s lots of good information hidden away; and I vaguely recall some others asking this question a while ago.

Cheers and best of luck! Would love to see more real world deployment of RL!

1 Like
4

Thanks for your detailed response!
My clearest question is how to use the policy network generated in unity and load it in python, such that I could send an observation-like and get back the compitable action?

1 Like
5

There’s information about the python ↔ unity communication pipeline on the github/in the docs. I generally recommend everyone who is using ML-Agents to sit down over a couple sessions and read the entirety of the documentation. There’s a lot of cool tips and tricks nestled in there.

https://github.com/Unity-Technologies/ml-agents/blob/main/docs/Python-API.md

6

I extract the onnx file after ML agents training. Load the ONNX file with Python, the input is the original input in unity,and action_mask is all set to 0, but the output result is very strange. Can anyone answer it? :slight_smile: Thanks.