Smart Karts Training - mlagents_envs.exception.UnityTimeOutException: The Unity environment took too

Unity Engine
,
1

Hello there,

I am new to Unity, and I am trying to train an RL agent using the Karting Mod: Smart Karts Training Guide. However, I am using mlagents==0.24.1 in place of 0.12.0 and python 3.8.5. My system runs Ubuntu 20.04 LTS and CUDA 11.0.

I have converted the config file to the new standard and when I run:

mlagents-learn Assets/Karting/Prefabs/AI/kart_mg_trainer_new-config.yaml --run-id=WindingTrack_ppo_1

I get the following error message.

Version information: 
ml-agents: 0.24.1, 
ml-agents-envs: 0.24.1, 
Communicator API: 1.4.0, 
PyTorch: 1.7.1 
2021-03-11 18:46:02 INFO [learn.py:245] run_seed set to 7706 
2021-03-11 18:46:02 INFO [torch.py:58] default Torch device: cuda 
2021-03-11 18:46:02 INFO [environment.py:207] Listening on port 5004. Start training by pressing the Play button in the Unity Editor. 
2021-03-11 18:46:07 INFO [environment.py:111] Connected to Unity environment with package version 1.0.7 and communication version 1.0.0 
2021-03-11 18:46:17 INFO [environment.py:279] Connected new brain: 
ArcadeDriver?team=0 
2021-03-11 18:46:17 INFO [stats.py:182] Hyperparameters for behavior name ArcadeDriver: 
trainer_type: ppo 
hyperparameters: 
batch_size: 512 
buffer_size: 10240 
learning_rate: 0.0002 
beta: 0.005 
epsilon: 0.2 
lambd: 0.95 
num_epoch: 3 
learning_rate_schedule: linear 
network_settings: 
normalize: False 
hidden_units: 128 
num_layers: 2 
vis_encode_type: simple 
memory: None 
reward_signals: 
extrinsic: 
gamma: 0.99 
strength: 1.0 
init_path: None 
keep_checkpoints: 5 
checkpoint_interval: 500000 
max_steps: 50000000000 
time_horizon: 64 
summary_freq: 1000 
threaded: True 
self_play: None 
behavioral_cloning: None 
2021-03-11 18:47:21 INFO [subprocess_env_manager.py:220] UnityEnvironment worker 0: environment stopping. 
2021-03-11 18:47:21 INFO [model_serialization.py:130] Converting to results/WindingTrack_ppo_1/ArcadeDriver/ArcadeDriver-0.onnx 
2021-03-11 18:47:21 INFO [model_serialization.py:142] Exported results/WindingTrack_ppo_1/ArcadeDriver/ArcadeDriver-0.onnx 
2021-03-11 18:47:21 INFO [torch_model_saver.py:116] Copied results/WindingTrack_ppo_1/ArcadeDriver/ArcadeDriver-0.onnx to results/WindingTrack_ppo_1/ArcadeDriver.onnx. 
2021-03-11 18:47:21 INFO [trainer_controller.py:81] Saved Model 
Traceback (most recent call last): 
File "/Project/Unity_Kart_ML/venv/bin/mlagents-learn", line 8, in <module> 
sys.exit(main()) 
File "/Project/Unity_Kart_ML/venv/lib/python3.8/site-packages/mlagents/trainers/learn.py", line 250, in main 
run_cli(parse_command_line()) 
File "/Project/Unity_Kart_ML/venv/lib/python3.8/site-packages/mlagents/trainers/learn.py", line 246, in run_cli 
run_training(run_seed, options) 
File "/Project/Unity_Kart_ML/venv/lib/python3.8/site-packages/mlagents/trainers/learn.py", line 125, in run_training 
tc.start_learning(env_manager) 
File "/Project/Unity_Kart_ML/venv/lib/python3.8/site-packages/mlagents_envs/timers.py", line 305, in wrapped 
return func(*args, **kwargs) 
File "/Project/Unity_Kart_ML/venv/lib/python3.8/site-packages/mlagents/trainers/trainer_controller.py", line 175, in start_learning 
n_steps = self.advance(env_manager) 
File "/Project/Unity_Kart_ML/venv/lib/python3.8/site-packages/mlagents_envs/timers.py", line 305, in wrapped 
return func(*args, **kwargs) 
File "/Project/Unity_Kart_ML/venv/lib/python3.8/site-packages/mlagents/trainers/trainer_controller.py", line 233, in advance 
new_step_infos = env_manager.get_steps() 
File "/Project/Unity_Kart_ML/venv/lib/python3.8/site-packages/mlagents/trainers/env_manager.py", line 124, in get_steps 
new_step_infos = self._step() 
File "/Project/Unity_Kart_ML/venv/lib/python3.8/site-packages/mlagents/trainers/subprocess_env_manager.py", line 310, in _step 
raise env_exception 
mlagents_envs.exception.UnityTimeOutException: The Unity environment took too long to respond. Make sure that : 
The environment does not need user interaction to launch 
The Agents' Behavior Parameters > Behavior Type is set to "Default" 
The environment and the Python interface have compatible versions.

I noted the while mlagetns-learn is listening on port 5004 ./venv/lib/python3.8/site-packages/mlagents_envs/communicator.py is listening on port 5005. I change this last to 5004, but the issue persisted.

2

Hi. As per the message, it may be the case that the example you are running is not compatible with later versions of ML-Agents. I would recommend trying with the versions described in the documentation, and seeing if the error persists.

3

Thanks for your replay.

Unfortunately, the Unity version used in Karting Mod: Smart Karts Training Guide is not available for Linux. I try using Unity 2019.44.18f1 a conda virtual environment, to downgrade to Python 3.7, along with a venv and mlagents==0.12.0. Under this configuration, I get an error saying:

/Project/Unity_Kart_ML/venv_py3.7/lib/python3.7/site-packages/tensorflow_core/python/compat/v2_compat.py:65: disable_resourc
e_variables (from tensorflow.python.ops.variable_scope) is deprecated and will be removed in a future version. 
Instructions for updating: 
non-resource variables are not supported in the long term
4

Just re-run the command and press the play button in Unity. You got the error because you waited too long to press the play button in Unity after you ran the mlagent-learn command.

Good Luck!

5

Thanks for your help. However, pressing the play button was not the issue as the Unity GUI was running for a while and randomly placing the Kart in different locations within the track before the error was display. After the error message, the Unity GUI was still running, and I needed to stop the environment by pressing the play button.

This time, I try a machine with windows and another with Linux. I used mlagents==0.16.1 as when I try installing mlagents version 0.12.0 and 0.13.0 I got the error below:

ERROR: No matching distribution found for tensorflow<2.1,>=1.7 (from mlagents==0.13.0)

When I tried to train (mlagents==0.16.1) in the Windows machine, I got a similar error as the original. The error for the Linux machine is attached below.

Version information: 
ml-agents: 0.16.0, 
ml-agents-envs: 0.16.0, 
Communicator API: 1.0.0, 
TensorFlow: 2.4.1 
Traceback (most recent call last): 
File "/Project/Unity_Kart_ML/venv2/bin/mlagents-learn", line 8, in <module> 
sys.exit(main()) 
File "/Project/Unity_Kart_ML/venv2/lib/python3.8/site-packages/mlagents/trainers/learn.py", line 554, in main 
run_cli(parse_command_line()) 
File "/Project/Unity_Kart_ML/venv2/lib/python3.8/site-packages/mlagents/trainers/learn.py", line 550, in run_cli 
run_training(run_seed, options) 
File "/Project/Unity_Kart_ML/venv2/lib/python3.8/site-packages/mlagents/trainers/learn.py", line 347, in run_training 
handle_existing_directories( 
File "/Project/Unity_Kart_ML/venv2/lib/python3.8/site-packages/mlagents/trainers/trainer_util.py", line 228, in handle_existing_directories 
raise UnityTrainerException( 
mlagents.trainers.exception.UnityTrainerException: Previous data from this run ID was found. Either specify a new run ID, use --resume to resume this run, or use the --force parameter t
o overwrite existing data.```