How do I get visual observations in headless environment?

Unity Engine
, ,
1

I am using azure VM for ml-agents, it works fine when I doing normal observations.
But when I added visual observations in my agent, I got the Environment Connection Timeout.

I have found an official document suggested that [quote]
If you need graphics for visual observations, you will need to set up xvfb (or equivalent).
[/quote]
Also, I found this which explained how I should install the xvfb.
But after following the instruction above, I still got the connection timeout error.

Am I missing something? Or there are some detail instructions which I can follow?

2

Let me flag this for the team to have a look, and pass along any guidance they have.

Which version of ML Agents, C#, and Python are you using?

1 Like
3

I have the same problem (Environment Connection Timeout). I’m trying to use mlagents-learn to perform training on a server (i.e. headless) build of the Visual Pyramids task with visual observations. It’d be great to get some guidance on how to set up Xvfb!

1 Like
4

mlagents==0.17.0
mlagents-envs==0.17.0
python==3.7.7 or 3.6.7 (since that I use azure VM for data science, I can use both by conda environment)

And I am not quite sure how to check C# version in ubuntu, seems like it is along with the version of .NET.
Here’s my .NET version.

.NET Core SDK (reflecting any global.json):
Version: 3.1.202
Host (useful for support):
Version: 3.1.4

Not sure if this is the information you want.

5

Hi,
We do not officially support running ml-agents headless on Linux.
The issue you pointed (mlagents + xvfb · Issue #1786 · Unity-Technologies/ml-agents · GitHub) to contains instructions on how to install xvfb on Ubuntu 18.04. First, you need to make sure you have the correct version of Ubuntu as different versions of Ubuntu require different versions of xvfb. Make sure you did not get any error during the installation of xvfb.
You then need to run mlagents-learn with xvfb. To do so, you need to call “xvfb-run mlagents-learn” to run the mlagents command with xvfb.