How can I preprocess mlagents camera image or use different source for the camera?

Unity Engine
, ,
1

Hi, I would like to perform GAUGAN on segmentation image in unity.
I was hoping to take segmentation image from unity, then perform a GAN method in it.
I can get image segmentation using Bitbucket

How can I use use this image as input for mlagents model?

2

hi i asked many similar questions here with no results, but I believe u can explore ISensor. I made mine work by editing the source code on python’s receiving end

3

hey, thanks for the reply, I could somehow get the image from camera.
but I am struggling with camera interpolation.
I made a segmentation map from camera image, but since it is resized to 84x84, all of the segmentation got jumbled. Do you know how to prevent this?

4

Can I ask how you did that? I’m looking to use a live camera feed from my local network.

5

May I know how did you obtain the custom visual observation in python? Thank you! @TiranianHoward @MrOCW