Publish game camera stream via ROS

Hello everyone,
I was wondering, how to stream the camera during game via ROS to a ROS subscriber via the robotics package?
Do you know where is it possible to find a script able to achieve that?

Thanks!

Marco

Hi marco, that would probably look something like this -

void SendImage(Camera sensorCamera, string topicName)
{
  var oldRT = RenderTexture.active;
  RenderTexture.active = sensorCamera.targetTexture;
  sensorCamera.Render();

  // Copy the pixels from the GPU into a texture so we can work with them
  // For more efficiency you should reuse this texture, instead of creating a new one every time
  Texture2D camText = new Texture2D(sensorCamera.targetTexture.width, sensorCamera.targetTexture.height);
  camText.ReadPixels(new Rect(0, 0, sensorCamera.targetTexture.width, sensorCamera.targetTexture.height), 0, 0);
  camText.Apply();
  RenderTexture.active = oldRT;

  // Encode the texture as a PNG, and send to ROS
  byte[] imageBytes = camText.EncodeToPNG();
  var message = new MCompressedImage(new MHeader(), "png", imageBytes);
  RosConnection.instance.Send(topicName, message);
}

Hi Laurie,
thank you a lot for the snippet. Do you know if this work with any rendering pipeline? I am constrained to URP/HDRP due to perception package compatibility.

Yes, should work in any pipeline. Here's the equivalent code from our Object Pose Estimation demo, which uses the perception package -
https://github.com/Unity-Technologies/Robotics-Object-Pose-Estimation/blob/7558d74b21f6e22544138c5b5f8525f24ff60c98/PoseEstimationDemoProject/Assets/TutorialAssets/Scripts/TrajectoryPlanner.cs#L198