Perception cam frame drop

Unity Engine
1

Hi,

I’m currently working on a multi-robot simulation in unity (RTX4090, Intel i7-13700F, 64 GB RAM).
The goal is a simulation containing at least 3 mobile robots equipped with 2 LiDARs (published via ROS2) and 1 camera each.
Since I want to use the camera images later for deep learning, I installed the perception package.
Problem is: frames are dropping dramatically.
When using 1 Robot with 2 lidars, the frame rate is around 60 fps.
When adding the perception cam with 2D bounding box label generation, the frames drop by around 5 fps.
When adding semantic segmentation drops to 30 fps and with adding depth images, it drops to 15 fps.
That wouldn’t be a problem for me since I want to record all of that data (LiDAR via ROS) and real-time is not needed.
But unfortunately the sim time in Unity seems to be somehow decoupled from the frame rate.
I tried to publish sim time to ROS but even after recording as a ROS bag, the data is not steadily published.
Is there a way to couple sim time and FPS?

Thanks in advance!

2

Hi floriegel,
have a look at the execution order. Here you find which call is made when: Unity - Manual: Order of execution for event functions
You have the greatest chance by deciding which process you would like to run in the update frame and which in the fixedUpdate frame. You can also have a look which package (ROS, perception, etc) is using what.

3

Ok. My first idea was to use Time.CaptureDeltaTime to slow down the overall simulation and to give my PC more time to execute the scripts. Unfortunately, this didn’t make any difference. I then tried to move everything into fixedUpdate and increase Time.FixedDeltaTime. But again, this made no difference.