How can I save camera render textures at 1,000 times (Hertz) a second?

Unity Engine
1

Hello everybody,

I’ve ran into a major wall in my project unfortunately. I want to save the image (render texture) from one of my cameras (not main camera) into my local hard drive at 1,000 times a second while my application is running. Unfortunately, I have not found anything in the Unity documentation or forums that explains if this is even possible, let alone how. I need something that is much much faster than the Update method. When I use a Coroutine or do something like this InvokeRepeating(“SaveCameraFrames”, 0.001f, 0.001f); my frame rate drops extremely low and the application is unplayable.

Does anybody have any insight as to how I can save 1,000 images per second into my local disk? Any tips would be greatly appreciated! Thanks in advance!

2

You don’t.

1000FPS maybe possible if your entire scene is empty and you’re making an advanced caching system to not to wait for the disk writing.

What is the use case? Why do you need 1000 images per second written on a hard drive during your game play?

2 Likes
3

First of all, notice that if you try to save more than once per frame, you’re just going to be saving several identical pictures, because the image won’t have changed. We often pretend that things in the game are happening smoothly and continuously, but that’s an illusion; time is actually passing in discrete chunks, and there is nothing “between” the chunks for you to look at. Your game only changes when there’s a new frame.

If you want something to happen “more often” than Update, you should almost always still do it inside Update, just multiple times per invocation.

Secondly, saving 1000 pictures per second to your hard drive might be pushing the performance limits of your hardware, let alone Unity. Hard drives tend to be slow compared to other parts of a computer, and pictures contain a lot of data. I think you’ll find that troublesome no matter what game engine you are using.

What’s the purpose of this application you’re writing?

If the whole point of your program is just to create those images, I think you’ll probably want to slow your game down. For example, if you take 100 real-world seconds to create 1 “second” worth of images at a modest 10 frames per second, your performance problems will probably all go away, and you’ll still get the same images when you’re done. Unity has a time scale you can alter to make things go in “slow motion”.

3 Likes
4

You will never get Unity itself to save an image 1000 times per second.

To even consider getting to that speed you’d need to be directly copying the video buffer (which Unity doesn’t give you any easy access to) and saving that to disk with as little to no processing done on it to save CPU cycles (so basically raw data).

And even then… as Antistone pointed out, you’re hitting hardware limits.

Lets consider you do a 1080p image, 1000 times per second, as raw 24-bit colour.

That’s 6,220,800,000 bytes per second.

Or about 6 gigs per second.

pci-e 3.0 gets just shy of 1GB/s bandwidth (985MB/s, and that’s theoretical limit… the idea that in the real world your system runs perfectly is slim).

Now consider that most nvme SSD drives run on a 4x pci-e 3.0 bus, that’s a max of ~4GB/s. Not enough bandwidth to even write to a single nvme. You’d either need a higher end pci-e 4.0 4x nvme, which would just barely have the bandwidth at 8GB/s. Or you could run 2 nvme drives on independent 4x lanes in a striped RAID array and ‘possibly’ get that performance. And that’s supposing you have a SSD whose chips actually allow saturating the entire bus.

Which honestly… they don’t.

But heck, this also implies having to pull that information out of VRAM… which on a dedicated graphics card is going to be throttled by that 16x pci-e bus. Which means you’re sharing that 15760MB/s of bandwidth between reading those frames, and forcing data into the graphics card for it to even render!

And this all hinges on if you were able to actually push the RAW data from VRAM directly to disk… otherwise you’d be passing it through RAM, which means you’d have the bottleneck of whatever your RAM is. Which honestly… there’s no consumer RAM out there that has that kind of bandwidth.

…

Now of course you could perform some processing on that video buffer, directly on the video card, to then compress it. Then pipe that lower sized image off the card.

But now you’re stealing cycles away from an already ludicrous frame rate that the graphics card is trying to push out.

…

Which also gets to a fundamental problem… can the graphics card even render that many frames per second. You’re going to need a very nice high-end graphics card that can handle pushing all of that bandwidth (let alone the compute power to render with). Considering it needs to cover the data being pushed to it from the CPU/RAM, the processing of video data just to render the game, the transcoding the images into another part of VRAM, and the reading of that data out of VRAM to disk. And of course have the available VRAM to do all of this!

…

I mean consider that slow-mo cameras out there like the Phantom Flex 4K record 1080p images at about 2000fps, or 4k images at about 1000fps.
https://www.phantomhighspeed.com/products/cameras/4kmedia/flex4k

And it does this through very expensive specialized hardware to pump that amount of data (raw data at that). Honestly I don’t know HOW they push that data onto their proprietary CineMag IV storage devices (since it’s proprietary after all).

And this is on hardware doing a single job… recording. Not rendering AND recording.

…

TLDR;

No, the potato computers consumers all use are no where near the capability of doing this.

2 Likes
5

As others have stated, you aren’t going to get 1,000 fps in Unity. You could get your framerate as high as possible and store the position of all moving objects each frame thus making a replay system. Then replay the movement frame by frame and create your images then. Still though, for what you want, I would say you would need some custom hardware and not using an engine so you can strip out as much as possible to reduce CPU usage/time

6

Perhaps we are all going about this wrong. If you have a very performant app, you can sustain 100 Hz (100 fps). What is your use case that requires ten times as many samples from that Scene? Especially, as it was pointed out, that if you sample a Frame ten times, those ten frames will all Show the same. What are you trying to accomplish? If it’s entirely simulated, you might get much better results if you Change the time scale - because then you can get to (simulated) 1000 fps, with all Frames correctly rendered.

1 Like
7

Wow, thanks for all the help everybody! I appreciate all the in-depth details many of you went into in order to explain the limitations of the Unity Engine as well as hardware limitations.

The use case is building a simulation of a professional motion capture system. It is being used for a simulation, not a game, so the main purpose of the app is to output exactly as many images + data points per second as the real MoCap system, which is currently at 1,000 Hertz. I’m working on a considerably powerful machine, so hardware limitations are not a huge issue. What I decided to try out is Time.timeScale to slow down the application in order to capture 1,000 images per second, as a few of you recommended. I got it up to 300 Hertz now, but I still need better performance until it is satisfactory. If anybody thinks of any other ideas, I’d be more than happy to consider them! Thanks again.

8

Just thought I would mention that the bandwidth mentioned here is theoretical maximum. There are no PCIe 4.0 drives on the market actually capable of that yet. GIGABYTE’s AORUS, for example, caps out at 5GB/sec read and 4.4GB/sec write.

1 Like
9

Agreed, I did mention that the numbers were theoretical in my post when I quoted the 1x 3.0 speeds.