Access the RGB camera (or Locatable Camera) at 30fps on HoloLens

Unity Engine
1

Does anyone know if Unity is planning to wrap the RGB (or LocatableCamera) in C# code that is accessible at 30fps? I’ve tried using PhotoCapture.TakePhotoAsync and WebCamTexture, but both of them drop the framerate to below 30fps. The most I’ve gotten is 20fps with WebCamTexture with an 896x504 image. I know it’s possible to get the RGB image at 30fps, because I’ve successfully ran Windows-universal-samples/Samples/CameraGetPreviewFrame at main · microsoft/Windows-universal-samples · GitHub on the HoloLens. However, in that code example, they’re sending the image to a windows UI element called CaptureElement and I’m not sure if we can access the image through that at 30fps.

I know Unity has a VideoCapture class but that only saves the video to a file and doesn’t let me access the raw pixel information for each frame.

Is Unity planning to write a wrapper to access the RGB video frames at 30fps?

2

Hi @bnuernberger

Anytime Mixed Reality Capture is used on the HoloLens, the fps will drop to 30 fps as the price of doing business. However I was unaware that it dropped below 30 fps. Do you have a repro project that I could evaluate? Either way I will look into this further. Thanks for bringing this to my attention!

3

Hi @BrandonFogerty I’m not doing Mixed Reality Capture, but just regular RGB video capture. And I don’t want to simply save the video frames to disk (e.g., with the VideoCapture class) but I want to access the raw pixels (e.g., to use it in a texture). Currently, I don’t have a clean project to send you, but if I am able to get one, I’ll send it to you ASAP.

Do you know if the frame rate for non mixed reality capture (i.e., only RGB camera capture) should make the fps drop to 30? Or is this only for mixed reality capture?