How to Do Mixed Reality Capture With Script

Unity Engine
1

Hello, I am trying to add Mixed Reality Capture support in my application to create a video file of the user interacting with holograms but I can’t seem to find any examples of how to do this. I tried using this example script: Unity - Manual: XR architecture but it only records the webcam video feed and does not include the holograms in the video. I see that it is possible to do this via Cortana or from the Windows device portal but it would be preferable for me to control it from Unity.

Is doing this possible and if so are there any good example scripts or Unity documentation on how to accomplish this?

2

That linked manual page sets hologramOpacity to zero on the CameraParameters object. You probably want to set it to one.

Though the scripting reference for that field (https://docs.unity3d.com/ScriptReference/VR.WSA.WebCam.CameraParameters-hologramOpacity.html) says that you “must have also indicated that you wanted to capture holograms when you created your capture object”. I’m not sure what that part entails, but just in case setting hologramOpacity to one instead of zero doesn’t do the trick for you on its own, I’ve poked the dev that wrote that feature about it and will update this thread with his answer when he gets back to me.

3

Okay I found Mixed reality capture overview - Mixed Reality | Microsoft Learn which says that this should be possible but the info there is fairly sparse. Is there any existing Unity example using one of those APIs?

4

Thanks I will try that out and see if it works for me.

5

I changed hologramOpacity to 1 instead of 0 and it didn’t seem to change anything for me. I still only see the webcam feed in the video and no holograms.

6

The dev that wrote the feature sits in the same room as me, but is out at lunch. I’ll let you know what I find out from him when he gets back.

7

I actually just managed to figure it out on my own.

VideoCapture.CreateAsync(false, delegate (VideoCapture videoCapture)

also needed to be changed to

VideoCapture.CreateAsync(true, delegate (VideoCapture videoCapture)

and along with changing the opacity to 1 records the holograms as well. Thank you for the help unity_andrewc.

8

Ah, I missed that. Glad you found it. (As an aside, this is my issue with passing bools to public methods we expose, it makes the code far less intuitive when reading - even if it’s only two allowed values, I wish we’d expose those as enums.)

9

This is only tangentially related to my original question but do you know if there is a maximum value that MaxRecordingTime can be set to? I have seen some info online saying that video recording is limited to only a few minutes at a time on HoloLens and I wasn’t sure if that was also the case here.

10

Is there a way to sent this capture over a webserver, and create possible video conferencing ?