Control over Hololens camera exposure

Target Platforms XR
1

We would like to combine our markers tracking system (computer vision) with Hololens. We did some preliminary tests for image and video quality using files from Device Portal. These are compressed therefore quality is affected but our main issue is camera exposure. Frames are (very) blurred when moving head such that tracking is very poor or plainly impossible. Is there a way to control Hololens camera properties such as exposure? I don’t see any such control in Device Portal. Maybe through coding?

2

Hello,

The device portal is a good way to grab video’s however it does impact frame rate while taking a video. I would suggest exploring the Video and Photo Capture API’s that you can use for scripting in your unity project. They might provide the answer you are looking for.

Link to Documentation:

Let us know if you have questions.

Thank you,
Wesley
QA Team

3

Hi @rcyr_cimmi ,

We do not currently provide an API that will allow you to adjust the camera’s exposure. We do however provide the following API “GetUnsafePointerToVideoDeviceController”

This method will return a pointer to the native IVideoDeviceController object. If you write a custom plugin, you can use this pointer to adjust the exposure yourself. You can find more information about the IVideoDeviceController object here.

4

Hi BrandonFogerty,

could you please help us with idea how to get the object from the pointer? What we try is:

        VideoDeviceController vdc = Marshal.PtrToStructure<VideoDeviceController>(m_PhotoCaptureObj.GetUnsafePointerToVideoDeviceController());
        Debug.Log("Exposure" + vdc.Exposure);

But we get MissingMethodException because VideoDeviceController is not support a default constructor.

Any hint will be a help! :slight_smile:

5

I originally started this post it has been a while now. You can basically call anything on this page: VideoDeviceController class for UWP

using System;
using System.Runtime.InteropServices;
using UnityEngine;

#if NETFX_CORE
using Windows.Media.Devices;
#endif

public sealed class VideoDeviceControllerWrapper : IDisposable
{
    private static readonly string PLATFORM_NOT_SUPPORTED_ERROR_MESSAGE = string.Format("[{0}] API is restricted to Universal Windows Platform", typeof(VideoDeviceControllerWrapper));

#if NETFX_CORE
    private VideoDeviceController m_VideoDeviceController = null;
#endif

    public VideoDeviceControllerWrapper(IntPtr unknown)
    {
#if !NETFX_CORE
        throw new PlatformNotSupportedException(PLATFORM_NOT_SUPPORTED_ERROR_MESSAGE);
#else
        m_VideoDeviceController = (VideoDeviceController)Marshal.GetObjectForIUnknown(unknown);
#endif
    }

    ~VideoDeviceControllerWrapper()
    {
        Dispose();
    }

    public void SetExposure(int exposure)
    {
#if !NETFX_CORE
        throw new PlatformNotSupportedException(PLATFORM_NOT_SUPPORTED_ERROR_MESSAGE);
#else
        if (!m_VideoDeviceController.Exposure.TrySetAuto(false))
        {
            Debug.LogErrorFormat("[{0}] HoloLens locatable camera has failed to set auto exposure off", typeof(VideoDeviceControllerWrapper));
        }
        
        if (!m_VideoDeviceController.Exposure.TrySetValue((double)exposure))
        {
            Debug.LogErrorFormat("[{0}] HoloLens locatable camera has failed to set exposure to {1} as requested", typeof(VideoDeviceControllerWrapper), exposure);
        }
#endif
    }

    public void Dispose()
    {
#if NETFX_CORE
        if (m_VideoDeviceController != null)
        {
            Marshal.ReleaseComObject(m_VideoDeviceController);
            m_VideoDeviceController = null;
        }
#endif
    }
}

And you use VideoDeviceControllerWrapper like this:

IntPtr unknown = m_PhotoCapture.GetUnsafePointerToVideoDeviceController(); // pointer to IUnknown COM interface
using (VideoDeviceControllerWrapper wrapper = new VideoDeviceControllerWrapper(unknown))
{
    int exposure; // TODO set proper exposure value
    wrapper.SetExposure(exposure);
}
6

Hey, it has been a while since this answer was posted, but how do you get the IntPtr?
I get “NullReferenceException: Object reference not set to an instance of an object” in the console when you do
“IntPtr unknown = m_PhotoCapture.GetUnsafePointerToVideoDeviceController();”

This is my class, the VideoDeviceControllerWrapper.cs is exactly the same:

using System;
using System.Collections;
using System.Collections.Generic;
using UnityEngine;
using UnityEngine.XR.WSA.WebCam;

public class WindowsVideoDevice : MonoBehaviour
{
    PhotoCapture m_PhotoCapture;
    int exposure = 1;

    // Start is called before the first frame update
    void Start()
    {
        Resolution cameraResolution = PhotoCapture.SupportedResolutions.OrderByDescending((res) => res.width * res.height).First();

        // Create a PhotoCapture object
        PhotoCapture.CreateAsync(false, delegate (PhotoCapture captureObject) {
            m_PhotoCapture = captureObject;
            CameraParameters cameraParameters = new CameraParameters();
            cameraParameters.hologramOpacity = 0.0f;
            cameraParameters.cameraResolutionWidth = cameraResolution.width;
            cameraParameters.cameraResolutionHeight = cameraResolution.height;
            cameraParameters.pixelFormat = CapturePixelFormat.BGRA32;

            // Activate the camera
            m_PhotoCapture.StartPhotoModeAsync(cameraParameters, delegate (PhotoCapture.PhotoCaptureResult result) {
            });
        });
        TestExposure();
    }

    void TestExposure()
    {
        IntPtr unknown = m_PhotoCapture.GetUnsafePointerToVideoDeviceController(); // pointer to IUnknown COM interface
        using (VideoDeviceControllerWrapper wrapper = new VideoDeviceControllerWrapper(unknown))
        {
            wrapper.SetExposure(exposure);
        }
    }
}
7

In the PhotoCapture#StartPhotoModeAsync’s callback, is the result a success? Have you enabled the Webcam device capability?