Interactive projections hardware

Hello,

I'm creating an interactive projection. And I'd like some pointers regarding the hardware and SDKs I might use with Unity.

Requirements:

  • The projection will be flat on a flat surface, so I'm not worried about projection mapping currently.
  • The content might be projected on a wall or on the floor.
  • Players will control the game via body movement OR by touching the display surface (the floor or wall).

What I know:
I'm assuming that I need two pieces of hardware:

  • A short throw projector (to avoid shadows on the display surface)

  • A depth sensor

Some projectors are interactive, so you can touch the display surface. But I assume that I only need a normal projector because I'm already using a depth sensor for body tracking. So I can also use it to detect surface touches.

For the depth sensors, the options I found are:

  • Azure Kinect (the older Kinect seems to be discontinued)

  • Intel Realsense stereo camera

The content I'm finding online is old. So I'm wondering if someone working on this currently (in the entertainment / installations space) can guide me towards hardware and software that integrates well with Unity.

For your depth-sensor, you could also look at using a smartphone, along with the XR systems (ARCore/ARKit).
Unity - Manual: Getting started with AR development in Unity (unity3d.com)
Particularly a (new) iOS-device (one with LiDAR) would work well for this.
AR Tank Exam using LiDAR & Unity ARFoundation - YouTube

1 Like

[quote]
But I assume that I only need a normal projector because I'm already using a depth sensor for body tracking. So I can also use it to detect surface touches.
[/quote]

Your mileage may vary. Using a depth sensor to try to capture surface touches will be a noisy signal with a likelihood of false positive and/or false negative depending on how you set up your system. If possible I would recommend a design that is more gesture-tracking based (playing to the strengths of a skeletal tracking system), rather than trying to use a skeletal tracking system to fulfill a surface-touch task. (Alternatively, consider using dedicated hardware to detect surface touch.)

Azure Kinect or Intel Realsense are both potentially viable body tracking sensors. For an installation like this in 2023 I would also consider an AI pose detection solution using a standard webcam, but I can't recommend any particular solution.

1 Like