I downloaded Unity a few weeks ago and tried to do a holographic effect using the Kinect. It was surprisingly simple to do using the already available Kinect Wrapper Package and some shader trickery. What it does is morph the image based on the viewers position around the screen to make it appear as if the objects are real 3D objects. In real life it does still require stereoscopy to be convincing though. I do have stereoscopy (anaglyph or side-by-side) in my own simple 3D engine, but I don’t think I can do that in the free edition of Unity.
The video is a top down view of some bad level design projected onto a table using a simple LED projector.
I did ran into a (apparently known) problem getting the joystick to run on Windows, so I had to use an ugly workaround by letting the joystick perform keypresses.
Anyway, here’s the video. I hope you like it.
Are you just moving the camera around to match the head tracking position, or are you changing the camera’s projection matrix? I only ask because it looks a bit distorted at times. If you’re just moving the camera around, look into how they do off center projection matrices for CAVE systems. Otherwise it’s looking pretty cool.
Oh, and you can do out of the box stereoscopy with the free version of Unity and 3D shutter glasses. I’ve played around with it some and it works well. No changes need to be done on Unity’s side. If it works or not would probably depend on your projector.
Thanks for the feedback. Actually the camera stays put directly above the scene. I use the vertex shaders to modify the meshes. It has (more or less) the same effect as modifying the projection matrices. The distortion you see is mainly do to the delay, jitter and lack of precision from the skeleton detection on the Kinect. Nevertheless, I’ll look further into the projection matrix solution, it seems more elegant. Thanks!
