3D Video Projection mapping

Unity Engine
1

Hello all,

I`m trying to project a scene on a real 3d shape and I want the projection of my objects to look real 3D for the persons who is going to watch it. To create this effect I want to use a Kinect to follows the persons watching it so that the projected image will change in the right way according to the persons view.

An example of a project like this is this : http://www.youtube.com/watch?v=sPA3W-dH7OA
I am now using KinectSample.Unity (http://wiki.etc.cmu.edu/unity3d/index.php/Microsoft_Kinect_-_Microsoft_SDK) like they say in the description on there website. But it isn`t working like I want it.

Am I right when I say I need to use RenderTexture and/or ProjectionMatrix?

As you can see I am just starting with Unity. Please help me.

2

How are you projecting the image exactly?

Cool video link by the way!

3

I`m going to project it with a 3lcd projector on to my shape that I build

1589305--95274--$2014-04-02 10.35.33.jpg
1589305--95274--$2014-04-02 10.35.33.jpg1691×3000 704 KB

I want to make objects in a 3D program and place it in my scene in Unity ( like a TL light) It has to look like the tl light is real. With head tracking and the Kinect I want to make it “more 3D”.

I also may have to look for a calibration for my shape.

4

1589341--95275--$projector.png
1589341--95275--$projector.png1152×648 6.65 KB

I’m going to attempt ASCII art. Here goes…

        |  object  |
        \          /
         \________/
          
    \
     \                /  user
      \              /
       \            /
        \ projector/

Ok so seems to me you have a projector and a user. You have a virtual scene (the safe in the other example) which you are pretending is actually occurring inside the object.

So what you would do is exactly reconstruct the scene inside Unity, getting distances etc all perfect. Call this scene ‘A’
You would create the object in a temporary material because you are going to texture map over the top of it later.

Next you would create another scene (call it ‘B’) except you would place the ‘fake’ scene (the safe stuff) inside the object in this version. You would render this scene from the point of view of ‘user’ to a texture.

Now you need to get this texture and texture map it back onto the outside of ‘object’ in scene A. You’d probably end up doing this in a shader, and for every pixel project the 3D point into the 2D camera space of ‘user’, and look up into the texture you rendered. Then when you look at the scene from the point of view of ‘user’ in scene ‘A’, scene ‘A’ looks exactly the same as scene ‘B’.

However render scene ‘A’ from the point of view of ‘projector’, and you have a video signal that you can output into your projector, and complete the illusion in real life.

…Hey, there’s a lot of words up there, how did I do, did this make any kind of sense?

edit just realised that I used the word ‘scene’ which has connotations in Unity. There might only be one Unity scene, but you would have two ‘scenes’ within it (I use the word ‘scene’ to mean ‘a collection of stuff’ laid out a certain way).

5

Thanks for trying :wink:
I think I get it…
Isn`t it something what you can do with RenderTexture ?

6

Yes: “You would render this scene from the point of view of ‘user’ to a texture.”

7

Hi there,
check out this article, it clarified a lot of how projection mapping works for me.

8

Hi, give this article a try.
http://en.wikibooks.org/wiki/Cg_Programming/Unity/Projection_for_Virtual_Reality
Seems like it’s supposed to do what you want.
You may just have to link it up with the position of the user’s head from the Kinect data.