displaying images from OpenCV plugin in Unity

Unity Engine
1

Hi There,

I’ve seen a few threads that state it is theoretically possible to take image data from OpenCV (IplImage) and apply to a Texture2D type within Unity.

The thread here http://answers.unity3d.com/questions/15574/import-video-from-camera suggests a method for doing this. Having looked into this, if this is to be done safely, it requires marshalling struct pointers from C# to C++ which is an avenue I would rather not explore unless absolutely necessary.

Has anyone managed to successfully pass images from an opencv dll plugin into Unity and if so would you be able to share some code? It would be good not to have to re-invent the wheel and it would also be good not to have to use unsafe functions from c#.

Thanks,

Sam.

2

I know there´s a port of opencv to C#, and I´m pretty sure they are doing all the pointer unsafe code mess you would have to be doing yourself. You can try that. Search the showcase area for opencv, because I do remember some guys managed to have opencv working with Unity in the past, before Kinect and OpenNI was available.

3

If you are using macs then there is something called Syphon which can send video/image data from one program to another with hardly any overhead at all, things stay on the GPU. There is stuff available which enables syphon stuff to appear in Unity. And openframeworks supports both syphon and OpenCV stuff, so it wouldnt be too hard to knock something up in that does the OpenCV side and uses Syphon to get the images into Unity.

http://syphon.v002.info/

This is not likely to be a great solution for certain scenarios but I thought I would mention it anyway.

4

Thanks for your replies. I managed to do it in the end by passing a pointer to a colour array from c# through to the c++ plugin, filling with image data in opencv via the plugin and then setting the texture pixels in c# in unity. Pretty fiddly but can now have web cam frames and movies from opencv as textures.

5

sm519,

Can you share your solution with us?

6

How it is possible to convert image from opencv(BGR, or RGB) to Texture2D in unity ?
Iam working already with this plugin inside unity engine. But , everytime I need to save image as bitmap on hardDisk, and after that refresh the Assets and load the image to Texture. It is really very stupid process and takes a lot of time.

7

Hi! I am searching exaclty want you managed to do. Please, could you share me the code? I am very lost in the process of accessing the image pixels from Unity to IplImages and viceversa.
Thanks!!

8

Hi I’m working on the exact same thing, so if you could please post some of the code it would be very helpful