OpenCV (EMGUCV wrapper) integration in Unity

Hi everyone,

As you know OpenCV is very useful library that let you do amazing and powerful things in Computer vision. So I passed a good time to figure out how to use it in Unity3d, I had many problems, and searching in the Net, I have found several suggestions but not one worked for me, so I wasted more time to figure out how to integrate it, and thanks god, now I’m using it in Unity, this is what I did for everyone who want to use it:

  • I’m using a Unity Pro 4.0
  • This version of Emgu CV (emgucv-windows-universal-gpu 2.4.9.1847)
  • My target for unity project is: windows and not web player

1- Install this version of EmguCV, witch create a folder EMGU in C: disc
2- In this folder you will have a single folder named “emgucv-windows-universal-gpu 2.4.9.1847”
3- Go to “bin folder”
4- From the “bin folder” you need to Copy all the dlls in the x86 folder (22 dlls), and also those in the bin directly (12 dlls: Emgu.CV, Emgu.CV.UI….) For X64 target, you can try the same thing, I think this will work by taking the dlls from the X64 folder.
5- Create a folder named Plugins, in your Unity project (Assets Plugins)
6- Paste all these 34 dlls file in the Plugins folder
7- From this 34 dll files, copy the “npp32_50_35” and “cudart32_50_35” into the Unity editor folder, in my case it’s “C:\Program Files (x86)\Unity\Editor”
8- You are done for the Dlls stuff.

This is how to test it:
1- Create a small script, that take a “picture1.jpg” and change the color of the first 200 diagonal pixels color to white color (255,255,255) and then save the new “picture2.jpg” This is what you have to put in the script:
using UnityEngine;
using System.Collections;
using Emgu.CV;
using Emgu.CV.Util;
using Emgu.CV.UI;
using Emgu.CV.CvEnum;
using Emgu.CV.Structure;
using System.Runtime.InteropServices;
using System;
using System.Drawing;

Image<Bgr, byte> picture = new Image<Bgr, byte>(“C:\picture1.jpg”);
Bgr myWhiteColor = new Bgr(255, 255, 255);
For (int i=0; i<200; i++)
{picture[i,i]= color;}
picture.Save(“C:\picture2.jpg”);

2- By adding the different dlls in the Plugins folder they will be referenced automatically in the Mono Editor, but probably you will have a problem with “System.Drawing” library, so don’t forget using System.Drawing; and also you have to reference it by going to your script editor, if MonoDevelop-Unity, go the solution explorer in the left, right click on Refrences Edit references and search for system.drawing in the left column and check it to see it in the right column  then press ok
3- Also don’t forget to change the build settings in Unityfilebuild settings in this way:

  • PC Mac and Linux instead of webplayer
  • Target= windows
  • Architecture= x86
  • Then push the “player settings button” go to the “other settings in the new screen” and set Api Compatibility lavel to .NET 2.0, instead of .NET 2.0 subset
    For now you’re free to test your script and do awesome work using OpenCV
    PS: I’m not sure why exactly the “npp32_50_35” and “cudart32_50_35” have to be in the Editor folder, but it’s the only situation that worked for me, and sorry if my English isn’t good.

Good luck for everyone

1 Like

Thank you. I had a shot in the past with emgucv but had many problems with unsafe pointers.

Hello, mbougaa.

First of all, thank you for your post.

I did everything as you said but I got the following error:

Internal compiler error. See the console log for more information. output was:
Unhandled Exception: System.Reflection.ReflectionTypeLoadException: The classes in the module cannot be loaded.

at (wrapper managed-to-native) System.Reflection.Assembly:GetTypes (bool)

do you have any idea ??

thanks in advance

Hi,

I’m not sure if this is the problem, but i found this

maybe you are working on webplayer mode, try to change it if it’s your case to (in Unity–> File–>build setting–> choose the second option in the Platform options-PC Mac and linux standalone- ),
Remark: you had to use Unity Pro and not a trial version

I hope this will work for you

Hi jerrylewisbh,
I had this problem and i’ve solved it with create a folder “Editor” in “Assets” folder and put “npp32_50_35” and “cudart32_50_35” in “Editor” folder

1 Like

Hi!

I have successfully compiled emgucv into a usable library for mac/monodevelop/unity.

However I am really struggling with somehow sending the webcamtexture from Unity side to EmguCV. Would anyone have any insights into this?

1 Like

I got this working but somehow some functions did not work and I got some errors like this example:

DllNotFoundException: C:/…/Assets/Plugins/cvextern.dll
Emgu.CV.Cvb.CvBlobs…ctor ()
TestOpenCV.GetBlobs (Emgu.CV.Image`2 source) (at Assets/TestOpenCV.cs:58)
TestOpenCV.Update () (at Assets/TestOpenCV.cs:45)

But I found that copying all opencv / emgu dlls into the project folder solved the issue.

p.s. I was using EmguCV version 2.9 (emgucv-windows-universal-cuda 2.9.0.1922)

Thanks for that! I was having the same problem, this is the Fix!!!

Hello Everyone,

If someone is trying to integreate EMGU inside Unity and is stuck some where, Follow the exact same as the On Post and after installing the emgucv-windows-universal-gpu 2.4.9.1847 Do remove the opencv dll that has GPU in the name, which will not be required and occupies a hectic 330+ mb

and if you are using a 64 bit system, dont copy dlls from x64 folder, it will not work, copy it from the x86 itslef. The reason is unity is compiled for 32 bit OS. and other than that it is the same as the OP.

While taking build, Copy the “npp32_50_35” and “cudart32_50_35” dlls to the build location out of data. The dlls should be in the same folder as the build.exe and build_data folder.

Good luck :slight_smile:

I’m confused with the internal compiler error. after following the instructions, the error is fixed!

i did everything you sayd but i get this error:

Assets/scriptEMGU.cs(11,14): error CS0234: The type or namespace name Drawing' does not exist in the namespace System’. Are you missing an assembly reference?

i dont get it when i put it as import in a javascript code “import System.Drawing;”

what do i do?

Copy the System Drawing DLL from Unity installation folder and paste it inside Asset/Plugins

Hello everybody,

I’m receiveng that error message trying to build:

“ArgumentException: The Assembly Microsoft.VisualStudio.DebuggerVisualizers is referenced by Emgu.CV.DebuggerVisualizers.VS2008. But the dll is not allowed to be included or could not be found.
UnityEditor.AssemblyHelper.AddReferencedAssembliesRecurse (System.String assemblyPath, System.Collections.Generic.List1 alreadyFoundAssemblies, System.String[ ] allAssemblyPaths, System.String[ ] foldersToSearch, System.Collections.Generic.Dictionary2 cache, BuildTarget target) (at C:/BuildAgent/work/d3d49558e4d408f4/Editor/Mono/AssemblyHelper.cs:113)
UnityEditor.AssemblyHelper.FindAssembliesReferencedBy (System.String[ ] paths, System.String[ ] foldersToSearch, BuildTarget target) (at C:/BuildAgent/work/d3d49558e4d408f4/Editor/Mono/AssemblyHelper.cs:148)
UnityEditor.BuildPlayerWindow:BuildPlayerAndRun()”

Any idea? I’ve checked all the step and I can’t find the problem.

Thanks mbougaa for the post.

DllNotFoundException: cvextern
Emgu.CV.HOGDescriptor…ctor ()
Emgu.CV.CvInvoke.cvLoad (System.String fileName, IntPtr memstorage, System.String name, IntPtr realName)
Emgu.CV.HaarCascade…ctor (System.String fileName)
NewBehaviourScript.Start () (at Assets/Scripts/NewBehaviourScript.cs:70)

I have the same question with you, I had already put 34 dlls into plugins, where are others?

Hi svenskefan!

I am trying to do the same thing but keep getting error in Unity.

"Internal compiler error. See the console log for more information. output was:
Unhandled Exception: System.Reflection.ReflectionTypeLoadException: The classes in the module cannot be loaded.
at (wrapper managed-to-native) System.Reflection.Assembly:GetTypes (bool)
at System.Reflection.Assembly.GetTypes () [0x00000] in :0
at Mono.CSharp.RootNamespace.ComputeNamespaces (System.Reflection.Assembly assembly, System.Type extensionType) [0x00000] in :0
at Mono.CSharp.RootNamespace.ComputeNamespace (Mono.CSharp.CompilerContext ctx, System.Type extensionType) [0x00000] in :0
at Mono.CSharp.GlobalRootNamespace.ComputeNamespaces (Mono.CSharp.CompilerContext ctx) [0x00000] in :0
at Mono.CSharp.Driver.LoadReferences () [0x00000] in :0
at Mono.CSharp.Driver.Compile () [0x00000] in :0
at Mono.CSharp.Driver.Main (System.String[ ] args) [0x00000] in :0 "

I am literally new to OpenCV…have started exploring yesterday. It would be great if you could help me.

Thanks in advance! :slight_smile:

Hi ! I solve this problem !
First - i found Microsoft.VisualStudio.DebuggerVisualizers.dll and put it in /Assets/Plugins/ folder
Second - i found Antlr3.Runtime.dll and put it in /Assets/Plugins/ folder
Third - i found mysql.dll and put it in /Assets/Plugins/ folder
Fourth - i download binaries from http://www.codeproject.com/Articles/20884/Introducing-the-LinFu-Framework-Part-I-LinFu-Dynam and put all dll’s it in /Assets/Plugins/ folder

And i built my project. If you any questions - write here

(in picture - my plugins folder structure)

Good luck !

1779729--112968--Снимо21231231231231к.PNG

I got this working but somehow some functions did not work and I got some errors like this example:

DllNotFoundException: E:/4.5.2Workspace/EmguCV/Assets/Plugins/cvextern.dll
Emgu.CV.CascadeClassifier…ctor (System.String fileName)
Test1.FaceRecognition () (at Assets/Scripts/Test1.cs:55)
Test1.Update () (at Assets/Scripts/Test1.cs:42)

But I found that copying all opencv / emgu dlls into the project folder solved the issue.

p.s. I was using EmguCV version 2.9 (emgucv-windows-universal-cuda 2.9.0.1922) Unity3d version 4.5.2 pro

Sharing my experience :
I am using unity pro Version 4.6.1f1 and Emgucv 2.4.10.1940, and got this error below with my first try:

“Internal compiler error. See the console log for more information. output was:
Unhandled Exception: System.Reflection.ReflectionTypeLoadException: The classes in the module cannot be loaded.”
I fixed it by set .NET 2.0 subset to NET 2.0, in the player setting.

then I got another error below:
"DllNotFoundException: opencv_core2410 "
fixed the error by copying the files “nppc32_65.dll” “nppi32_65.dll” “npps32_65.dll” “cudart32_65.dll” from Emgucv folder to the unityEditor folder

Now Its working properly
thanks for all the post above!

Can this be used in unity free?

Is it possible to do swipe detection using the EmguCv asset?