Native c++ plugin works in editor but not in build

Unity Engine
1

I have a minimal c++ script that’s called from a minimal c# script. The dll is compiled on Windows 10 using mingw g++ and it works fine in the editor. Build and run however leads to the following error message:

Plugins: Failed to load 'C:/(...)/dll-example_Data/Plugins/libTest.dll' with error 'A dynamic link library (DLL) initialization routine failed.
Here’s the .cs file:

using System.Collections;
using System.Collections.Generic;
using UnityEngine;
using System.Runtime.InteropServices;
public class TestScript : MonoBehaviour
{
[DllImport("libTest", CallingConvention = CallingConvention.StdCall, ExactSpelling = true)]
public static extern int TestFunction();
void Start()
{
Debug.LogFormat("libTest says: {0}", TestFunction());
}
}

And the Test.cpp:

extern "C" {
int __stdcall __declspec(dllexport) TestFunction() {
int* test = new int[1]; // removing this line will make everything work when building
return 4;
}
}

Here’s how I build it:

g++ -O2 -g -Wall -c -fmessage-length=0 Test.cpp
g++ -shared -o libTest.dll Test.o

As soon as I delete the marked line (int* test = …), recompile and replace the .dll everything works fine using the build .exe AND the editor (with this line in.

Any suggestions would be appreciated!

2

Using the Microsoft Debug Diagnostic Tool I found out that I get following exception:

In dll-example__PID__(...)C0000005.dmp the assembly instruction at libTest!TestFunction+ee80 in C:\(...)\dll-example_Data\Plugins\libTest.dll has caused an access violation exception (0xC0000005) when trying to read from memory location 0x92fff800 on thread 0

That made me think the error could be due to something -fPIC related but adding this flag to the compiler didn’t help either.

3

I’m guessing it’s trying to find the library behind the new allocator in C++ and it can’t. What if you replace the line with just a reference to a static local, like so:

static int myOneInt;
int* test = &myOneInt;

I assume that will work just fine, even if it isn’t what you want in terms of function. That might isolate if it is a library linkage problem.

For my native libraries under Unity3D I build windows DLLs with entrypoints marked up like so:

__declspec(dllexport) void foo() { ... }

But that looks equivalent to yours.

Finally I like it up with:

gcc -o libtest1.dll -s -shared *.o -Wl,--subsystem,windows

Could be those extra args get you what you need? I am not using new/delete in my code however, but I am using malloc()/free() everywhere (it’s old C code in a C++ wrapper).

ALSO: I found this link to some stuff I read when I did it all:

4

Dear Kurt,

thank you so buch for your reply!

The problem really only occurs as soon as I add the new operator. So using malloc/alloca etc. works fine but as soon as I reference some C++ library, or only use e.g. the new operator, it stops working. Another example would be this:

int test() {
        int* test = new int[3];
        return test[2];
}

extern "C" {
int __stdcall __declspec(dllexport) TestFunction() {
        // we're not even using test() here!
        return 9;
}
}

Thanks a lot for the -Wl,--subsystem,windows hint and the link. The example from that page works fine but it’s again pure C code. Adding “new” and compiling with g++ also breaks these examples.

Btw: the size of the .dll gets much bigger (300kB instead of 124kB) as soon as I add “new”. dumpbin.exe -exports libTest.dll however shows the same output with and without “new”

5

Yeah, it has to be something about either the bindings to new/delete, or the bindings between new/delete and the underlying malloc/free mechanism.

It is possible there is some startup code that normally gets called BEFORE you arrive at main() in a standalone program that is missing here. In C that is usually c0.obj back in the day, but I don’t know what the gcc convention is, or if there is a different C++ one necessary to use new/delete.

Oooh, check this: google for “c++ new delete failing in dll” … I saw this blurb, second block of text:

https://groups.google.com/forum/#!topic/microsoft.public.vc.language/CHogNArGAFQ

Quote: "It sounds like you’re not linking with the shared DLL version of the
library.

Keep in mind how the EXE and DLL are made. They must share the same
new/delete operators. Often times (especially inlines) the code for these
will be compiled into the executable file. This means the EXE and DLL have
their own copies of the operators. Make sure you check the box that says
something like “Multithreaded DLL” in the
Build->Settings->C/C+±>Category:Code Generation->Use Run-time
Library:Multithreaded DLL window."

That might be something to look at. There is probably a gcc option for the “Multithreaded DLL” option.

6

That’s brilliant! Overriding new[ ] fixes the issue:

#include <new>
#include <cstdlib>

void* operator new[](std::size_t sz){
    return std::malloc(sz);
}

extern "C" {
int __stdcall __declspec(dllexport) TestFunction() {
        int* test = new int[3]; // removing this line will make everything work when building
        return test[2];
}
}

That’s definitely a good fix as overriding the new/free operators is anyway useful. However I’d still be interested in a proper way to fix this. That “Multithreaded DLL” keyword didn’t help me so far but I’m still searching…

Thank you so much for your help!

7

Fixed it: It turned out that my environment was messed up a bit. Instead of using the mingw-64 g++ compiler I accidentally used the tdm-gcc compiler. Using the current mingw-64 everything works fine even without defining new/delete. So it seems to be a tdm related bug.

8

I know this is not particularly to the question but if you want to write some code that is for sure memory safe you could also write in Rust instead of C++. The usage is quite simple:

But nvm just though you guys might want to check this out.
Cheers :wink: