Unable to load simplest plugin on Unity 5

I am using the simplest plugin example given here on unity 5 and mac. However it throws errors as below:

Fallback handler could not load library /Users/aswinthomas/Downloads/Untitled.app/Contents/Frameworks/MonoEmbedRuntime/osx/ASimplePlugin`

In case someone is having the same problem, just place the dll/so with the executable