MonoDevelop unable to attach to Android device


I’m using Unity 3.4.2f3 and MonoDevelop 2.4.2 (shipped with Unity). Building and deploying Android applications works fine on both my Motorola Milestone (Droid) and my Galaxy Nexus with ICS, I even see them listed in the Attach to Process dialog of MonoDevelop.

However when I try to attach MonoDevelop for debugging to any android device running a development build with script debugging enabled, I get the following exception displayed in a popup window inside MonoDevelop. Does anybody know how to fix this, is there any network or phone setting I am missing?

System.Exception: Unable to attach to —> System.Net.Sockets.SocketException: No connection could be made because the target machine actively refused it

Server stack trace:
at System.Net.Sockets.Socket.DoConnect(EndPoint endPointSnapshot, SocketAddress socketAddress)
at System.Net.Sockets.Socket.Connect(EndPoint remoteEP)
at Mono.Debugger.Soft.VirtualMachineManager.ConnectInternal(Socket dbg_sock, Socket con_sock, IPEndPoint dbg_ep, IPEndPoint con_ep)
at System.Runtime.Remoting.Messaging.StackBuilderSink._PrivateProcessMessage(IntPtr md, Object args, Object server, Int32 methodPtr, Boolean fExecuteInContext, Object& outArgs)
at System.Runtime.Remoting.Messaging.StackBuilderSink.PrivateProcessMessage(RuntimeMethodHandle md, Object args, Object server, Int32 methodPtr, Boolean fExecuteInContext, Object& outArgs)
at System.Runtime.Remoting.Messaging.StackBuilderSink.AsyncProcessMessage(IMessage msg, IMessageSink replySink)

Exception rethrown at [0]:
at System.Runtime.Remoting.Proxies.RealProxy.EndInvokeHelper(Message reqMsg, Boolean bProxyCase)
at System.Runtime.Remoting.Proxies.RemotingProxy.Invoke(Object NotUsed, MessageData& msgData)
at Mono.Debugger.Soft.VirtualMachineManager.ConnectCallback.EndInvoke(IAsyncResult result)
at Mono.Debugger.Soft.VirtualMachineManager.EndConnect(IAsyncResult asyncResult)
at Mono.Debugger.Soft.VirtualMachineManager.Connect(IPEndPoint endpoint, IPEndPoint consoleEndpoint)
at Mono.Debugger.Soft.VirtualMachineManager.Connect(IPEndPoint endpoint)
at MonoDevelop.Debugger.Soft.Unity.UnitySoftDebuggerSession.OnAttachToProcess(Int64 processId)
— End of inner exception stack trace —
at MonoDevelop.Debugger.Soft.Unity.UnitySoftDebuggerSession.OnAttachToProcess(Int64 processId)

Any solution to this?