VNC Client Code for Unity

Unity Engine
1

I would like to build a VNC client using Unity. I am wondering if there are any plugins, libraries, packages, tutorials, etc that I can use to help me along the way?

Any input or guidance on how I might achieve this will be greatly appreciated!!

Thanks!

1 Like
2

Find a .net vnc library whose api has some way to get at the stream of image data.

Draw that image data to a Texture2D.

Attach that Texture2D to some surface in unity.

Really, youā€™re asking for something really weird. Youā€™re not really going to find many unity users that have implemented this in itself.

Thing is, youā€™ll definitely find lots of blogs/tuts/info about implementing VNC in .Net/Mono. All you need to do make the connection from unity to thatā€¦ which is rendering the stream to a Texture2D, and forwarding the mouse/keyboard events to the server.

1 Like
3

Thanks for the info! I figured this isnā€™t being done or hasnā€™t been done by very many Unity users but I do have a need for it all of a sudden :-/

You wouldnā€™t happen to have any links to any of those blogs/tuts/info you mentioned that might work within Unity?

4

Not that I know of, especially not one that would work off the bat with unity.

Hereā€™s what Iā€™d doā€¦

Here is a vnc implementation in .Net:
https://cdot.senecacollege.ca/projects/vncsharp/doc.html

The source code is available for download. This means we can rip it apart and take a look at how stuff is done.

Now, the directions on the site tell you to use the WinForms control called ā€˜RemoteDesktopā€™ for including it in your projectā€¦ but that we canā€™t do, because weā€™re in unity. So what we need to do is mimic this winforms control.

So if we load up the project, and open that file, we can see how they connect the vnc, and how they draw it to winforms control. Youā€™ll be connecting to vnc in the same exact way, but instead of drawing to the winform control, youā€™ll write it to a texture2d.

Digging through it, it appears to store a System.Drawing.Bitmap, and on the VncUpdate event of the vnc client, it draws the image data to this Bitmap. Which then in the OnPaint event (this is a winforms event used to draw the graphics of the control) it draws that bitmap to the control.

This, is the part we wantā€¦ thing is System.Drawing isnā€™t supported in Unity. So we need to rip this dependency out of it. How this is getting drawn is the VncUpdate hands us an IDesktopUpdater that accepts a Bitmap to draw to. All we have to do is convert this IDesktopUpdater to support a Texture2D call, implement said interface, and then go into the factory that creates the IDesktopUpdater and make it create our custom version again.

Itā€™d probably be a good measure to purge any and all code that references System.Drawing and System.Windows.Forms. As just having it can cause issues in unity, even if you donā€™t access it.

1 Like
5

I did some extra digging. To purge the System.Windows.Forms and System.Drawing dependenciesā€¦

thereā€™s 6 classes to remove:
RemoteDesktop
PasswordDialog
#these are just the display controls which we want to mimic

VncClippedDesktopPolicy
VncScaledDesktopPolicy
VncDesktopTransformPolicy
VncDesignModeDesktopPolicy
#these are behaviour policies for how the controls above should act depending the state of the winform.

Also, in VNCClient, thereā€™s a bit of code at line 390 that attempts to sync with the WinForms UI threadā€¦ this should be removed, just call the VncUpdate event. You will have to sync to the unity thread when handling the eventā€¦ but honestly, that syncing should be handled by the event receiver, not the event sender (not sure why the put such a dependency into the VncClient classā€¦ thatā€™s bad design).

see:

// In order to play nicely with WinForms controls, we do a check here to
// see if it is necessary to synchronize this event with the UI thread.
if (VncUpdate.Target is System.Windows.Forms.Control) {
    Control target = VncUpdate.Target as Control;
    if (target != null)
        target.Invoke(VncUpdate, new object[] { this, e });
} else {
    // Target is not a WinForms control, so do it on this thread...
    VncUpdate(this, new VncEventArgs(er));
}

And lastly, several classes have minor references to types like ā€˜Pointā€™, ā€˜Rectangleā€™ and ā€˜Bitmapā€™ from the System.Drawing namespace. Youā€™d just change these to use ā€˜Vector2ā€™, ā€˜Rectā€™, and ā€˜Texture2Dā€™ instead.

6

OMG youā€™re so awesome. Iā€™m going to start digging into this asap (tonight hopefully) will keep updating this thread as I go. Thank you so much!

7

This was a helpful article for my pet projectā€¦ thanks!

8

@iPedro

Any luck?

9

No luck - the project never got off the ground although I still think itā€™s an interesting one.

10

Maybe this project has some relevant hints ā€¦

1 Like
11

Iā€™m really interested in this, was any progress made on this?

Iā€™m a Java dev thatā€™s new to C#/.Net and Windows APIs in general (and totally new to Unity or anything like it). So my progress will be slow, but for now Iā€™m going to poke with lordofductā€™s suggestions.

1 Like
12

@barab , Iā€™m in a similar boat. Did you make any progress?

Right now Iā€™m converting all of the Rectangles and Bitmaps to their respective Rects and Texture2Ds, but Iā€™m running into troubles converting BitmapData into a Texture2D equivalent.

13

Hi, looking for the same to create a virtual desktop concept in VR. If you guys are able to share code or discuss id like to help with getting that last issue with texture solved

14

Hello all,

Iā€™ve made an open source project that is based on a csharp integration of a vnc client.
It is available here : GitHub - cfloutier/Unity-VNC-Client: Unity 3D Vnc Client : Port of the VncSharp: A .NET VNC Client Library

The csharp decoding is quite slow but it is a good stating point.

3 Likes
15

Hey there,
6 years later, this trail still have interests.

@YohanB did you succeed in your project? Iā€™m looking to do exactly the same.
@cfloutier Thanks for sharing. Did you had any progress on your project? Iā€™m about to try your integration.

Do you guys have any other leads about remote desktop streaming frameworks?
Cheers

16

Hi, Iā€™m also interested about remote desktop streaming.
@Untoldecay did you have any luck with the integration of @cfloutier 's project? I see that it is quite old, made with Unity 5.

17

Really cool thread. Might try playing with this.

18

Looks like this pull request updated the unity version to 2017 and there is an assigned open issue to upgrade to 2020LTS.
https://github.com/cfloutier/Unity-VNC-Client/pull/5
https://github.com/cfloutier/Unity-VNC-Client/issues/8