Unity Web build websocket error

Hello everyone! I’m working on a little project, where you can control the unity game with camera. The image processing happens in python from where i send the data via websockets to unity. First I want to make it locally, but I’m running into a problem where, in the unity editor the C# script connects fine to the python websocket, but once i build the game in WebGL (run it locally with python -m http.server), with the same method the game wont connect anymore. Does anyone know why this happens, and what would the solution for it be? I attached the mentioned c# script that handles the connecting in case i overlooked something in it. Any help would be greatly appreciated, and thanks in advance! Since i cant attach files because i am a new user, i can show my script only this way:

using System;
using System.Collections.Concurrent; // For thread-safe data queue
using UnityEngine;
using UnityEngine.UI;
using WebSocketSharp;
using System.Threading.Tasks;
using TMPro;

public class PythonSocketClient : MonoBehaviour
{
    private WebSocket ws;  // WebSocket instance
    public Text ClientConnect;
    public TMP_InputField urlInputField;  // InputField for user to input the WebSocket URL

    private ConcurrentQueue<int> gestureQueue = new ConcurrentQueue<int>(); // Thread-safe queue for gestures

    public async void Start()
    {
        
        string url = urlInputField.text; // Get the URL from the InputField

        if (string.IsNullOrEmpty(url))
        {
            ClientConnect.text = "Please enter a valid URL!";
            return;
        }
        
        // Automatically format the full WebSocket URL
        string fullUrl = $"{url}/hand_data";
        Debug.Log($"WebSocket URL: {fullUrl}");
        try
        {
            // Initialize the WebSocket connection
            ws = new WebSocket(fullUrl);

            // Event for when WebSocket opens
            ws.OnOpen += (sender, e) =>
            {
                Debug.Log("Connected to Python WebSocket server.");
                ClientConnect.text = "Successfully Connected to Camera!";
            };

            // Event for when WebSocket receives a message
            ws.OnMessage += (sender, e) =>
            {
                try
                {
                    int gesture = int.Parse(e.Data.Trim());
                    // Queue the gesture for processing on the main thread
                    gestureQueue.Enqueue(gesture);
                }
                catch (Exception ex)
                {
                    Debug.LogError($"Error parsing gesture data: {ex.Message}");
                }
            };

            // Event for when WebSocket encounters an error
            ws.OnError += (sender, e) =>
            {
                Debug.LogError($"WebSocket error: {e.Message}");
                ClientConnect.text = $"Failed to Connect to Camera!\n{e.Message}";
            };

            // Event for when WebSocket closes
            ws.OnClose += (sender, e) =>
            {
                Debug.Log("Disconnected from WebSocket server.");
                ClientConnect.text = "Disconnected from Camera!";
            };

            // Connect to the WebSocket server
            ws.Connect();

            // Await a short delay to update the UI
            if (ws.ReadyState == WebSocketState.Open)
            {
                await Task.Delay(3000); // Wait for 3 seconds
                ClientConnect.text = "";
            }
            else
            {
                Debug.LogError("Failed to connect to WebSocket server.");
                ClientConnect.text = "Failed to Connect to Camera!\nRefresh Page or Play with Traditional Controls";
                await Task.Delay(6000); // Wait for 6 seconds
                ClientConnect.text = "";
            }
        }
        catch (Exception e)
        {
            Debug.LogError($"WebSocket connection error: {e.Message}");
            ClientConnect.text = "Failed to Connect to Camera!\nRefresh Page or Play with Traditional Controls";
            await Task.Delay(6000); // Wait for 6 seconds
            ClientConnect.text = "";
        }
    }

    void Update()
    {
        // Process all queued gestures
        while (gestureQueue.TryDequeue(out int gesture))
        {
            // Update the singleton with the new gesture
            GestureManager.Instance?.UpdateGesture(gesture);
        }
    }

    void OnApplicationQuit()
    {
        if (ws != null && ws.ReadyState == WebSocketState.Open)
        {
            ws.Close(CloseStatusCode.Normal); // Gracefully close the WebSocket connection
        }
    }
}

Task.Delay() is not supported in WebGL. This will likely await forever. Use coroutines instead.

I applied the modifications suggested, and added some debugging steps to see if there is an error, when running the built game, and trying to connect to the websocket, at line 89 the code catches the following execption : “Object reference not set to an instance of an object”. Note: this only happens in the built game, in unity editor connection is made successfully!

Hi!

Does the library WebSocketSharp support WebGL? It is of course possible to use WebSockets for connectivity in a browser but this library would need to go through the browser API for the websocket connection and not use .NET networking code or C/C++ plugins.
It would need to use a different implementation for the Web that uses a JavaScript plugin.

1 Like

Check the callstack. You need to find out which object that is. If you have multiple managed object references on that line, separate them into multiple lines and try again.

I hadn’t noticed you are trying to use WebSocketSharp which does not work with Unity WebGL as has been pointed out here for instance.

Unfortunately, Unity Transport won’t work for you either since it’ll only establish connections to other Unity Transport based applications. So you need to write your WebSocket code in Javascript or you change the server endpoint to also be a Unity application, which could then interface with Python locally.

1 Like

Thank you for the advice, I switched to Javascript, and made this .jslib file for the project, and modified my C# script also to be compatible with this method, here is the javascript i wrote:

    // Connect WebSocket (Ensure no return value)
    WebSocketConnect: function (urlPtr) {
        var url = UTF8ToString(urlPtr); // Convert URL pointer to string
        console.log("Connecting to WebSocket at: " + url);

        try {
            websocket = new WebSocket(url);

            websocket.onopen = function () {
                console.log("WebSocket connection established.");
            };

            websocket.onmessage = function (event) {
                console.log("WebSocket message received: " + event.data);
                messageQueue.push(event.data); // Add the message to the queue
            };

            websocket.onerror = function (error) {
                console.error("WebSocket error:", error);
            };

            websocket.onclose = function () {
                console.log("WebSocket connection closed.");
            };
        } catch (e) {
            console.error("Error initializing WebSocket:", e);
        }

        // No return value here, just the connection process
    },

    // WebSocket send function
    WebSocketSend: function (messagePtr) {
        var message = UTF8ToString(messagePtr);
        if (websocket && websocket.readyState === WebSocket.OPEN) {
            websocket.send(message);
            console.log("WebSocket message sent: " + message);
        } else {
            console.error("WebSocket is not connected.");
        }
    },

    // WebSocket close function
    WebSocketClose: function () {
        if (websocket) {
            websocket.close();
            console.log("WebSocket connection closed.");
        }
    },

    // WebSocket receive function (modified to return a pointer to the message)
    WebSocketReceive: function () {
        if (messageQueue.length > 0) {
            // Dequeue the first message
            var message = messageQueue.shift();
            var buffer = allocate(intArrayFromString(message), ALLOC_NORMAL);
            return buffer;
        }
        return 0; // Return null if no messages are available
    }
});

And my c# script:

using System;
using System.Collections.Concurrent;
using System.Collections;
using UnityEngine;
using TMPro;
using UnityEngine.UI;

#if UNITY_WEBGL && !UNITY_EDITOR
using System.Runtime.InteropServices; // For WebGL plugin interop
#else
#if !UNITY_WEBGL || UNITY_EDITOR
using WebSocketSharp; // For WebSocketSharp in non-WebGL builds
#endif
#endif

public class PythonSocketClient : MonoBehaviour
{
    private bool isWebGL = false; // Flag to check if running in WebGL

#if !UNITY_WEBGL || UNITY_EDITOR
    private WebSocket ws; // WebSocketSharp instance (non-WebGL)
#endif

    public Text ClientConnect; // UI Text for connection status
    public TMP_InputField urlInputField; // InputField for WebSocket URL input
    private ConcurrentQueue<int> gestureQueue = new ConcurrentQueue<int>(); // Thread-safe queue for gestures

#if UNITY_WEBGL && !UNITY_EDITOR
    // Import WebGL plugin functions
    [DllImport("__Internal")]
    private static extern void WebSocketConnect(string url);

    [DllImport("__Internal")]
    private static extern void WebSocketSend(string message);

    [DllImport("__Internal")]
    private static extern void WebSocketClose();

    [DllImport("__Internal")]
    private static extern IntPtr WebSocketReceive();
#endif

    public void Start()
    {
        isWebGL = Application.platform == RuntimePlatform.WebGLPlayer;

        string url = urlInputField.text;

        if (string.IsNullOrEmpty(url))
        {
            ClientConnect.text = "Please enter a valid URL!";
            return;
        }

        string fullUrl = $"{url}/hand_data";
        Debug.Log($"WebSocket URL: {fullUrl}");

        try
        {
            if (isWebGL)
            {
#if UNITY_WEBGL && !UNITY_EDITOR
                WebSocketConnect(fullUrl); // Call JavaScript interop to connect to WebSocket
                ClientConnect.text = "Connecting to WebSocket...";
                StartCoroutine(WebGLReceiveMessages());
#endif
            }
            else
            {
#if !UNITY_WEBGL || UNITY_EDITOR
                ws = new WebSocket(fullUrl);

                ws.OnOpen += (sender, e) =>
                {
                    Debug.Log("Connected to Python WebSocket server.");
                    ClientConnect.text = "Successfully Connected to Camera!";
                };

                ws.OnMessage += (sender, e) =>
                {
                    try
                    {
                        int gesture = int.Parse(e.Data.Trim());
                        gestureQueue.Enqueue(gesture);
                    }
                    catch (Exception ex)
                    {
                        Debug.LogError($"Error parsing gesture data: {ex.Message}");
                    }
                };

                ws.OnError += (sender, e) =>
                {
                    Debug.LogError($"WebSocket error: {e.Message}");
                    ClientConnect.text = $"Failed to Connect to Camera!\n{e.Message}";
                };

                ws.OnClose += (sender, e) =>
                {
                    Debug.Log("Disconnected from WebSocket server.");
                    ClientConnect.text = "Disconnected from Camera!";
                };

                ws.Connect();
#endif
            }
        }
        catch (Exception e)
        {
            Debug.LogError($"WebSocket connection error: {e.Message}");
            ClientConnect.text = "Failed to Connect to Camera!\nRefresh Page or Play with Traditional Controls";
        }
    }

    private IEnumerator WebGLReceiveMessages()
    {
#if UNITY_WEBGL && !UNITY_EDITOR
        while (true)
        {
            IntPtr messagePtr = WebSocketReceive();
            if (messagePtr != IntPtr.Zero)
            {
                string message = Marshal.PtrToStringAnsi(messagePtr); // Convert to string
                if (!string.IsNullOrEmpty(message))
                {
                    try
                    {
                        int gesture = int.Parse(message.Trim());
                        gestureQueue.Enqueue(gesture);
                    }
                    catch (Exception ex)
                    {
                        Debug.LogError($"Error parsing gesture data: {ex.Message}");
                    }
                }
            }
            yield return null; // Wait for the next frame
        }
#else
        yield break;
#endif
    }

    void Update()
    {
        while (gestureQueue.TryDequeue(out int gesture))
        {
            GestureManager.Instance?.UpdateGesture(gesture);
        }
    }

    void OnApplicationQuit()
    {
#if UNITY_WEBGL && !UNITY_EDITOR
        WebSocketClose();
#else
        if (ws != null && ws.ReadyState == WebSocketState.Open)
        {
            ws.Close(CloseStatusCode.Normal);
        }
#endif
    }
}

Altough this should work in theory, im getting quite an annoying build error when trying to build the WebGl, these two errors keep appearing, altough i already checked that the return type of the functions match, for some reason the compiler throws the error anyway. (i tried to remove all the websocketsharp parts of the c# script, but that doesnt change anything)

Building Library\Bee\artifacts\WebGL\build\debug_WebGL_wasm\build.js failed with output:
wasm-ld: error: function signature mismatch: WebSocketConnect
>>> defined as (i32) -> void in E:/Rock paper/Rock Paper Scissors/Library/Bee/artifacts/WebGL/il2cppOutput/build/GameAssembly.a(99hnddnfza5b.o)
>>> defined as (i32) -> i32 in E:/Rock paper/Rock Paper Scissors/Library/Bee/artifacts/WebGL/il2cppOutput/build/GameAssembly.a(fqzb4c2hbp3k.o)
emcc: error: 'E:/Unity/6000.0.24f1/Editor/Data/PlaybackEngines/WebGLSupport/BuildTools/Emscripten/llvm\wasm-ld.exe @C:\Users\Alpar\AppData\Local\Temp\emscripten_pcm3_hou.rsp.utf-8' failed (returned 1)
Build completed with a result of 'Failed' in 78 seconds (78231 ms)
Building Library\Bee\artifacts\WebGL\build\debug_WebGL_wasm\build.js failed with output:
wasm-ld: error: function signature mismatch: WebSocketConnect
>>> defined as (i32) -> void in E:/Rock paper/Rock Paper Scissors/Library/Bee/artifacts/WebGL/il2cppOutput/build/GameAssembly.a(99hnddnfza5b.o)
>>> defined as (i32) -> i32 in E:/Rock paper/Rock Paper Scissors/Library/Bee/artifacts/WebGL/il2cppOutput/build/GameAssembly.a(fqzb4c2hbp3k.o)
emcc: error: 'E:/Unity/6000.0.24f1/Editor/Data/PlaybackEngines/WebGLSupport/BuildTools/Emscripten/llvm\wasm-ld.exe @C:\Users\Alpar\AppData\Local\Temp\emscripten_pcm3_hou.rsp.utf-8' failed (returned 1)

UnityEngine.GUIUtility:ProcessEvent (int,intptr,bool&)

Is there a more simple method of processing websockets trough javascript or have i just overlooked something stupidly simple in these codes?

Is WebSocketConnect(string) defined anywhere else in your codebase? If not did you change it from int to void return type at some point?
I would try to do a clean build/delete Library folder and see if it works. It could be an old build artifact is accidentally included in your build.

Oh and some more tipps:
You can also use callbacks to get updates from the WebSocket into C#

This way you don’t have to poll for new messages in the coroutine.

You can also return strings in your JS plugin and they will automatically be converted to a C# string object. This will also automatically manage the memory allocation and free it when no longer needed.

mergeInto(LibraryManager.library, {
  StringReturnValueFunction: function () {
    var returnStr = "bla";
    var bufferSize = lengthBytesUTF8(returnStr) + 1;
    var buffer = _malloc(bufferSize);
    stringToUTF8(returnStr, buffer, bufferSize);
    return buffer;
  },
});
using UnityEngine;
using System.Runtime.InteropServices;

public class NewBehaviourScript : MonoBehaviour {
    [DllImport("__Internal")]
    private static extern string StringReturnValueFunction();

    void Start() {       
        Debug.Log(StringReturnValueFunction());
    }
}