HuggingFace WebGL app, load a nn model from idbfs

Unity Engine
,
1

Hello, Thomas Simonini from Hugging Face.

I’m trying to make a webgl version of Unity ML-Agents environments where people could watch their agents performing using a hosted model on the Hub for instance ThomasSimonini/MLAgents-Pyramids at main

The idea is to be able to visualize the results of your training or other trainings directly in the webapp.

I currently have a problem, I’m able to download the model from the hub using DownloadHandlerFile but how can I load the NNModel given a path?

Dirty version:

    IEnumerator DownloadFile()
    {
        var uwr = new UnityWebRequest("https://huggingface.co/ThomasSimonini/MLAgents-Pyramids/resolve/main/Pyramids.nn", UnityWebRequest.kHttpVerbGET);
        string path = Path.Combine(Application.persistentDataPath, "model.nn");
        uwr.downloadHandler = new DownloadHandlerFile(path);
        yield return uwr.SendWebRequest();
        if (uwr.result != UnityWebRequest.Result.Success)
            Debug.LogError(uwr.error);
        else
            Debug.Log("File successfully downloaded and saved to " + path);
            Ttext.text = "File successfully downloaded and saved to " + path;

    }

Output => The model is download and saved to idbfs: File successfully downloaded and saved to /idbfs/0b9c32a95ac380a503897024fc05e0df/model.nn

But I don’t understand if and how I can access it from Unity? I know there’s Resources.Load but it’s only for assets inside Resources. So do you know if there is a solution to that?

Thanks!

2

Hey Thomas, that sounds like a fantastic project, bringing ML agent visualizations to the browser. In addition, I also found a good ML tool, Adept（https://www.gate2ai.com/tools/research/adept）， does a fine job integrating human-computer interactions in ML research. For people eager to try such tools, Worth a try！