Uncompressed asset bundles and memory in WebGL

So I have a few questions about uncompressed asset bundles in WebGL. For various reasons, we are “emulating” the Resources folders in Unity by using an uncompressed asset bundle and loading it with LoadFromFile. This works great as it allows us to dynamically load assets from the asset bundle without having to keep the asset bundle itself loaded in memory.

However with WebGL we can’t use LoadFromFile since we have to download the asset bundle first (in other platforms we just keep it in Streaming Assets). It seems like from this chart using WWW.LoadFromCacheOrDownload will keep the asset bundle in memory the first time it’s downloaded, but then load it from disk subsequent times (assuming caching works). Is this the correct understanding? Will the uncompressed asset bundle still remain in memory (the first time or when caching fails) even after I dispose of the WWW object? And in “WebGL land”, does that count against Unity’s memory heap? I would really like to avoid having the asset bundle itself in memory. I believe this is technically possible (I think assets in Resources folders that haven’t been loaded yet normally don’t count against the heap) but I’m not sure how to achieve this in WebGL.

  • asset bundles are cached, assuming the browser supports indexedDB.
  • once downloaded, or loaded from cache, they will be stored in the Unity Heap
  • memory will be deallocated once you dispose the WWW object

so, ideally you dispose the asset bundle as soon as you don’t need it anymore.

hm…
I’m still using: new WWW(asset) and www.assetBundle; because: `WWW.LoadFromCacheOrDownload - didn’t work on FireFox, trying in unity 5.0

at the moment, there is no known issue regarding asset bundle caching. It should work fine with Unity 5.3 and Firefox 43.

1 Like

Thanks for the info. So essentially for WebGL, we should never use uncompressed asset bundles correct (since no matter what it’s put on the heap)? Is there any reason why I shouldn’t just always use LZ4 compressed bundles for WebGL?

Unfortunately I can’t unload the asset bundle that “emulates” the resources folder (we have to load objects on demand); do you recommend some other approach to load assets on demand without being on the memory heap (i.e. is there any way I can load objects on demand from indexedDB)?

what is the progress of this thread?

Use this plugin instead of LoadFromCacheOrDownload (and the corresponding caching functionality in UnityWebRequest): CachedXMLHttpRequest | Tutorial Projects | Unity Asset Store

It will download and cache your asset bundle (any non-custom GET request really) without putting it on the Unity memory heap. It’s a must have for Unity webgl.

I don’t know how this plugin works, i guess it automatically take over all www request and cache them.?

I have already imported Unity WebGL utilities provided by Knogregate and it has
Updated CachedXMLHttpRequest but still don’t know it is working or not in my project. I can see that there is plugin in my project. while i am loading asset bundle like this (thanks to).

IEnumerator DownloadAndCache(string bundleURL, string assetName = "")
    {
        // Wait for the Caching system to be ready
        while (!Caching.ready)
        {
            //Debug.Log("cache not loading");
            yield return null;
        }

        // if you want to always load from server, can clear cache first
        //        Caching.CleanCache();

        // get current bundle hash from server, random value added to avoid caching
        UnityWebRequest www = UnityWebRequest.Get(bundleURL + ".manifest?r=" + (Random.value * 9999999));
        //Debug.Log("Loading manifest:" + bundleURL + ".manifest");

        // wait for load to finish
        yield return www.SendWebRequest();

        // if received error, exit
        if (www.isNetworkError == true)
        {
            Debug.LogError("www error: " + www.error);
            www.Dispose();
            www = null;
            yield break;
        }

        // create empty hash string
        Hash128 hashString = (default(Hash128));// new Hash128(0, 0, 0, 0);

        // check if received data contains 'ManifestFileVersion'
        if (www.downloadHandler.text.Contains("ManifestFileVersion"))
        {
            // extract hash string from the received data, TODO should add some error checking here
            var hashRow = www.downloadHandler.text.ToString().Split("\n".ToCharArray())[5];
            hashString = Hash128.Parse(hashRow.Split(':')[1].Trim());

            if (hashString.isValid == true)
            {
                // we can check if there is cached version or not
                if (Caching.IsVersionCached(bundleURL, hashString) == true)
                {
                    Debug.Log(this.name + " Bundle with this hash is already cached!");
                }
                else
                {
                    Debug.Log(this.name + " No cached version founded for this hash..");
                }
            }
            else
            {
                // invalid loaded hash, just try loading latest bundle
                Debug.LogError("Invalid hash:" + hashString);
                yield break;
            }

        }
        else
        {
            Debug.LogError("Manifest doesn't contain string 'ManifestFileVersion': " + bundleURL + ".manifest");
            yield break;
        }

        // now download the actual bundle, with hashString parameter it uses cached version if available
        //www = UnityWebRequest.GetAssetBundle(bundleURL + "?r=" + (Random.value * 9999999), hashString, 0);
        www = UnityWebRequestAssetBundle.GetAssetBundle(bundleURL + "?r=" + (Random.value * 9999999), hashString, 0);

        // wait for load to finish
        //yield return www.Send();
        yield return www.SendWebRequest();

        if (www.error != null)
        {
            Debug.LogError("www error: " + www.error);
            www.Dispose();
            www = null;
            yield break;
        }

        // get bundle from downloadhandler
        AssetBundle bundle = ((DownloadHandlerAssetBundle)www.downloadHandler).assetBundle;

        GameObject bundlePrefab = null;

        // if no asset name is given, take the first/main asset
        if (assetName == "")
        {
            bundlePrefab = (GameObject)bundle.LoadAsset(bundle.GetAllAssetNames()[0]);
        }
        else
        { // use asset name to access inside bundle
            bundlePrefab = (GameObject)bundle.LoadAsset(assetName);
        }

        // if we got something out
        if (bundlePrefab != null)
        {

            // instantiate at 0,0,0 and without rotation
            //nstantiate(bundlePrefab, Vector3.zero, Quaternion.identity);
            GameObject abObject = (GameObject)Instantiate(bundlePrefab, gameObject.transform.position, gameObject.transform.rotation);
            abObject.transform.parent = this.transform;
            if (AssetBundleLoadedAndInstantiated != null)
            {
                AssetBundleLoadedAndInstantiated(this.name);
            }
            //SetTreeShaderSettings(abObject);

            /*
            // fix pink shaders, NOTE: not always needed..
            foreach (Renderer r in go.GetComponentsInChildren<Renderer>(includeInactive: true))
            {
                // FIXME: creates multiple materials, not good
                var material = Shader.Find(r.material.shader.name);
                r.material.shader = null;
                r.material.shader = material;
            }*/
        }

        www.Dispose();
        www = null;

        // try to cleanup memory
        Resources.UnloadUnusedAssets();
        bundle.Unload(false);
        bundle = null;
    }

Do i need to change the way ?

Yes, you are double caching (assuming you have CachedXMLHttpRequest enabled). Like I said, do not use Unity’s internal caching system as that puts all cached asset bundles on the heap (which will likely lead to memory crashes/issues). CachedXMLHttpRequest is automatic and you can verify it working by enabling logging (read the manual that comes with CachedXMLHttpRequest) or checking out your browser’s network tab (cached bundles with be like ~100 bytes instead of their normal download size).

3 Likes

Thanks for clarification, but how should i download the asset bundle ? I have updated my code to this, is this right:

 IEnumerator DownloadAndCache(string bundleURL, string assetName = "")
    {
        // get current bundle hash from server, random value added to avoid caching
        UnityWebRequest www;
            // now download the actual bundle, with hashString parameter it uses cached version if available
        //www = UnityWebRequest.GetAssetBundle(bundleURL + "?r=" + (Random.value * 9999999), hashString, 0);
        www = UnityWebRequestAssetBundle.GetAssetBundle(bundleURL);

        // wait for load to finish
        //yield return www.Send();
        yield return www.SendWebRequest();

        if (www.error != null)
        {
            Debug.LogError("www error: " + www.error);
            www.Dispose();
            www = null;
            yield break;
        }

        // get bundle from downloadhandler
        AssetBundle bundle = ((DownloadHandlerAssetBundle)www.downloadHandler).assetBundle;

        GameObject bundlePrefab = null;

        // if no asset name is given, take the first/main asset
        if (assetName == "")
        {
            bundlePrefab = (GameObject)bundle.LoadAsset(bundle.GetAllAssetNames()[0]);
        }
        else
        { // use asset name to access inside bundle
            bundlePrefab = (GameObject)bundle.LoadAsset(assetName);
        }

        // if we got something out
        if (bundlePrefab != null)
        {

            // instantiate at 0,0,0 and without rotation
            //nstantiate(bundlePrefab, Vector3.zero, Quaternion.identity);
            GameObject abObject = (GameObject)Instantiate(bundlePrefab, gameObject.transform.position, gameObject.transform.rotation);
            abObject.transform.parent = this.transform;
            if (AssetBundleLoadedAndInstantiated != null)
            {
                AssetBundleLoadedAndInstantiated(this.name);
            }
        
        }

        www.Dispose();
        www = null;

        // try to cleanup memory
        Resources.UnloadUnusedAssets();
        bundle.Unload(false);
        bundle = null;
    }

And CachedXMLHttpRequest is on by default. I have also added some settings in the module

     var gameInstance = UnityLoader.instantiate("gameContainer", "%UNITY_WEBGL_BUILD_URL%", {onProgress: UnityProgress, Module: {
//       TOTAL_MEMORY: 268435456,
     // true in order to enable caching of the initially loaded .js, .data and .mem files  
     CachedXMLHttpRequestLoader : true,    
     //Set Module.CachedXMLHttpRequestSilent to true in order to disable cache logs in the console.
     CachedXMLHttpRequestSilent : true    ,
     //   Set Module.CachedXMLHttpRequestDisable to true in order to fully disable indexedDB caching.
     //CachedXMLHttpRequestDisable : true,
 
     }});

Can you confirm that this is right way to do this job?