Bug: 4GB limit to Textures in standalone build

[lloydsummers]

I second making the build outright fail

[mattis89]

How can I see if I have this issue with 4gb limit? I have the same screwd textures… this sucks Unity!!!

Is this the file in build ; sharedassets0.assets.resS ? its a littile over 4gb…

Does it count EVERYTHING thats in the project folder OR ONLY the textures you actually use in your scene? Can I make them smaller? they are 4k…

@karl_jones

[karl_jones]

@mattis89

How can I see if I have this issue with 4gb limit? I have the same screwd textures… this sucks Unity!!!

Is this the file in build ; sharedassets0.assets.resS ? its a littile over 4gb…

Does it count EVERYTHING thats in the project folder OR ONLY the textures you actually use in your scene? Can I make them smaller? they are 4k…

@karl_jones

If it’s over 4gb then its the same problem. Newer versions of Unity now have an error message for this. Making them less than 4k should reduce the size. Its not everything in the Projectfolder, we do not include assets that are not referenced in scenes.

[mattis89]

@karl_jones

If it’s over 4k then its the same problem. Newer versions of Unity now have an error message for this. Making them less than 4k should reduce the size. Its not everything in the Projectfolder, we do not include assets that are not referenced in scenes.

Okay thanks! where can I find more info about the Asset Bundle? I found some free one on the Asset Store… The same one Github… Is it easy to use? Just drop textures there or ? …

[MrG]

@karl_jones

If it’s over 4k then its the same problem. Newer versions of Unity now have an error message for this. Making them less than 4k should reduce the size. Its not everything in the Projectfolder, we do not include assets that are not referenced in scenes.

You probably meant 4GB there.

[karl_jones]

@mattis89

Okay thanks! where can I find more info about the Asset Bundle? I found some free one on the Asset Store… The same one Github… Is it easy to use? Just drop textures there or ? …

https://docs.unity3d.com/Manual/AssetBundlesIntro.html
https://unity3d.com/learn/tutorials/topics/scripting/assetbundles-and-assetbundle-manager

They have a learning curve but once you get up and running you will find they are not too bad.

[mattis89]

@karl_jones

https://docs.unity3d.com/Manual/AssetBundlesIntro.html
https://unity3d.com/learn/tutorials/topics/scripting/assetbundles-and-assetbundle-manager

They have a learning curve but once you get up and running you will find they are not too bad.

Thanks! one question tough… I understand that theres need to be a script so it can load the assets…

var prefab = myLoadedAssetBundle.LoadAsset<GameObject>(“MyObject”);
Instantiate(prefab);

does that code line be for EVERY prefab and texture or ?

[karl_jones]

@mattis89

Thanks! one question tough… I understand that theres need to be a script so it can load the assets…

var prefab = myLoadedAssetBundle.LoadAsset<GameObject>(“MyObject”);
Instantiate(prefab);

does that code line be for EVERY prefab and texture or ?

You dont need to load the dependencies for the prefab, just the prefab.

[mattis89]

@karl_jones

You dont need to load the dependencies for the prefab, just the prefab.

Okay thank you.

What if I have a gameobject in my scene that is a house (prefab) and I have furniture in the house and water bottle with script and food with script (pickup & use)… The house is the parent, furniture and food & water is children… I drag the house (gameobject) to the asset bundle manager window and it goes there and I name it “House”. Okay wohoo I have a asset bundle, I build and it creates the META and whatever…

I was thinking only to have houses etc as asset bundles…

BUT

When I build the game the default way, how does Unity know NOT to include the asset bundle asset & its belongings into the resource file?

Should I build the asset bundle asset to the same folder as the real game?

And the script that loads the asset should it be in the scene on some gameobject?

[mattis89]

@SoloChristian

This is a Silly issue if you consider things like SpatialOS and World Streamer. We now are building Huge worlds and this Limit needs to be removed. I just wasted 2 days on this issue until I found this post. And I question how Unity is storing Terrain Textures. Each should be loaded into memory once and reused. Since my scene has only a grid of terrains and their textures, I can tell you that it is not storing the 5 4K textures in an efficient manner…

Isnt the asset you are using for the terrain reusing textures?

[mattis89]

@ccsander

I appreciate this thread as it helped get me past the same issue. I was considering moving to AssetBundles because of this, but AssetBundles seem like too much trouble for my project.

I found what I think is a nice solution that doesn’t involve modifying much in existing setups or using the load merge functionality and find object on the merged scenes. Here is how it works:

1. keep your original >4GB resource scene with all the linked prefabs/assets/etc. intact. You won’t need to modify it for this to work.
2. create a start scene with very few linked assets in it. Most games/apps have this anyway as a splash screen. Make this scene 0 in build list when building. Add an empty gameobject to scene and attach this script on that gameobject that loads your main scene at start:

using UnityEngine;
using UnityEngine.SceneManagement;

public class SceneLoader : MonoBehaviour {

    public string sceneName;
    public bool loadOnStart;
    public bool loadAsync = true;

    public void LoadScene() {
        if (loadAsync) {
            SceneManager.LoadSceneAsync(sceneName, LoadSceneMode.Single);
        } else {
            SceneManager.LoadScene(sceneName, LoadSceneMode.Single);
        }
    }
  
    void Start () {
        if (loadOnStart) {
            LoadScene();
        }
    }
  
}

3. create a bunch of new scenes, one for each asset group you want. In these scenes, create an empty gameobject and attach this script to it:

using UnityEngine;
public class PrefabHolder : MonoBehaviour {

    public Transform[] prefabsToHold;

}

4. Now link some of your prefabs on the gameobject prefabsToHold list. Save the scene. Make as many scenes as you want to break up your prefabs/assets. The PrefabHolder is your way of organizing your prefabs/assets into different buckets. You could extend/modify the script to work with other assets besides just Transform(prefab) if you like. The key to this working is the link to resource file.
5. When building project, include the start scene 1st, followed by all your asset scenes using the PrefabHolder, and lastly have your main scene. Order is important as Unity puts assets into resource file based on scene they appear 1st in. Later scenes can reuse the assets from earlier scenes.

You will find sharedassets0.resS contains start scene assets. sharedassets1…(N-1).resS contain the assets you linked in the PrefabHolder component for each scene. Lastly sharedassetsN will contain your main scene assets that were not linked in earlier scenes.

Multiple benefits to this approach:

1. fast loading start scene
2. partition of assets how you want into sharedassets1…(N-1) which is beneficial for incremental game updates to users
3. works around the 4GB limitation on .resS file.

ccs

Does this work for things like Watersystems, ufps, vegetation studio rendering solution?

[mattis89]

Unable to open archive file: C:/Users/Unity/Documents/Asset Bundler Test/Assets/StreamingAssets/AssetBundles
UnityEngine.AssetBundle:LoadFromFile(String)
loader:Start() (at Assets/loader.cs:9)

I got that error with this code_ <<<

using UnityEngine;
using System.Collections;
using System.IO;

public class loader : MonoBehaviour
{
void Start()
{
var myLoadedAssetBundle = AssetBundle.LoadFromFile(Path.Combine(Application.streamingAssetsPath, “AssetBundles”));
if (myLoadedAssetBundle == null)
{
Debug.Log(“Failed to load AssetBundle!”);
return;
}

var prefab = myLoadedAssetBundle.LoadAsset(“Plane”);
Instantiate(prefab);

myLoadedAssetBundle.Unload(false);
}
}

[mattis89]

@ccsander

I appreciate this thread as it helped get me past the same issue. I was considering moving to AssetBundles because of this, but AssetBundles seem like too much trouble for my project.

I found what I think is a nice solution that doesn’t involve modifying much in existing setups or using the load merge functionality and find object on the merged scenes. Here is how it works:

1. keep your original >4GB resource scene with all the linked prefabs/assets/etc. intact. You won’t need to modify it for this to work.
2. create a start scene with very few linked assets in it. Most games/apps have this anyway as a splash screen. Make this scene 0 in build list when building. Add an empty gameobject to scene and attach this script on that gameobject that loads your main scene at start:

using UnityEngine;
using UnityEngine.SceneManagement;

public class SceneLoader : MonoBehaviour {

    public string sceneName;
    public bool loadOnStart;
    public bool loadAsync = true;

    public void LoadScene() {
        if (loadAsync) {
            SceneManager.LoadSceneAsync(sceneName, LoadSceneMode.Single);
        } else {
            SceneManager.LoadScene(sceneName, LoadSceneMode.Single);
        }
    }
  
    void Start () {
        if (loadOnStart) {
            LoadScene();
        }
    }
  
}

3. create a bunch of new scenes, one for each asset group you want. In these scenes, create an empty gameobject and attach this script to it:

using UnityEngine;
public class PrefabHolder : MonoBehaviour {

    public Transform[] prefabsToHold;

}

4. Now link some of your prefabs on the gameobject prefabsToHold list. Save the scene. Make as many scenes as you want to break up your prefabs/assets. The PrefabHolder is your way of organizing your prefabs/assets into different buckets. You could extend/modify the script to work with other assets besides just Transform(prefab) if you like. The key to this working is the link to resource file.
5. When building project, include the start scene 1st, followed by all your asset scenes using the PrefabHolder, and lastly have your main scene. Order is important as Unity puts assets into resource file based on scene they appear 1st in. Later scenes can reuse the assets from earlier scenes.

You will find sharedassets0.resS contains start scene assets. sharedassets1…(N-1).resS contain the assets you linked in the PrefabHolder component for each scene. Lastly sharedassetsN will contain your main scene assets that were not linked in earlier scenes.

Multiple benefits to this approach:

1. fast loading start scene
2. partition of assets how you want into sharedassets1…(N-1) which is beneficial for incremental game updates to users
3. works around the 4GB limitation on .resS file.

ccs

Should the sceneloader script be on the 0 scene (splash scene)?

[mattis89]

@karl_jones

I’m not ignoring your questions. I was waiting for a reply from the build team, this is their area.
There are no current plans to switch to 64 bit anymore, instead the idea is to move all resources like texture, audio, to separate data files so serializedfile just becomes a small header. So this will fix your issue in the future.
Yes using uncompressed assets is something commonly recommended for faster loading, its also possible to customise what gets compressed inside each bundle so you can compress the stuff that is ok to load slower and keep the important stuff uncompressed.

Any news about the resource file fix?

[mattis89]

@ccsander

I appreciate this thread as it helped get me past the same issue. I was considering moving to AssetBundles because of this, but AssetBundles seem like too much trouble for my project.

I found what I think is a nice solution that doesn’t involve modifying much in existing setups or using the load merge functionality and find object on the merged scenes. Here is how it works:

1. keep your original >4GB resource scene with all the linked prefabs/assets/etc. intact. You won’t need to modify it for this to work.
2. create a start scene with very few linked assets in it. Most games/apps have this anyway as a splash screen. Make this scene 0 in build list when building. Add an empty gameobject to scene and attach this script on that gameobject that loads your main scene at start:

using UnityEngine;
using UnityEngine.SceneManagement;

public class SceneLoader : MonoBehaviour {

    public string sceneName;
    public bool loadOnStart;
    public bool loadAsync = true;

    public void LoadScene() {
        if (loadAsync) {
            SceneManager.LoadSceneAsync(sceneName, LoadSceneMode.Single);
        } else {
            SceneManager.LoadScene(sceneName, LoadSceneMode.Single);
        }
    }
  
    void Start () {
        if (loadOnStart) {
            LoadScene();
        }
    }
  
}

3. create a bunch of new scenes, one for each asset group you want. In these scenes, create an empty gameobject and attach this script to it:

using UnityEngine;
public class PrefabHolder : MonoBehaviour {

    public Transform[] prefabsToHold;

}

4. Now link some of your prefabs on the gameobject prefabsToHold list. Save the scene. Make as many scenes as you want to break up your prefabs/assets. The PrefabHolder is your way of organizing your prefabs/assets into different buckets. You could extend/modify the script to work with other assets besides just Transform(prefab) if you like. The key to this working is the link to resource file.
5. When building project, include the start scene 1st, followed by all your asset scenes using the PrefabHolder, and lastly have your main scene. Order is important as Unity puts assets into resource file based on scene they appear 1st in. Later scenes can reuse the assets from earlier scenes.

You will find sharedassets0.resS contains start scene assets. sharedassets1…(N-1).resS contain the assets you linked in the PrefabHolder component for each scene. Lastly sharedassetsN will contain your main scene assets that were not linked in earlier scenes.

Multiple benefits to this approach:

1. fast loading start scene
2. partition of assets how you want into sharedassets1…(N-1) which is beneficial for incremental game updates to users
3. works around the 4GB limitation on .resS file.

ccs

Aaah now I get it! This is clever! Much better than the assetbundle if you allready have big project do you know how to instead of prefabs lets say use it for textures?

Is it okay if I make a youtube tutorial about this?

[exltus]

This limitation is really bad news for me. I am working on game where I have a lot of different cloths and attachements for player (Almost 6 bg of different meshes + HD textures). Problem is that having that many assets means, that is impossible to have all of them referenced by some kind of config with standard unity references, because if I make references like this, than all of this assets will be stored in RAM memory on device. So I really need to use resources or something like this. Or is there any other way how to reference some kind of “database” of prefabs without loading all of them to memory? Or how to deal with this situation?

[truefx001]

@karl_jones

I’m not ignoring your questions. I was waiting for a reply from the build team, this is their area.
There are no current plans to switch to 64 bit anymore, instead the idea is to move all resources like texture, audio, to separate data files so serializedfile just becomes a small header. So this will fix your issue in the future.

Any ETA on this this needs to be fixed asap without developers having to add work around, this issue is long known and should be fixed by now. For a true 64bit engine there should be no 32bit limitations plain and simple. A Unity dev should respond to this please. We are experiencing this issue with 2017.1.0

[karl_jones]

@exltus

This limitation is really bad news for me. I am working on game where I have a lot of different cloths and attachements for player (Almost 6 bg of different meshes + HD textures). Problem is that having that many assets means, that is impossible to have all of them referenced by some kind of config with standard unity references, because if I make references like this, than all of this assets will be stored in RAM memory on device. So I really need to use resources or something like this. Or is there any other way how to reference some kind of “database” of prefabs without loading all of them to memory? Or how to deal with this situation?

That sounds like using asset bundles would be much better. Reference an asset bundle instead of an asset.

[karl_jones]

I

@truefx001

Any ETA on this this needs to be fixed asap without developers having to add work around, this issue is long known and should be fixed by now. For a true 64bit engine there should be no 32bit limitations plain and simple. A Unity dev should respond to this please. We are experiencing this issue with 2017.1.0

Its not something that will be fixed like a bug. Its a feature that will improve the situation in the future but it won’t be backported like a bug fix. I know it’s frustrating but we can’t just change a core element like that, it would break a lot of stuff. I’ll ask the team for an update.

Update:
Fixing this issue is a priority for the build team. The current plan is to address the serialized files and upgrade them so they can be 64 bit. Although this is a lot more complex than it sounds as we need to go throughout the entire engine and update any areas where we expect 32 bit and then be able to handle upgrading of old projects and assets. We can’t provide a version when this will be ready at this stage but will try and provide an update later in the year after 2018.2 is released.

We also have some feature in 2018.2 which allow for better management of resources and loading assets so keep an eye out for them.

[truefx001]

@karl_jones

I

Its not something that will be fixed like a bug. Its a feature that will improve the situation in the future but it won’t be backported like a bug fix. I know it’s frustrating but we can’t just change a core element like that, it would break a lot of stuff. I’ll ask the team for an update.

I’m fine with not being backported like a bug fix, but the feature has to improve the situation now, I mean look how old this thread is since 2016! We are in the era of 4K , I’m actually shocked that this limitation is something you guy’s have not fixed or commented in your doc’s or even stated as a warning message in older versions by older I mean at least Unity 5 and Unity 2017. We lost couple of day’s taking everything apart until we found this post here, that’s unacceptable for a subscription based Engine. Please let us know when this feature will be implemented ( In my opinion that’s not a future feature but something a 64bit engine should off had from the start)