I appreciate this thread as it helped get me past the same issue. I was considering moving to AssetBundles because of this, but AssetBundles seem like too much trouble for my project.
I found what I think is a nice solution that doesn’t involve modifying much in existing setups or using the load merge functionality and find object on the merged scenes. Here is how it works:
keep your original >4GB resource scene with all the linked prefabs/assets/etc. intact. You won’t need to modify it for this to work.
create a start scene with very few linked assets in it. Most games/apps have this anyway as a splash screen. Make this scene 0 in build list when building. Add an empty gameobject to scene and attach this script on that gameobject that loads your main scene at start:
using UnityEngine;
using UnityEngine.SceneManagement;
public class SceneLoader : MonoBehaviour {
public string sceneName;
public bool loadOnStart;
public bool loadAsync = true;
public void LoadScene() {
if (loadAsync) {
SceneManager.LoadSceneAsync(sceneName, LoadSceneMode.Single);
} else {
SceneManager.LoadScene(sceneName, LoadSceneMode.Single);
}
}
void Start () {
if (loadOnStart) {
LoadScene();
}
}
}
create a bunch of new scenes, one for each asset group you want. In these scenes, create an empty gameobject and attach this script to it:
using UnityEngine;
public class PrefabHolder : MonoBehaviour {
public Transform[] prefabsToHold;
}
Now link some of your prefabs on the gameobject prefabsToHold list. Save the scene. Make as many scenes as you want to break up your prefabs/assets. The PrefabHolder is your way of organizing your prefabs/assets into different buckets. You could extend/modify the script to work with other assets besides just Transform(prefab) if you like. The key to this working is the link to resource file.
When building project, include the start scene 1st, followed by all your asset scenes using the PrefabHolder, and lastly have your main scene. Order is important as Unity puts assets into resource file based on scene they appear 1st in. Later scenes can reuse the assets from earlier scenes.
You will find sharedassets0.resS contains start scene assets. sharedassets1…(N-1).resS contain the assets you linked in the PrefabHolder component for each scene. Lastly sharedassetsN will contain your main scene assets that were not linked in earlier scenes.
Multiple benefits to this approach:
fast loading start scene
partition of assets how you want into sharedassets1…(N-1) which is beneficial for incremental game updates to users
The 4GB limit is an unprofessional kindergarden joke!
I’m expecting the unity devs should update the 32bit serialized file format to 64bit in unity 2017 in the first place.
I agree. It was shocking to run into this silly limitation, and it was not obvious what the problem was when I ran into it. The textures break in the builds, but the Unity editor never tells what the real problem is. Having a hidden 32bit limitation in a 64bit build is crazy, especially for a product in 2016.
The multi-scene workaround is effective enough for a short term solution. I am still having good results using multi-scene.
wow… if I only knew about this before… Do you have any idea how much time I would have saved?
What a terrible system in place to cause such a nightmare for me, and months set backs due to a Unity bug?
Is there clearly stated ANYWHERE in the documentation?
I am seriously really disappointed in this, Sorry Unity - not everywhere who uses Unity is building mobile games.
Wait what?? This issue is still undocumented? Unity get your act together. At least document your god damn limitations for once. You do such perfect job trying to constatnly sell us your subsrciption services but having a mention about the 4gb build in a documentation seem like an impossible task.
I remember when i encountered this issue, i was lucky to get a solution in few hours since a friend pointed me to this thread. It took days for him to figure out this too.
did you know, that the limitation count on compressed textures only?
Or is the limitation count calculating all original files sizes are used as textures eg. a .psd with 1024x1024 and 25mb?
And does all resourses folders are calculated into a big one?
What is the difference between a resources data file and asset-bundle folder in build?
How can existing used references exceed 4gb because these data structure should be also go into any indexed file like asset-bundles and resource folder files?
Following example scenario:
A building that contains ~40 Rooms. Each of the rooms contain textures and meshes from another room as well, but 50% of each room has it’s own unique data. Loading all data contains 16GB, so not possible to load all the stuff at once. Each room can be entered at any time.
For sure while entering a room the data should be loaded. But what is the preferred method to do in order to load the required stuff as fast as possible without hiccups and extra ordinary academic work.
Storing the rooms in one scene load/unload-async and let unity to handle all the references?
Using asset bundles, build a info structure contains what asset bundles to load/unload?
And is it the best way to asset-bundle.unload(true) because to hold all the material references in memory?
But wait…
If a bundle contain a texture must be loaded and the material already exist as loaded bundle, then the references doesn’t work (as the documentation say), so I have to unload the asset-bundle.unload(false) with the existing material, load the asset-bundle containing the texture and reload the previous bundle with the corresponding material again?
But wait again…
What happen if there any references from new load bundles includes the Material are broken during last bundle.reload? In this case I must recursively search through all broken links and reload all existing bundles contain textures… then reloading these ones with materials again?
So if you suggesting me to use bundles, you must be assured and be able to explain exactly how to use these in this case.
Im not sure I understand your question. The resources file can not exceed 4gb. You should use asset bundles. You do not need to store everything in a single asset bundle, thats a bad way to go about things.
If asset bundles are the recommended way to use for streaming too, why not going the logical way and build a folder name > AssetBundles in the project window.
Every additional child folder acts like a real single asset bundle itself. Further, managing the asset bundles can be done by the project manager as well, like the asset bundle browser tool does currently.
The AssetBundlesBrowser can rebuild as an AssetBundles::ServerFileManager where the AssetBundle-files have to be located or save/loaded from 1) http server 2) local host system 3) wlan network 4) etc.
Then the whole asset bundle system make sense in the workflow of unity.
We REALLY need you to make this change and allow >4gb in the resources file. Karl, you are suggesting using AssetBundles so I took off running with your idea… I was all excited for a few days, only to discover that loading my mobs from StreamingAssets takes WAY TOO LONG (60+ seconds to start my game), whereas when the same assets were in “resources.assets.resS”, the game loaded in just a couple seconds.
I’m having to do something really unconventional (having as many as possible in resources, and the overflow as AssetBundles). This is not acceptable. As I add more to my game, it’s going to take longer and longer to load, even in the editor.
P.S. loading just the asset I need at the time I need it was not an option. This is an MMORPG game and even loading one asset (a monster) took long enough that it could kill a player while its assets are loading.
Either need to make that pointer 64 bit (preferred option) or do something about the speed of AssetBundle loading. How can that huge 4gb resources.assets.resS file be loaded (or accessed) so quickly? Unity needs to do the same for external asset files.
This is really unprofessional and shortsighted of Unity to let this pointer-size issue just ride. Since when did we have 32-bit file systems with a 4gb file size limit? FAT32? How long ago was that? To add insult to injury, they ignore my questions. Well, if anyone is interested in knowing my progress, I discovered something that may help you. If you took Karl’s suggestion and switched to using AssetBundles and you’re experiencing very slow asset loading time, I found that you can save the assets as UNCOMPRESSED and they’ll load much quicker. Of course, this takes up a lot more disk space but it works for now.
I’m not ignoring your questions. I was waiting for a reply from the build team, this is their area.
There are no current plans to switch to 64 bit anymore, instead the idea is to move all resources like texture, audio, to separate data files so serializedfile just becomes a small header. So this will fix your issue in the future.
Yes using uncompressed assets is something commonly recommended for faster loading, its also possible to customise what gets compressed inside each bundle so you can compress the stuff that is ok to load slower and keep the important stuff uncompressed.
Given that Unity seems to be making a push to attract more cinematic quality projects(which would subsequently require larger and larger data files) with their tech demos, it seems to me odd that a user should have to jump through more hoops just to work around a limitation such as this one.
Here is a better question. Is there an ETA for documentation where you point out the limitation? More and more people are going to run on this issue and the answer is nowhere to be found. I have no problem with the issue that there are and will be technical limitations due backwards compatibility and etc… (i am doing the workarounds pointed in this thread), but what i find unacceptable is that UT is unable to document this information properly.
This rant is not directed to you dev’s. I understand that this is a large organisation and it is not your job to keep up the documentation. It is just very frustrating that UT finds a time to constantly spam me with their useless marketing fluff about services that do not help me in any way to make better games. But at the same time pointing out some very important issues are going unnoticed for years. People running on this issue are wasting days of their dev time until they accidentally find this thread by an abstract “my standalone build textures are messsed up” google search.
That is the exact problem with the current situation. When I ran into this problem, I literally wasted days trying to figure out what was wrong. Each person that ran into this problem would have loved a message in the Unity Editor to let us know what was going on.
If Unity generates a resource file larger than 4GB, the application will not work (properly).
If adding true 64bit resources support isn’t simple, perhaps you can make the build fail instead (as a short-time solution)? That way we don’t even have to run the game to figure out it isn’t working and don’t have to spend days trying to find the cause.
If Unity detects a resource file larger than 4GB while building, it could fail with an appropriate error message, like:
This is a Silly issue if you consider things like SpatialOS and World Streamer. We now are building Huge worlds and this Limit needs to be removed. I just wasted 2 days on this issue until I found this post. And I question how Unity is storing Terrain Textures. Each should be loaded into memory once and reused. Since my scene has only a grid of terrains and their textures, I can tell you that it is not storing the 5 4K textures in an efficient manner…
