There appears to be a hard limit of 4GB of textures in the standalone build. If your project has more than 4GB of textures, then some of the textures in standalone build will be messed up. While this limitation may be acceptable for mobile projects, this limit is completely unacceptable for PC projects.
There are currently two threads in the forum about this bug:
Can somebody from Unity please comment on this bug? Thanks.
This is a limit of the maximum file size for resources. Resources are packed into a single file during the build. The file contains a header and one of the fields inside the header is size, this is stored as a 32 bit unsigned integer, the limit of which is 4GB. There has been some research into changing the size property to a 64bit unsigned integer but it is not a simple task due to backwards compatibility.
I suggest you look at using AssetBundles as an alternative, they are a much better way to work with large amounts of assets.
Thank you for looking into this issue and commenting on this topic. As a short term workaround, I will look into using AssetBundles. I thought AssetBundles were for games that needed a way to download additional content from a server, which is a different use case from this.
Big picture wise, there are several possible long term solutions for this issue:
Unity could add an optional resource file format that supports a 64bit unsigned integer for supporting projects with more than 4GB of resources. Unity could leave the existing 32 bit file solution in place for most use cases, and only use the 64 bit file format when the total resources exceeded 4GB. This would allow Unity to properly support large projects on PC without possibly breaking or slowing projects on mobile platforms. This could be similar to how Microsoft simultaneously offered multiple disk formats (FAT16, FAT32, and NTFS) during the transition.
Unity could implement splitting the large resources file into multiple smaller files using AssetBundles without forcing the developers to manually set up AssetBundles. This would offer a solution that provided support for large projects without breaking the existing file format.
Unity could add a pop up message in the Unity Editor during builds to let developers know their project is too big to fit into a 4GB resource file, and then include a link to a web page about the issue. At a minimum, Unity should immediately do this. Every one of us that ran into this problem has wasted literally days trying everything to fix the problem. Showing a detailed message at build time would have saved us all a lot of time.
What is not acceptable is letting developers run into a hidden limitation and then waste days of development trying to figure out why textures in large builds are suddenly broken. I get the concern about backwards compatibility, but that is not a good enough reason to let developers run into this problem. At a minimum, pop up a warning explaining the issue.
Also, leaving this 32 bit file size limitation in Unity is not really a viable long term solution. A lot of developers already think of Unity as a mobile-centric game engine. The only reason PC and console developers are not complaining more about this issue is because most people don’t realize this limit/bug exists.
Hi,
AssetBundles are perfectly fine for local use and are what we consider the correct way to handle the use case you are describing. We really dont recommend using Resources for large amounts of textures. Have a read of this: Unity Connect
Then read about AssetBundles more here Unity Connect
I am not using the Resources folder at all. I understand the Resources folder is not the proper solution. I link to my prefabs from my C# code through public GameObject variables and let Unity manage which resources get included in the build.
That is the frustrating part of this. I am expecting Unity to be able to manage the build properly. As a game developer, I should be able to expect Unity to handle the build properly. In this case, AssetBundles generates more work for me to handle something that Unity should be able to handle automatically.
It should just work fine with > 4GB if not using resources folder. Perhaps file a bug and link case here?
Then it sounds like you have a different problem. Are you using a single scene? Its worth filing a bug report so we can investigate
Is this your bug report? Unity Issue Tracker - [Standalone] Corrupted textures in a huge project.
That is not my bug report. I submitted a bug report, but received an email that my bug report was a duplicate of that bug report.
My project is a single scene that I dynamically load everything into at runtime.
My Resources folder only contains an empty folder called Prefabs. There is nothing else in my Resources folder. Should I delete the Resources folder since I am not using it?
The 4gb limit applies to all our files so I suspect that your scene file may be too large? What are the sizes of the generated files, is the scene 4gb?
My scene file is only 13KB. The scene itself is mostly empty. The scene is just a handful of GameObjects that launch all of my prefabs dynamically.
For example, one of my GameObjects contains all of my object pool managers, which is small itself but loads thousands of projectiles into the scene. Similarly, I have another GameObject that manages all of the procedurally generated space scenes, which is another small object that launches a bunch of larger objects.
Are you saying the scene can only contain 4GB of static content, or are you saying the scene cannot dynamically load more than 4GB?
Are you building a 64bit executable of the game or the default x86? What does the contents of the built project look like, file sizes?
I am building a 64bit executable.
I did some additional experimenting with this problem today. I opened my backup from Friday, November 11. That is a point in my project before I purged enough textures to get the total build size under 4GB. Then I deleted the empty Resources folder, just to be sure that was not an issue. Then I built a Windows 64bit standalone of that project.
The messed up textures problem was there, just like in the following screenshot:
The projectname_Data folder contains the following files:
app.info (1KB)
globalgamemanagers (179KB)
globalgamemanagers.assets (73KB)
level0 (1.91MB)
resources.assets (1.91MB)
resources.assets.resS (60.2MB)
sharedassets0.assets (834MB)
sharedassets0.assets.resS (4.54GB)
sharedassets0.resource (143MB)
I believe that referencing over 4gb of assets in one scene is the problem. I need to confer with some of the multi scene devs why this is, i thought the shared asset files were split. A workaround that we have used for our own projects is to split the scene into multiple scenes(so that one scene does not reference over 4gb of assets) and then load all of those scenes additivly. The multi scene system is very good for this so it should be something you can do quite quickly.
Work arounds like that make it sound like Unity is only somewhat 64bit.
You have mentioned both AssetBundles and Multi-Scene as possible work arounds for this bug/limitation. I guess I will need to test both to see what helps.
I re-organized my project to use an AssetBundle. I am getting a crash when trying to use the AssetBundle that I generated. My AssetBundle is just over 3GB. What is the max size for an AssetBundle?
I gave up on AssetBundles, since there seems to be a limit on size of an AssetBundle. I refactored my project to use Multi Scene instead of AssetBundles, and that seems to be working. The next thing I will do is add a bunch of texture back into the project to push the total size up again to see if I can avoid the texture corruption bug as long as I do not reference more than 4GB of content per scene.
Multi Scene seems to be the ticket. I added a couple gigabytes of content to the scene, and retested. My total size was 5GB and I did not see any texture corruption in the Windows standalone build now.
What I did was move my SpaceSceneCreator GameObject to another scene called SpaceSceneAdditive. Then one of the classes in my main scene calls SceneManager.LoadScene (with LoadSceneMode.Additive) from its own Awake function, and then uses GameObject.Find from the Start function to locate that object.
My SpaceSceneCreator GameObject has a script that has references to about 3GB of content that I use to procedurally generate the background space scenes at runtime. I render the generated space scenes to a different camera, and then I bake that camera to a skybox cubemap. Then I use the 96MB material from that cubemap to use as the skybox on the main camera in the Main scene. So even though I have 3GB of space scene contents referenced, I only actually use the 96MB skybox material during actual gameplay. Anyway, this is all working great now.
Anyway, if anybody runs into a similar problem in their own projects with texture corruption on scenes with a lot of content, the Multi Scene option in Unity will work to get around the bug/limitation.
Now the big question is why that limit exists at all within Unity?
That’s good. As for asset bundles I didn’t expect you to try and put it all into one asset bundle ;), generally they are used for one asset per bundle so you can load assets on demand. For your use case the multi scene is ideal, the limit is related to the serialised file format we use which as I explained earlier we do plan to improve in the future. By breaking it into multiple scenes you split the assets into multiple files.
Thanks for your help with this, Karl. Multi scene is working great. I should be able to complete my current project using that as a work around.
