LZ4 Compression for assets not in asset bundles?

Will we be able to utilize the new LZ4 compression for assets that are included in the game itself and not in asset bundles? If not, why not?

One platform where this would be especially useful is iOS. Even though apps are compressed when downloaded from the App Store, iOS has to decompress the app before it can be used. This means that uncompressed assets will take a lot of disk space, which is often very limited on low-end devices.

For example, a single 2048x2048 texture encoded with PVRTC4 takes about 2MB uncompressed, which is how much disk space it would currently use if the game is deployed on an iOS device. If this texture was further compressed with some other compression algorithm (such as LZ4), it would obviously take much less space. (Maybe as little as 50kB - 300kB in most cases.)

An even worse example are 32-bit truecolor textures. A single 2048x2048 texture takes about 16MB of disk space, which makes them almost unusable on iOS in current state. Compressed, the same texture would in most cases take only a fraction of the space.

Hi!

[quote=“Wiliz”, post:1, topic: 599784]
Will we be able to utilize the new LZ4 compression for assets that are included in the game itself and not in asset bundles? If not, why not?
[/quote]

That will happen, but we can’t guarantee if this will be available in 5.3 for all platforms, as this feature requires more time for testing and per-platform tuning (e.g. android with obb files).

Right now you can use LZ4 compressed asset bundles and deploy them with your game using StreamingAssets approach.
This might be better solution at least because of:

  • incremental data builds,
  • ability to explicitly unload unused assets and reduce assets GC time.

And it requires minimal changes into a resource loading pipeline (if you use Resources.Load*), see e.g. https://unity3d.com/learn/tutorials/topics/scripting/assetbundles-and-assetbundle-manager

Player’s data compression will be supported for WebGL in a later 5.3 beta builds, other platforms will be included later.
Also take in mind that player’s data compression might affect final installer package size.

Thank you for the response!

I also realized while writing my post that you could probably achieve pretty much the same thing by putting assets into asset bundles that you include in the game. It is a workable approach, but still adds some unnecessary complexity in cases where you don't need the asset bundles for other reasons.

[quote=“alexeyzakharov”, post:2, topic: 599784]
Hi!
That will happen, but we can’t guarantee if this will be available in 5.3 for all platforms, as this feature requires more time for testing and per-platform tuning (e.g. android with obb files).
[/quote]
Is this still an accurate prediction, i.e. the compression won’t necessarily be available for non-bundled assets in 5.3? I’m not currently using asset bundles, but am doing an Android/iOS game that is most likely going to be released with 5.3, and compressed assets would definitely come in handy. Especially since Apple and others still release 16GB devices even though it’s 2015!

I could probably switch to using asset bundles, but it’d be just simpler to let Unity compress and decompress them for me automagically. :slight_smile:

[quote=“alexeyzakharov”, post:2, topic: 599784]
Right now you can use LZ4 compressed asset bundles and deploy them with your game using StreamingAssets approach.
[/quote]

Will we be able to open LZ4 compressed bundles using AssetBundle.CreateFromFile?

If the ProceduralMaterial texture cache was LZ4 compressed along with the bundles, I would be very, very happy :smile: You would be surprised how much a DXTC/PVRTC/ETC texture will still compress losslessly.

[quote=“Wiliz”, post:1, topic: 599784]
Will we be able to utilize the new LZ4 compression for assets that are included in the game itself and not in asset bundles? If not, why not?

One platform where this would be especially useful is iOS. Even though apps are compressed when downloaded from the App Store, iOS has to decompress the app before it can be used. This means that uncompressed assets will take a lot of disk space, which is often very limited on low-end devices.

For example, a single 2048x2048 texture encoded with PVRTC4 takes about 2MB uncompressed, which is how much disk space it would currently use if the game is deployed on an iOS device. If this texture was further compressed with some other compression algorithm (such as LZ4), it would obviously take much less space. (Maybe as little as 50kB - 300kB in most cases.)

An even worse example are 32-bit truecolor textures. A single 2048x2048 texture takes about 16MB of disk space, which makes them almost unusable on iOS in current state. Compressed, the same texture would in most cases take only a fraction of the space.
[/quote]

Hi. I added recently LZ4 support in my compression/decompression multiplatform plugin.

http://forum.unity3d.com/threads/7zip-lzma-lz4-fastlz-zip-native-multiplatform-plugins.211273/

Hi!

[quote=“mh114”, post:4, topic: 599784]
Is this still an accurate prediction, i.e. the compression won’t necessarily be available for non-bundled assets in 5.3? I’m not currently using asset bundles, but am doing an Android/iOS game that is most likely going to be released with 5.3, and compressed assets would definitely come in handy. Especially since Apple and others still release 16GB devices even though it’s 2015!
[/quote]

Yeah, only WebGL will use Resources folder LZ4HC compression in 5.3 (by default). Support for other platforms is planned, but not for 5.3.
You can still use AssetBundles approach to achive the same results.

[quote=“PixelParts”, post:5, topic: 599784]
Will we be able to open LZ4 compressed bundles using AssetBundle.CreateFromFile?
[/quote]

Yes, sure. AssetBundle.LoadFromFile[Async] will support asset bundles of all types in 5.3.
Please, take a look the overview table here https://docs.google.com/document/d/1XZSLTY54w8hfrU4t5G8FRS8C1jlAWVoXKjDEHJQs6a8/edit#heading=h.98sfunf5yht3
LZ4 compressed bundle has no decompression overhead during opening, LZMA will be decompressed to memory.

[quote=“alexeyzakharov”, post:8, topic: 599784]
Yeah, only WebGL will use Resources folder LZ4HC compression in 5.3 (by default). Support for other platforms is planned, but not for 5.3.
You can still use AssetBundles approach to achive the same results.
[/quote]

Alright. You might want to rephrase the roadmap text though: “Introduces realtime LZ4 decompression for cached AssetBundles and Player’s Data folder.” To me, it implies that all assets will be compressed, asset bundle or not. That would be a nice thing to have later on, if not in 5.3.

Thanks! We changed the text on Roadmap page.
In 5.3 it will be possible to use LZ4 compression only with AssetBundles (including the bundles shipped within a game in StreamingAssets folder)

I've been holding back on releasing the iOS version of my 2d game because I've been waiting for something to get released that addresses the huge install size on iOS due to uncompressed textures (in my case, 16bit rgba.)

I don't believe I can use AssetBundles since I'm using 2dToolkit. I was counting on Unity 5.3 being able to compress the player data folder, but that seems to not be happening now...at least for iOS.

Anything on the horizon coming that could help me get the install size down? Maybe iOS crunch textures (if that's a possible thing) ? A projection date on iOS player data getting compressed?