[RELEASED] [Free] Better Streaming Assets

[gwiazdorrr1]

Hi guys!

TL;DR: Read Streaming Assets 10 times faster on Android for free, out on Asset Store (Unity Asset Store - The Best Assets for Game Making). Works on other platforms, too.

Some time ago I came up with a way to read Streaming Assets directly (i.e. without WWW) on Android. APK/OBB are basically ZIP archives and since Streaming Assets, while being archived, remain uncompressed, they can be read - you just need an offset from the beginning of APK/OBB and size.

The difference is huge. Direct read on Sony D5503 is ~10 times faster than WWW, regardless of file size. Also, because WWW.bytes property effectively clones internal data (leading to memory spikes in case of large files), direct read is infinitely more memory efficient It can be also used in any thread, works with streams etc.

Finally, to make things nice and pretty I created cross-platform API, that internally falls back to System.IO.File etc. for other platforms. Examples below.

The plug-in is out on Asset Store free and waiting for feedback!

[caletbak2]

Hi,

It seems a very useful plugin.

When the file is stored in the OBB this is something transparent for your plugin? So a file stored in StreamingAssets can be easily found when building a full APK without OBB, and can be found when building with OBB by using the same file path?

Cheers,
Calet

[gwiazdorrr1]

Yes. If you look at the source, it uses Application.dataPath to get the path to APK. According to Unity documentation:

Android: Normally it would point directly to the APK. The exception is if you are running a split binary build in which case it points to the the OBB instead.

Streaming Assets get put in OBB, so it should be fine. Just make sure OBB exists before you call BetterStreamingAssets.Initialize. If it doesn’t, Application.dataPath will point to APK and no Streaming Assets will be discovered.

[capu087]

does it work on web gl ?

[gwiazdorrr1]

No. AFAIK WebGL doesn’t allow synchronous file access.

[Blaze5565]

Hi,

I’m having some trouble with your asset? I’ve got some files in my StreamingAssets folder:

And I’m using the BetterStreamingAssets.GetFiles() method to try to get the names of these 3 files (all of which are .json files):

But the end result only seems to have 2 files:

I tried this earlier as well, with just 2 files, and only one of them worked (the one with the Japanese name, specifically).
Could you explain what might be going on or give me a hand?

Thanks,
Cory

[gwiazdorrr1]

Are you by any chance using Android with App Bundle build?

If so, see this issue: Asset Bundle, not finding any files · Issue #10 · gwiazdorrr/BetterStreamingAssets · GitHub. Basically sometimes some Streaming Assets get compressed - I couldn’t find a pattern and am waiting for Unity to fix it.

Also, what Unity version are you on? Are you able to reproduce it reliably or does it change between builds?

EDIT: here’s a thread about what I’m talking about: Asset Bundles in StreamingAssets directory can load slower when building the Android App Bundle
A Unity bug report: Unity Issue Tracker - [Android] Loading assets from AssetBundles takes significantly more time when the project is built as an AAB
An interesting take: StreamingAssets files are compressed in APK when "Build App Bundle (Google Play)" option is used

So, since you already have the files in the root of StreamingAssets, try to change their name to just lowercase and see if it works.

[Blaze5565]

@gwiazdorrr_1

So, since you already have the files in the root of StreamingAssets, try to change their name to just lowercase and see if it works.

I was using app bundles, yes, and I changed the filenames to lowercase, and now they’re showing up! I’m using Unity 2019.2.8f1, and the missing files change only when I change what files are in StreamingAssets (as in even just adding new files without touching the old ones). Maybe helpful (or at least informative), the icon in Unity also changed when I removed the uppercase:

Although, all four worked correctly this time despite the icon being different in one from the other now-working ones. Originally, the only one that worked was the last one, which has always had this icon (and always worked) (oh, and, I had deleted it when I originally posted the question to see if the icon meant anything). Anyway, yes, everything seems to be working now, so thank you!

[Jovaan]

Because of our app size (>250MB with videos), we need to still use APK + OBB to launch in Play Store. I am facing a problem now with only builds loaded from Play store, apk + obb. Trying to use BetterStreamingAssets.GetFiles() results in IndexOutOfRangeException:

All the files are in root of StreamingAssets, non-capital letters. They are video files with names formatted like “xxx-yy.mp4”.

Works with full APK build, not APK + OBB, no other changes.

[gwiazdorrr1]

Interesting.

1. Do you call Initialize after OBB is mounted? This is needed if say you download OBB from Google Play yourself.
2. I’m going to need the arguments for your GetFiles() call and list of files in StreamingAssets to reproduce it. You can PM me.

[Jovaan]

@gwiazdorrr_1

Interesting.

1. Do you call Initialize after OBB is mounted? This is needed if say you download OBB from Google Play yourself.
2. I’m going to need the arguments for your GetFiles() call and list of files in StreamingAssets to reproduce it. You can PM me.

1. I call Initialize in Awake of the first (and only) scene the app has. The error manifests in first and all later starts of the app, so I am assuming the OBB must be loaded. AFAIK there is no direct way to check in Unity if OBB is actually loaded. I might be wrong.
2. Arguments are simple: BetterStreamingAssets.GetFiles(Path.DirectorySeparatorChar); so I am reading the names of all files in StreamingAssets root folder. All files are non-capital letters .mp4 files.

I have tried including Write permissions to External also, but no change in this behaviour. Exactly same crash.

[gwiazdorrr1]

Ok I finally found the time to fix this. Basically in OBB directories end up as zip entries, in APK - they do not. The fix is on github (GitHub - gwiazdorrr/BetterStreamingAssets: A plugin for Unity that lets you access Streaming Assets directly on Android.), it will take a while for it to make it into Asset Store.

EDIT: The fix is already in the Asset Store.

[xucian]

Thanks man! This is exactly what I was looking for! What a relief!

I’m wondering, could you make it so that we can list the directories in a directory, not just files?

– Lucian

[gwiazdorrr1]

Simply put I’ve never needed that It’s not totally trivial to do it the right and robust way, but if you can live something quick and dirty, this should work:

var directories = BetterStreamingAssets.GetFiles(path, "*", SearchOption.AllDirectories)
    .Select(x => Path.GetDirectoryName(x))
    .Distinct()
    .ToList();

Note that this is not going to work with empty directories.

[idioti]

Hi, why didn’t you do the serialization, and the Save part also? Does it still not work on webGL?

[gwiazdorrr1]

Well, for instance on Android StreamingAssets are read only (embedded in APK). Also AFAIK altering Streaming Assets on platforms that let you (PC?) is a bad practise.

As for webGL, there’s no way to access a file synchronously there, everything needs to be a web request.

[xucian]

@gwiazdorrr_1

Simply put I’ve never needed that It’s not totally trivial to do it the right and robust way, but if you can live something quick and dirty, this should work:

var directories = BetterStreamingAssets.GetFiles(path, "*", SearchOption.AllDirectories)
    .Select(x => Path.GetDirectoryName(x))
    .Distinct()
    .ToList();

Note that this is not going to work with empty directories.

Didn’t think of that. It’s hacky, yes. But maybe it’ll be my last resort at some point (I figured another way ATM for my particular case).
Thanks. And maybe in the future, if possible, you’ll find a simpler way and post it here.

[musicdeveloper]

Thank you for this asset!

I’m wondering if there’s any way to include multiple extensions in the search pattern? For instance, when reading through a folder of audio assets, I want to include “.wav" ".ogg” “*.mp3”

(I tried “.wav|.mp3”, which was one suggestion from the internet and that didn’t work…)

Thank you

[gwiazdorrr1]

It is possible, but that’d mean compatibility with System.IO.Directory.GetFiles is broken - something I want to avoid, as the plug-in is meant to be an almost drop-in replacement.

[nyonge]

Looking forward to using this asset A couple questions. The website seems to be down, I don’t know if docs were there or not but the Github docs are pretty limited.

Can you explain the difference between Initialize and InitializeWithExternalDirectories? I plan on reading files of a bunch of different types (meshes and textures, even unity stuff like materials and physicmaterials) from StreamingAssets and want to make sure I’m using the right initialization.

The Asset Store page says it’s cross-platform, can you clarify which platforms? Clearly Android, likely PC, I suspect others…?

Also, the “message me” link on the Asset Store page is broken too, it leads to a 404 instead of opening up a conversation with you.

Thank you!