Downloading data with UnityWebRequest

Since a few releases ago, UnityWebRequest is available as an “experimental” feature.

Although it’s still experimental, i am looking at it as a replacement for the WWW class that we extensively use in our code.

One point for us to consider moving to this new infrastructure is performance. I was wondering if transitioning to UnityWebRequest will benefit in any way with regards to performance ?

Here are a few scenarios we’re interesting in:

  • Retrieving large JSON files from a remote server
  • Retrieving textures at runtime from file/remote server
  • Downloading and loading asset bundles from local file/remote server

Things I’d like to see possible are:

  • Memory: avoiding C# heap allocations (e.g: when accessing www.bytes or www.text). This could be done by reusing constant data buffers or even getting access to the raw native buffers somehow.
  • Performance: Faster execution (faster downloads) by a better underlying implementation of the networking code.

Are these supported? what are the current improvements of UnityWebRequest over the WWW class? (by current i mean with 5.4 or latest 5.3 versions)

1 Like
  1. At least the AssetBundles support this low memory allocation mode

“All of these operations occur in native code, eliminating the risk of expanding the managed heap. Further, this Download Handler does not keep a native-code copy of all downloaded bytes, further reducing the memory overhead of downloading an AssetBundle.”
at 3.4.4. AssetBundleDownloadHandler in Assets, Resources and AssetBundles - Unity Learn

UnityWebRequest would give you optimized texture and asset bundle loads.

However, at least today I am apt to look askance at giving access to raw buffers for www.bytes or www.text.

It’s not that I don’t like to see memory being used conservatively, quite the contrary. The feature you want simply looks like a way to invite memory access disasters. For instance, consider a case when the lifetime of the download handler is shorter than the lifetime of the buffer you want to work on. You see, upon destruction the download handler would free the memory and you will be left with a buffer whose internal pointers are wild.

For this reason www.text and www.buffer return managed copies of the internal buffer.

@Justinas thanks for your response :wink:

Regarding the case you present (data buffer lives longer than the download handler:

First of all, this will probably always be the case (according to how i see the feature).

Youwould allocate a managed array, big enough for your needs upfront, then give that to the DL handler and it should fill that up with data once its done. It can destroy the data once its done, but that means that the DLhandler was collected so nobody can access it anymore anyway.

Imagine this usage scenario (saving downloaded data to disk):

  • Allocate 4 mb byte[ ] in advance (this is enough for all current uses).
  • Construct a DownloadHandler, pass the buffer to it.
  • Once its done downloading, i can access the data, (save it to disk for example) without any GC alloc.

You could define that as a new DownloadHandler actually to avoid any confusions or “disasters” as you call them :slight_smile:

The scenario that you describe does not need any exotic download handlers. There is a DownloadHandlerScript, from which you can derive and implement your own download handler.

Your custom download handler may do these things :wink:

  • In its c-tor allocate 4 Mb byte[ ] in advance.

  • Once it’s done downloading (during downloading, too) you can access the data, save the stream to disk or whatnot.

The same technique could be applied for many other things, e.g., streamed decompression/decoding, save to file, etc., all with minimal memory and runtime overhead.

I see now that DownloadHandlerScript accepts a preallocatedBuffer parameter in its ctor.

A few other questions pop to mind:

  1. Why isn’t that an abstract class?
  2. Some methods on DownloadHandler are virtual, for example the ‘data’ property which internally calls GetData(), however GetData() is not overriden in DownloadHandlerScript to return the preallocated buffer. why is that ?
  3. If i have a preallocated buffer, how do i know what is the length of the content that was actually downloaded

It is not an abstract class for technical reasons. Native code needs to be able to bind to the hooks in the DownloadHandlerScript.

Also, I didn’t override the GetData() because in order to override this method I would need to add preallocatedBuffer to the class definition, i.e., the users that don’t use preallocatedBuffers would have to pay for something they don’t use. The current interface is a compromise, it allows you to pass the preallocated buffer and you can override GetData() to return what you want.

You need to track the length in ReceiveData. Also, before you receive your data a webserver may give you a hint what length you can expect via ReceiveContentLength, but I would advise to take this hint with a grain of salt. Webserver implementations are many, and UnityWebRequest cannot guarantee that this hint is sensible. It simply reports Content-Length back.

1 Like

In that case how can i know how much of the buffer was actually used for the download?

It would simply be the total length as received by ReceiveData then ?

Yes.

1 Like

Cool i’ll check it out.

Does UnityWebRequest currently have any other advantages over WWW (when downloading textures or asset bundles) ?

On transport level – no. Data is data and it needs to be transferred. I will note, though, that WWW is going to be deprecated at some point, so getting acquainted with UnityWebRequest might be a good idea. Besides, I think it’s nicer to work with and allows better opportunities for custom data stream handling.

If someone tries to use the UnityWebRequest class to fetch a file from a url that has double slashes “//” like the following “https://s3.amazonaws.com/zoo.misc//misc/llgames_fourpics_data-el_GR.txt” he will see that the double slashes get transformed to single slash “/” so the GET request now tries to get the file from here “https://s3.amazonaws.com/zoo.misc/misc/llgames_fourpics_data-el_GR.txt

Why does this happen??
Using the WWW class the GET request is correct (double slashes are preserved).

1 Like