Why does accessing www.text blocks the coroutine using WWW class?

Questions & Answers
1

I have written the following co-routine that send a JSON post request to webservice that also return a JSON object which can be 30-40MB large containing some data. This JSON object contains a base64 string which can be decoded into binary data of any type. After decoding, I want to saved this binary data as a file on the local machine. I want to make the whole process as asynchronous as possible and so far the download progress from WWW.progress is working fine but I encounter a strange block when accessing WWW.text:

IEnumerator WWWJSONPost(string url, string jsonString)
	{		
				
		var encoding = new System.Text.ASCIIEncoding();
		byte[] formData = encoding.GetBytes(jsonString);
		
		//Create the headers for this POST
		Hashtable theHeaders = new Hashtable();
		print("Request Headers:");
		theHeaders.Add("Content-Type", "application/json");
		print("Content-Type,application/json");
		theHeaders.Add("Content-Length", formData.Length);
		print("Content-Type,"+formData.Length);
		theHeaders.Add("Accept", "application/json");
		print("Accept,application/json");
		
		string userpwd = user + ":" + password;
		string encodedUserpwd = Base64Codec.EncodeTo64(userpwd);
		theHeaders.Add("Authorization", "Basic " + encodedUserpwd);
		print("Authorization," + "Basic " + encodedUserpwd);
		//++++++	
		
		//The blocking WWW object that initiate request upon construction
		WWW theWWW = new WWW(url, formData , theHeaders);
		while(!theWWW.isDone)
		{
			//Report progress
			if(m_ProgressHandler != null)
			{
				m_ProgressHandler(theWWW.progress);//Works
			}
			yield return null;
		}
		
		yield return theWWW;

//When www.text is access, all following codes are blocked for a while (1-2 sec)
    	print("Output...");		
    	//Do something with received data
    	print("WWW byte count: " + theWWW.bytes.Length);    		
    	if(theWWW.error != null)
		{
			print("Error: " + theWWW.error);
			yield break;
		}
		
		print("Deserializing...");
		byte[] WWWBytes = theWWW.bytes;//Very fast, does not block
        string JSONString = theWWW.text;//blocks for awhile for a www with byte count 37205271, when this line is commented, no blocking occurs
    	yield return null;
	}

Initially, I thought that creating another coroutine to decode the www.text would work but it did not. Now I’m pretty much stuck here. Is there anyway to prevent the block or should I just allow it to happen?

2

You are trying to convert 37MB of binary data into a string. This is going to be expensive. If you can, use the byte-array or use a BinaryReader to convert your string in smaller chunks so that you can ‘yield’ in between.

If you make the assumption that your binary data is all ASCII, you could also write out the byte-array manually, using a StringBuilder and a simple cast from byte to char.

Personally, I would strongly suggest you move away from JSON for binary bulk transfers. At the moment you are allocating first a byte-array, then a string, then you parse that string into some new byte-array, which you then write. (And you don’t even parse yet - add the JSON parser and the base-64 decoder and you have another waiting time added here.) Skip the parsing non-sense and let your server return binary data for these requests.