Hello,
Like the post title says, I’m wondering if JsonUtility serialization/deserialization is threaded under the hood? Knowing this is particularly important for WebGL builds at the moment since threading is broken.
Thanks,
Carlos
No, it’s not and especially in WebGL it would not be threaded since there are currently no threads in WebGL. Well actually Unity has started implementing threading support for web assemblies if the browser supports web assembly threads and if you enable them. Though this is still an experimental feature.
So regardless of the threading development state the JsonUtility is not threaded at all
Well, UnityWebRequest uses threads under the hood according to this unity form post and another example of something that is threaded under the hood is Newtonsoft.Json… both dependencies will build to WebGL and they kinda work (I say kinda, because as per this [other unity forum post]( Multithreading and WebGL page-2) code that is multithread C# is not reliable in WebGL right now). So, unsure what kind of “mapping” it is done from when building the Asm.js but I think there is some mapping to something if the original code is coming with treads…
I want to make sure JsonUtility doesn’t use Threads under the hood. I can’t tell because I can’t inspect the source code of it since it’s internal to Unity.
Newtonsoft.Json uses threads under the hood? I’ve never heard this and it wouldn’t make any sense. UnityWebRequest uses threads on most platforms to actually carry out the request, however this is completely transparent to the user and you never actually reach into that background thread. When building for WebGL Unity simply uses the browser internal async functionality to perform a web request. Again, a browser may use threads / threadpools under the hood, but that shouldn’t really bother you as from the Unity’s side it’s just an AsyncOperation that you can wait for in a coroutine which runs completely on the main thread. So what’s your actual concern here? The JsonUtility iis not even an asynchronous method. It returns synchronously the results. Even if it would use threads for internal processes somehow, why would that matter?
You said:
What makes you think that?
You also said
Unity does not support any managed threading support on WebGL at all at this moment. All the effords they put in is about engine internal functionality. There is currently no support for managed threading. You linked the forum thread about the state of implementation but you haven’t read it yourself? Maybe some statements from the employee’s weren’t that clear. However [this post]( https://discussions.unity.com/t/774290 page-2#post-7778838) is a good summary.
Well… the source code is available so this is actually very easy to verify, here’s the proof.
My actual concern is to inadvertently use a Unity component that uses threads under the hood. I’m trying to fix an issue in my WebGL application that has been plagued our production build for years. I don’t know how is it that NewtonSoft works if it’s a third party library with managed threads TBH, but as you saw in their open source repository… it is being used extensively and different versions of NewtonSoft give me different % rate of that error when transforming a Json.
So, NewtonSoft is not reliable 100% of the time; sometimes serialization fails. Can I rely on JsonUtility? Or do I need to write my own Json parser? That’s the thing.
Another API that gives weird errors is UnityWebRequest. Coincidentally, these two features use threads under the hood, and since UnityWebRequest works in WebGL it is doing something with the threaded code, right? UnityWebRequest is a wrapper around a C library… will that count as “engine internal functionality”; regardless, it’s broken.
So in a nutshell, my question is geared more towards a Unity Employee that has access to the source code. Can I rely on JsonUtility for WebGL builds to do millions of serializations per day with 100% accuracy provided that the objects to be serialized are always in a good state? Is my occasional error rate in serialization a victim under the hood threading that was bad ported to WebGL or some other stuff?
None of that proof demonstrates that it uses threads underneath.
Everything there is either setting the CurrentCulture during tests. Which tests aren’t really the library, and setting the CurrentCulture isn’t really “threading” even though its access property is a member of the thread.
Or the use of ‘ThreadSafeStore’ which is a wrapper around a lookup dictionary key->instance using a passed in factory delegate for creating those instances. This doesn’t mean it uses threads underneath, it’s what allows it to be thread safe in case you happen to use it in your own threads.
https://github.com/JamesNK/Newtonsoft.Json/blob/bf2e2a78e8febf0006ec647f9bde3aa5bbe0ce72/Src/Newtonsoft.Json/Utilities/ThreadSafeStore.cs
Basically for example in here:
https://github.com/JamesNK/Newtonsoft.Json/blob/f0dcd5b434371bcc59fcb4150979ce35a6a9338d/Src/Newtonsoft.Json/Serialization/DefaultContractResolver.cs
In this specific class the ThreadSafeStore is doing a type->jsoncontract lookup. What makes it thread safe is that it locks on a lock object when creating an instance for a specific key to avoid doubling up instances and modifying the dictionary if 2 threads happen to ask for the same contract at the same time.
That’s not using threads though… that’s just making sure your code is written in a manner that if someone accesses your library from multiple threads bugs don’t occur.
…
It can be argued that json.net does accept streams and what not using ReadAsync methods. And those streams underneath may use threads themselves in certain cases (depending on the implementation). But that’s not really json.net using threads. That’s json.net using the async library which may indirectly use threads.
Case in point searching for “Task.Run” which or “ThreadPool” doesn’t really bring up much outside of tests:
https://github.com/JamesNK/Newtonsoft.Json/search?q=Task.Run
https://github.com/JamesNK/Newtonsoft.Json/search?q=ThreadPool
4 Likes
You jump to conclusions way too fast. I highly doubt there are any issues in the Newtonsoft Json library as it is used by literally million peoples around the world and is still maintained. As lordofduct just explained, Json.NET does not use any threads. I have a copy of the repositiory on my machine. They use async and await in certain parts, however that does not involve any threads. It’s based on promise objects just like in javescript. Javascript in the browser is actually single threaded. The exception are the new background workers which however are not yet used and aren’t even supported in all browsers.
Even if Unity finally manages to implement managed threads for us to use, it’s very unlikely they would cause any issues since almost all Json.Net methods are blocking. JsonUtility is 100% blocking. So there’s no room for any threading issues.
If you have parsing issues, in 99% of all cases that’s because the json you want to parse has issues. You can use jsonlint.com to test your json if it’s valid or not. Though sometimes you can get other subtle issues like a json file that includes a BOM at the start (Byte Order Mark). This often times throws off parsers. You should get rid of it. Perferably in the actual source file.
Note that JSON is a strict subset of javascript. Keys always have to use double quotes. In javascript single quotes are also allowed, not in JSON.
I’ve written my own lightweight Json parser / generator that is just a single. It’s not an object mapper but simply parses the json data into distinct json types that just simplifies the access / usage of the data. Though since you are extremely worried about reliability that might not be an option for you.
You said you have occational issues. What have you done to debug those issues? Have you logged the data when you noticed issues? How have you actually identified that there are issues?
3 Likes
You’re right, it’s not the same to say that a piece of code is “thread safe” than “it is threaded” so my bad…should have been more specific.
I am trying to avoid code that is using System.Thread altogether, be that because of some Mutexes, or thread safe data structures, or whatnot, to see if we see a reduction in errors.
I’m just trying to the debug an issue we are experiencing for real in the context on the Unity WebGL build. I’m not saying NewtonSoft is broken in general… And out the extensive list of things my team has tried to debug the issue we are having I’m trying a new one which is, making sure whatever code gets used in our codebase is not importing System.Threading or any of its components in any shape/way or form.
This is a 1 year old post that my team did summarizing debugging efforts so far. It was up until this year that we started looking at serialization/deserialization and http requests. We got a significant reduction in errors when we changed the NetwonSoft Version (we actually did a regression on an older version). So far the two components that are giving us issues are UnityWebRequest and NetwtonSoft in one way or another. What they have in common is that they have System.Threading usage internally. Might be a coincidence? Maybe… For example, take this code snipped, it’s very straightforward:
unityWebRequest = new UnityWebRequest(url, "POST");
unityWebRequest.uploadHandler = new UploadHandlerRaw(postData);
unityWebRequest.downloadHandler = new DownloadHandlerBuffer();
unityWebRequest.SetRequestHeader("Content-Type", "application/json");
string requestDataToBeUploaded = Encoding.UTF8.GetString(unityWebRequest.uploadHandler.data);
if (requestDataToBeUploaded != jsonRequest) {
// We emit metrics in this section of the code, when this mismatch happens. And it does happen. And we also print the output json to see if the actual data was right/wrong to begin with.
}
yield return unityWebRequest.SendWebRequest();
// some more code..
// then we check again, for the same comparison as before
string uploadedRequestData = Encoding.UTF8.GetString(unityWebRequest.uploadHandler.data);
if (uploadedRequestData != jsonRequest) {
// And yes, we end up here as well.
}
For the volume of calls we do per day all systems combined we are in the order of millions of calls and we get a number of this kind of errors that is very, very small… around 0.00001% or something. I was just curious to know about JsonUtility because It could be a good replacement for NewtonSoft. I don’t know if that will make things better or not. Right now UnityWebRequest is the thing that fails more and my only explanation for it is that either it is broken for some reason when working at that scale, or that whatever gets generated on a WebGL build for it, is not 100% reliable at that scale.
So I’m on the point where I was thinking:
NewtonSoft → Replace with JsonUtility if it doesn’t use System.Threading
UnityWebRequest → Just write a plugin that uses plain Javascript for HTTP and compare the error rate
Btw, thanks for the help with the comments, ideas and whatnot.
Well, we don’t know what kind of application this is. It’s quite unusual to have a webgl application running for such a long time. However, I see several approaches how this could be debugged. If you use firefox as browser you can open the developer tools and in the network tab you can choose that logs should not be cleared on page reload. That way all requests get preserved (unless you clear the list manually). The network tab has both, the outgoing data as well as the incoming data.
Of course searching through those logs is kinda difficult. However, you still haven’t mentioned how you actually detect that something has gone wrong, so we can’t tell if you can react “timely” to check the last x requests and see what data actually went both ways.
An alternative to the developer tools in the browser could be a proxy (charles proxy) or wireshark. Though this would be more complicated when you use https since the traffic would be encrypted and you have to break it (usually by a local root certificate). Of course you could try switching to unencrypted http and see if that changes anything on its own. Using an actual packet analyser you can use proper filters which could help to pinpoint the issues.
Another approach could be to implement a seperate logging server and have all requests, client and server sending those to the logging server.
Parallel I would recommend to also log identifying details of each request to the console (again, server and client) with a timestamp so they can be correlated with the actual network requests.
As already said, it’s difficult to give advice without knowing anything about the application. It’s still very unlikely that the issue is on the parsing side. Speculating that just the use of a parser would somehow affect the transport of the message would be a really far stretch. If such issues would be a thing (that unrelated systems affect each other) it would be much more likely that the system crashes.
What kind of errors?
You’ve mentioned errors a few times now, even in the post from a year ago that you linked (I also note that a unity person responded asking for more details, but y’all never responded to them).
So I ask… what sorts of errors? Can you show an example of the error in question?
What about them makes you think they’re threading related? Are there actual exceptions occurring that make reference to System.Threading?
I understand that you are looking to avoid System.Threading… but JsonUtility doesn’t even use C# at all and taps into the unity C/C++ logic as you can see here:
https://github.com/Unity-Technologies/UnityCsReference/blob/master/Modules/JSONSerialize/Public/JsonUtility.bindings.cs
It’s straight up just a binding class into unity.
And as for json.net, sure it accesses the System.Threading/System.Threading.Tasks namespaces, but it only does so for access to the ‘Task’ promises to work in tandem with the various Async methods like Stream.ReadAsync. And I you don’t haver to call those methods. Just call the blocking versions instead.
We personally use the json.net package provided by unity:
https://docs.unity3d.com/Packages/com.unity.nuget.newtonsoft-json@2.0/manual/index.html
And as you can see in some example code here we just avoid the async methods when reading/writing:
public void Load(System.IO.TextReader reader)
{
JToken jt = null;
try
{
using (var jreader = new JsonTextReader(reader) { CloseInput = false })
{
jt = JToken.Load(jreader);
}
}
catch (System.Exception ex)
{
throw new PersistentDataContextException("Failed to load content.", ex);
}
if (jt == null) throw new PersistentDataContextException("Failed to load content.");
_root = jt;
}
public async UniTask LoadAsync(System.IO.TextReader reader)
{
JToken jt = null;
try
{
using (var jreader = new JsonTextReader(reader) { CloseInput = false })
{
#if UNITY_WEBGL
jt = JToken.Load(jreader);
await UniTask.Yield();
#else
jt = await JToken.LoadAsync(jreader).AsUniTask();
#endif
}
}
catch (System.Exception ex)
{
throw new PersistentDataContextException("Failed to load content.", ex);
}
if (jt == null) throw new PersistentDataContextException("Failed to load content.");
_root = jt;
}
public void Save(System.IO.TextWriter writer)
{
try
{
using (var jwriter = new JsonTextWriter(writer) { CloseOutput = false })
{
_root.WriteTo(jwriter);
}
}
catch (System.Exception ex)
{
throw new PersistentDataContextException("Failed to save content.", ex);
}
}
public async UniTask SaveAsync(System.IO.TextWriter writer)
{
try
{
using (var jwriter = new JsonTextWriter(writer) { CloseOutput = false })
{
#if UNITY_WEBGL
(_dataContext as JsonPersistentDataContext).RootJObject.WriteTo(jwriter);
await UniTask.Yield();
#else
await _root.WriteToAsync(jwriter).AsUniTask();
#endif
}
}
catch (System.Exception ex)
{
throw new PersistentDataContextException("Failed to save content.", ex);
}
}
We have no problems running on webgl doing this.
And note that it’s not actually json.net that’s having the problem here… it’s because the TextWriter we pass in is using a stream that attempts to use threads. So json.net just incidentally causes a thread to spin up because I told it to read the stream asynchronously which the stream did by using a thread.
…
TLDR;
If you want to avoid System.Threading, yes, JsonUtility will avoid that (but also bring in some other complexities as its not as configurable as json.net… and I don’t know how you have your json.net configured).
But I’m hesitant to believe Json.net using threads is the actual issue you’re having. And swapping over may not actually fix your problem.
Hell… you’ve had a year now, you could have swapped it out and ran a barrage of tests against it to see.
1 Like
Thanks for both replies. So, I’m relatively new to the team and due to prioritization on other things and extremely low error rate overall, I am on the understanding that the issue was left as it is for a while. I picked the issue because I wanted to give it a shot… The WebGL build runs on machines that are not on our side but on the clients side. I can’t debug the logs from there. All I can do is emit metrics and send back that to our systems identify the errors. We have thousands of clients devices running most of the day.
In the code snippet I have posted before you can see the types of comparisons… but to summarize here:
- Before doing a POST request, the Json Serialization fails sometimes. We do a validation and emit a metric to see if what we are sending is valid or not. Client logs are not on our side, so as Bunny83 said we can catch the error and save that log entry into one of our systems.
- After doing a POST, the back-end will get a JSON with empty fields, or garbage data. We also emit metrics on the back-end, is on these instances that I can easily see the logs because they are on our side.
- After getting the POST response back from the client.
So, basically we have mismatches between unityWebRequest.uploadHandler.data and the actual JsonData that we send.
What I know for a fact and I mean an observable, measurable fact is that metrics we emitted for errors regarding serialization of the JSON’s on the client side went significantly down after we changed the NewtonSoft version. Just changing the version reduced the error rate around 90% for client errors regarding serialization. And this is using the same calls, same data.
Yes, I mentioned before that I had identified that it was a binding around some C/C++ library. But I’m also curious if that C/C++ had c++ threads of some sort.
Like I said before, I’m just relying on my engineering intuition. The more complex a system is, the more chances are that something has bugs. And I assume doing transforms/mappings from C# to C/C++ and then to Asm.js might not be perfect and have bugs and that those transforms are more likely to contain bugs if there is threading involved, because it’s objectively harder to get the implementation right.
So, in a nutshell…IMO the simpler (i.e less complex) the code we have that will get transformed to WebGL is, the better. Less chances for something to break.
Let me simplify that for you:
Simpler code is better.
Have you considered grabbing a hex dump of the HTTP request body? I recently written my own DNS server (just for fun) and I have added a hex dump of the incoming and my outgoing data. This has helped to identify certain issues during the development and also helps to analyse spam requests I receive and what kind of requests are tried.
You said:
So did you actually compare the data before the json encoding with the binary data that you send out through the uploadhandler? So you actually decode the upload data again? Or how do you compare them?