I’m working on loading/parsing JSON from a server. I’ve been having such a difficult time with this, I’ve reduced the JSON down to just a single name/value pair. The JSON file is returned from the server successfully and I convert it string and pass it to my JSON wrapper class, but I keep getting the following error:
using UnityEngine;
[System.Serializable]
public class FakeInfo
{
public string name;
public static FakeInfo CreateFromJSON(string jsonString)
{
return JsonUtility.FromJson<FakeInfo>(jsonString);
}
}
Additionally, no error is thrown by JsonUtility.FromJson() if I serve the JSON file locally:
var jsonString = File.ReadAllText (Application.dataPath + "/Resources/data/terrainial_content.json");
var foo = JsonUtility.FromJson<ArticlesCollection>(jsonString);
Debug.Log ("foo: " + foo.articles[0].title);
Served locally I can successfully work with the JSON as a data object, iterate on its array, instantiate objects from its values, but when called via WWW sonUtility.FromJson() casts the Invalid Value error:
@Brathann I had Debug scattered throughout, but one was just above the JsonUtility.FromJson() call, and output exactly the same value (as far as my eyes could tell) as that generated from retrieving the JSON file locally (which is parsed correctly by the same JsonUtility call). In fact I ran that output through a JSON validator and it validated fine.
The issue seems to be that there are exactly 3 extra bytes on the head of the response. The fix is to use WWW.bytes instead of WWW.text, then slice off the extra 3 bytes. Thusly,
It’s particularly strange given ALL the documentation I could find indicated using WWW.text. Seems like Unity should add a data property to WWW that would solve this problem.
Thanks for answer, saved me!
Maybe somebody knows the reason of this behaviour? My unity-php-mysql setup worked pretty well until I’ve made some content changes today (just deleted rows) - then thiss issue happend
I’ve since learned this is an encoding artifact called BOM, or Byte Order Mark, and it wastes an incredible amount of engineering effort and valuable time every single day all over the world. It is one of those classic “Terrible Engineering Ideas.” The specifics of it are here:
“BOM use is optional. Its presence interferes with the use of UTF-8 by software that does not expect non-ASCII bytes at the start of a file but that could otherwise handle the text stream.”
It pretty regularly crops up and irritates everybody it comes in contact with because it is optional and completely invisible in most editors, and its presence or absence are completely editor/toolchain dependent.
It gets even more infuriating when some toolchains account for it and quietly ignore it, and others completely choke up mysteriously on it, with nothing even remotely useful to help you diagnose it.
The only sure defense is to suspect it immediately whenever dealing with text, and to have a handy hex dump program nearby (such as the xxd utility) in order to inspect the first few lines of the file, see if there is anything unexpected.
Now you know, hopefully you won’t waste any more time chasing this ghost.
I would also love it if Unity’s JsonUtility.FromJson() would give indication of why it fails with “JSON parse error: Invalid value”. Unity, if you’re gonna build a black box (closed source product) and have me use it, you should write reasonable error responses. It’s not a nice moment when I’ve run a JSON linter on my text, experimented with removing whitespace, double-checked the class members, etc. and still see the opaquely unhelpful error message. I was pretty close to ditching this JsonUtility class in favor of 3rd-party code or just writing my own.
I mean… you should, honestly. JsonUtility is nice in that it’s builtin and uses the same serialization paradigms as the rest of the engine, but it is very lacking in features. Json.NET or LitJson are much more capable.
I’d take it one step further. The in-built JSON utility is pretty much a Fisher Price module as far as usefulness. It’s supposed to be optimized for performance by not handling “harder” things like Dictionaries, but that’s just no excuse for writing crappy software, IMNSHO.
As Star points out, Lit or JSON .NET way mo bettah. I use JSON .NET because it comes straight off the asset store and Just Works ™.
You do know what “error” means, right? If they had considered that the incoming text might contain a BOM they would have handled it properly. Since they did not consider this possibility all the parser sees is an invalid character. That is exactly what they tell you. They did not tell you that they see a BOM because in this case the parser would need to know what a BOM is in which case they would have considered it, which we already know they didn’t ^^.
Don’t get me wrong, they really should check for a BOM and throw it away if there is one. However they didn’t so we have to live with it. Either handle the situation yourself (by checking for a byte order mark and remove it) before you pass your text to the parser, or just make sure you save your text files without BOM, always.
For UTF8 it’s completely useless anyways since UTF8 consists of single bytes and the order is the same on all platforms. UTF8 is even ingenious enough so if, for whatever reason, the byte order of a multi byte UTF8 character is the wrong way round, that can be easily detected and corrected. Because when an UTF8 reader detects a continuation byte without getting the first byte it knows that the stream is either corrupted or the order might be the other way round. So it can simply continue reading the other continuation bytes until it arrives at the first byte and it can reconstruct the character.
This horrible habit to include a BOM for UTF8 text was mainly an issue with microsoft’s notepad. However as far as i know the latest version shipped with win10 by default does not include it. However even in notepad you always had the option to change this in the save dialog (at least since notepad actually supports UTF8). Other editors like Notepad++ should also default to not include a BOM but you can change the format in the Format menu.
I just tested to create a text file in visual studio community 2017 and it also by default includes a BOM… You can choose Save as and click the little arrow next to the save button and choose “save with encoding”. By default it uses “Unicode (UTF-8 with signature)”. Change that to “Unicode (UTF8 without signature)”. It’s crazy that even VS behaves that stupid and that there is no setting to set a default format.
I’d like to add that Unity’s JsonUtility is not the only piece of software that can’t handle the BOM properly. Even Microsoft’s Xml parser in .NET has this issue. It can be circumvented by routing the bytes through a StringReader / Writer which should remove the BOM as the BOM should never be part of a string.