System.Xml in the web player

Unity Engine
1

Hi everyone, new Unity pro user here, just started a new project.

I’m having some trouble with the usage of System.Xml on the web player.
For my project, I need to read a big XML file (over 4MB in size!) from an online server, parse it and use the data to build geometries.
I tried first using some of the XML Parser scripts other people posted on this forum, and they work fine, however the parsing is amazingly slow (minutes for such huge XML files).
Then I tried importing System.XML and it’s amazingly fast! Parses in less than a second the same files! The code works great on standalone Windows, Mac and Android (didn’t try iPhone yet). However the web player seems to ignore LoadXml()

So if I do something like

import System.Xml;


class XmlDataLoader
{	
	var GuiData:String;
	var www:WWW;
	var activeXml:XmlDocument;	
	var xmlData:String = "";
	var guiText:String = "";	
	var XmlIsLoaded:boolean = false;

	
	
	function LoadObjectXML(xml:String)
	{			
		XmlIsLoaded = false;
				
		www = new WWW(xml);		
	}
	
	function OnGUI ()
	{
		var textrect = Rect (10, 10, 220, 150); 
		if (GuiData) GUI.TextArea( textrect, GuiData ); 	
	}
	
	function Update()
	{
		if (this.www  !XmlIsLoaded)
		{			
			GuiData = "Downloading XML... "+(Mathf.Round(this.getProgress()*1000)/10).ToString()+"% done"+"\n"+evLoader.guiText;		
			this.Update();
		}
		if (!XmlIsLoaded  www.isDone )
		{
			XmlIsLoaded = true;
			xmlData = www.text;				
			
			buildObjectsFromData();
		}
		
	}
		
	function getProgress()
	{
		if(www)
		{
			return www.progress;
		}
		else
		{
			return -1;
		}
	}
	
		
	function buildObjectsFromData()
	{			
		guiText+="Parsing...";   
		activeXml = new XmlDocument();
		
		if (xmlData)
		{			
			guiText+= "\nThe file exists and has "+xmlData.length+" characters\n";			
			activeXml.LoadXml(xmlData);
		}
		else
		{
			guiText+="\nNo XML data!!!!"; 
		}
		
		guiText+="\nParsing complete"; 
		
		//Then use the data for whatever
	
		
	}
}

(This is based on my actual code, but not all of it nor exactly like it. Serves for illustration of the problem only)

If then I create a XmlDataLoader object and make it load an XML posted online trough LoadObjectXML(url), it should:
-Load it, displaying a GUI text indicating the progress
-Call buildObjectsFromData(). Display “Parsing…”. Validate the existence of the string correspondent to the XML, and display its number of characters (this is for debug purposes, for me to be sure the XML was really loaded). Then parse the XML. Then display “Parsing complete”. Then use the data

All of this works fine in every build platform except the web player. It gets to verify the existence of the XML string data, displaying its number of characters (in test versions I even made it display the first 200 characters, just to make sure it was really reading the file correctly!)… And then… Nothing. It never gets to display “Parsing complete”.
However I have a few objects in the scene, and a camera controller, and it’s possible to keep flying around. So it’s not frozen! Only it never gets to parse.

Anyone has a clue on what may be happening?

Thanks!

2

The reason why people are using the amazingly slow parser is that including System.Xml in the web player makes it amazingly big. You’re seeing these results because we quite simply removed System.Xml from the webplayer.

If you need to run data loading fast, why then use XML in the first place?

3

AngryAnt,

Including System.Xml in the web player increased its size by ~320Kb, which is fine (even at dial-up speeds it would take much less time to download it than the extra parsing time with the alternative parser). I understand you removed it from the player, that’s fine. But if I include it in my build, should’t it be able to parse my files?
It just happens that I’m developing a scientific visualization tool, and the raw data is compiled to XML. No way around that, sorry. I wish.

So let me ask the question again, in another way. Can I use System.Xml in the webplayer at all?

4

Yes, you can use System.Xml in the web player. I do at work with version 2.6.1.

Although, I do not use the XmlDocument.LoadXml method; I use XmlTextReader and XmlSerializer. Perhaps its an issue with XmlDocument and that particular method?

EDIT: Maybe you can use its Load method instead and pass in a TextReader or XmlReader or a the XML as a stream:

5

Hey FizixMan,

Thanks for your input! Just tried your suggestion of using the Load method with a XmlTextReader

var textReader = new XmlTextReader(new StringReader(xmlData));
activeXml.Load(textReader);

Again it works great on the standalone player, but not on the web. I guess there’s some problem with XmlDocument on the web player.
Will try XmlSerializer next

6

its not xmldocument but what it relies on for load and thats system.net httprequest which is only present on the standalone windows / osx build and not usable anywhere else, neither webplayer nor iOS has it.

WWW + XMLSerialize will work fine on all platforms.

7

This sucks! I guess with XmlSerializer I can’t simply parse an XML file and retrieve the nodes relevant for the application…
It would be great to have the choice of using System.Xml in the web player if we wish so, even if it “bloats” the application.

I never used XmlSerializer before (n00b, I know) but from the examples read and simple tests done over the last few hours it seems inappropriate for what I want. How do I get it to ignore tags not defined in my type class? I keep getting an error message.

8

You can use the various attributes such as XmlIgnore, XmlAttribute, or XmlElement, etc.

Here’s a class I used in Unity a while ago (and would be very happy to go back and rewrite), it’s probably more complicated than it needs to be, but serves as an example:

using System;
using System.Xml.Serialization;

namespace Genexis.Domain.Commands
{
    [XmlType("item")]
    [XmlRoot("item")]
    public class ItemCommand
    {
        private int m_ID = 0;
        [XmlAttribute("id")]
        public int ID { get { return m_ID; } set { m_ID = value; } }

        private ItemCommandAction m_Action = ItemCommandAction.Add;
        [XmlIgnore]
        public ItemCommandAction Action { get { return m_Action; } set { m_Action = value; } }
        [XmlAttribute("action")]
        public string ActionString
        {
            get
            {
                return m_Action.ToString().ToUpper();
            }
            set
            {
                if (!String.IsNullOrEmpty(value))
                {
                    this.m_Action = (ItemCommandAction)Enum.Parse(typeof(ItemCommandAction), value, true);
                }
            }
        }

        private int m_ModelConfigID = 0;
        [XmlAttribute("mcid")]
        public int ModelConfigID { get { return m_ModelConfigID; } set { m_ModelConfigID = value; } }

        private double m_X = 0;
        [XmlAttribute("x")]
        public double X { get { return m_X; } set { m_X = value; } }

        private double m_Y = 0;
        [XmlAttribute("y")]
        public double Y { get { return m_Y; } set { m_Y = value; } }

        private double m_Z = 0;
        [XmlAttribute("z")]
        public double Z { get { return m_Z; } set { m_Z = value; } }

        //private double m_RotationX = 0;
        //private double m_RotationY = 0;
        private double m_RotationZ = 0;

        [XmlAttribute("rotation")]
        public double Rotation { get { return m_RotationZ; } set { m_RotationZ = value; } }

        private bool m_Cache = false;
        [XmlIgnore]
        public bool Cache { get { return m_Cache; } set { m_Cache = value; } }
        [XmlAttribute("cache")]
        public string CacheString
        {
            get
            {
                return m_Cache.ToString();
            }
            set
            {
                m_Cache = (value == null ? false : value.ToLower() == Boolean.TrueString.ToLower());
            }
        }

        private bool m_Flip = false;
        [XmlIgnore]
        public bool Flip { get { return m_Flip; } set { m_Flip = value; } }
        [XmlAttribute("flip")]
        public string FlipString
        {
            get
            {
                return m_Flip.ToString();
            }
            set
            {
                m_Flip = (value == "1");
            }
        }

        private BoundingType m_BoundsMode = BoundingType.Box;
        [XmlIgnore]
        public BoundingType BoundsMode { get { return m_BoundsMode; } set { m_BoundsMode = value; } }
        [XmlAttribute("boundsmode")]
        public string BoundsModeString
        {
            get
            {
                return m_BoundsMode.ToString();
            }
            set
            {
                try
                {
                    m_BoundsMode = (BoundingType)Enum.Parse(typeof(BoundingType), value, true);
                }
                catch
                {
                    //TODO log this
                    //probably don't want to throw an exception as this will occur on deserialization
                }
            }
        }

        private bool m_Selectable = true;
        [XmlIgnore]
        public bool Selectable { get { return m_Selectable; } set { m_Selectable = value; } }
        [XmlAttribute("selectable")]
        public string SelectableString
        {
            get
            {
                return m_Selectable.ToString();
            }
            set
            {
                m_Selectable = (value == null ? false : value.ToLower() == Boolean.TrueString.ToLower());
            }
        }

        private PropertyTransformCollection m_Data = new PropertyTransformCollection();
        [XmlElement("data")]
        public PropertyTransformCollection Data 
        { 
            get 
            { 
                return m_Data; 
            } 
            set 
            {
                m_Data = value ?? new PropertyTransformCollection();
            } 
        }

    }
}

You can also see in the last property “Data” it uses a PropertyTransformCollection which is another similarly adorned class that contains other serializable properties. In the end, it serializes/deserializes XML that looks like:

<?xml version="1.0" encoding="utf-8"?>
<item xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema" id="1" action="CHANGE" mcid="28" x="0" y="0" z="0" rotation="0" cache="True" flip="False" boundsmode="None" selectable="True">
  <data>
    <properties>
      <p n="Colour" v="#FFFF0000" />
    </properties>
    <transformations>
      <t t="Scale" v="0.75" />
    </transformations>
  </data>
</item>

So you can create classes in Unity that with the correct design and attributes can likely match whatever the XML schema is that you’re trying to read/write.

EDIT: you can also see in the class an example of how I use the XmlIgnore attribute for the Flip property. With this, my code can use the boolean but I can apply special formatting for serialization/deserialization with the “FlipString” property instead which is what’s included in the XML. The XML specification was “1” or “0”, but I wanted my code to treat that as a boolean true/false.

EDITx2: actually, there’s probably a bug in that Flip getter property when sending data out (which we don’t use which is why we never noticed it), it should look like this instead:

public string FlipString
        {
            get
            {
                return m_Flip ? "1" : "0";
            }
            set
            {
                m_Flip = (value == "1");
            }
        }

EDITx3: Also, when you go to try and create the classes and format them to match the XML specification, I find it easier to serialize those classes and manually compare the XML to see what needs tweaking/changing instead of trying to deserialize the incoming XML. Reason being, if way you setup the attributes/classes isn’t exactly correct, the deserialization will throw not-very-helpful exceptions. Better to just create a dummy instance of those classes, serialize it and manually compare the resultant XML.

9

FizixMan,

I really appreciate your help. But unfortunately the XML files I need to read aren’t always formatted the same way. Some tags may exist in certain files and not in others. So if I deserialize an XML which does not correspond to the format of the class, there’s an error and the program will not proceed. However I know very well what tags/attributes to look for, and a parser may handle that.

I guess I’ll resort again to a script-based parser. Dang! So frustrated!

10

I think you can use the XmlDefaultValue attribute to handle cases where tags/attributes are missing in the XML.

Additionally, if you implement the IXmlSerializable interface, then you can perform the xml node reading/writing manually:

11

Hey Fizixman, IXmlSerializable is nice indeed, but its XmlReader() method needs a System.Xml.XmlReader, which won’t work on the web player.
You say above you use XmlTextReader, but can you really make it work in the browser? It also depends on System.Xml…

12

Yup, here, I’ll post the code from our old Serialization class. Again, probably not the best example of best code (and it’s even been completely scrapped/overhauled for our Silverlight application) but you’ll get the idea. In particular, note the FromXml(Stream, Type) method as that’s the meat potatoes of deserializing.

In fact, looking at this now, we didn’t actually need the XmlTextReader initially. We used to just convert the xml string into a MemoryStream and pass that into the XmlSerializer. The only reason we added the XmlTextReader was to handle some crap about how whitespace/newlines were being handled/translated between our various mediums.

using System;
using System.IO;
using System.Xml.Serialization;
using System.Xml;

namespace Genexis.Common.Utilities
{
    public class Serialization
    {

        private static ILog LOG = LogManager.GetLog(System.Reflection.MethodBase.GetCurrentMethod().DeclaringType);

        public static T FromXml<T>(string xmlString)
        {

            try
            {
                using (MemoryStream stream = new MemoryStream(StringEncoding.StringToUTF8ByteArray(xmlString)))
                {
                    return FromXml<T>(stream);
                }
            }
            catch (Exception ex)
            {
                LOG.Error(ex);
            }

            return default(T);

        }

        public static T FromXml<T>(Stream xmlStream)
        {
            try
            {
                return (T)FromXml(xmlStream, typeof(T));
            }
            catch (Exception ex)
            {
                LOG.Error(ex);
                while (ex.InnerException != null)
                {
                    ex = ex.InnerException;
                    LOG.Error("INNER: " + ex.Message + ", TRACE: " + ex.StackTrace);
                }
            }

            return default(T);
        }

        /// <summary>
        /// Deserializes the given xml string
        /// </summary>
        /// <param name="xmlString"></param>
        /// <returns></returns>
        public static object FromXml(string xmlString, Type type)
        {
            try
            {
                using (MemoryStream stream = new MemoryStream(StringEncoding.StringToUTF8ByteArray(xmlString)))
                {
                    return FromXml(stream, type);
                }
            }
            catch (Exception ex)
            {
                LOG.Error(ex);
                while (ex.InnerException != null)
                {
                    ex = ex.InnerException;
                    LOG.Error("INNER: " + ex.Message + ", TRACE: " + ex.StackTrace);
                }
                throw;
            }
        }

        /// <summary>
        /// Deserializes the given xml stream
        /// </summary>
        /// <param name="xmlStream"></param>
        /// <returns></returns>
        public static object FromXml(Stream xmlStream, Type type)
        {
            try
            {
				//needed to add this wrapper because it was normalizing newlines from \r\n to just \n
                XmlTextReader reader = new XmlTextReader(xmlStream);
                reader.WhitespaceHandling = System.Xml.WhitespaceHandling.Significant;
                reader.Normalization = false;
                reader.XmlResolver = null;

                XmlSerializer deserializer = new XmlSerializer(type);
                return deserializer.Deserialize(reader);
            }
            catch (Exception ex)
            {
                LOG.Error(ex);
                while (ex.InnerException != null)
                {
                    ex = ex.InnerException;
                    LOG.Error("INNER: " + ex.Message + ", TRACE: " + ex.StackTrace);
                }
                throw;
            }
        }
        /// <summary>
        /// Serializes the given object into an xml string
        /// </summary>
        /// <param name="obj"></param>
        /// <returns></returns>
        public static string ToXml(object obj)
        {
            if (obj == null)
            {
                throw new ArgumentNullException("obj");
            }

            try
            {
                using (MemoryStream stream = new MemoryStream())
                {
                    XmlSerializer serializer = new XmlSerializer(obj.GetType());
                    serializer.Serialize(stream, obj);

                    string serializedData = StringEncoding.UTF8ByteArrayToString(stream.ToArray());

                    //we need to remove all characters before the first '<'.  
                    //These are characters indicating the formatting and they screw up some applications trying to parse the XML.
                    serializedData = serializedData.Substring(serializedData.IndexOf('<'));

                    return serializedData;
                }
            }
            catch (Exception ex)
            {

                LOG.Error(ex);
                while (ex.InnerException != null)
                {
                    ex = ex.InnerException;
                    LOG.Error("INNER: " + ex.Message + ", TRACE: " + ex.StackTrace);
                }
                throw;
            }
        }
    }
}

This is all running in 2.6.1. Perhaps there’s a bug with all this in the current version 3?

EDIT: There’s usages in there of “StringEncoding” which is another class of ours. All it does is convert to/from strings and UTF8 byte arrays.

13

maybe external call and parse xml with javascript on webpage then send data back to player?

14

Hi,

This answer is probably too late already, but if someone is interested. I got XmlDocument.Load() to work in the webplayer.

First of all I use Resources.Load() to load my textasset dynamically and then I use a XmlTextReader for the XmlDocument.Load() call. At first it did not work at all and I seemed to get the following error in the player log:

MethodAccessException: Attempt to access a private/protected method failed.

After inspecting the logs I noticed that the error always occurred when the following methods were called (note that they are in reverse order):

  at System.Security.SecurityManager.ThrowException (System.Exception ex) [0x00000] in <filename unknown>:0 
  at System.Xml.XmlResolver.ResolveUri (System.Uri baseUri, System.String relativeUri) [0x00000] in <filename unknown>:0 
  at System.Xml.XmlUrlResolver.ResolveUri (System.Uri baseUri, System.String relativeUri) [0x00000] in <filename unknown>:0 
  at System.Xml.DTDReader.PushParserInput (System.String url) [0x00000] in <filename unknown>:0 
  at System.Xml.DTDReader.GenerateDTDObjectModel () [0x00000] in <filename unknown>:0 
  at Mono.Xml2.XmlTextReader.GenerateDTDObjectModel (System.String name, System.String publicId, System.String systemId, System.String internalSubset, Int32 intSubsetStartLine, Int32 intSubsetStartColumn) [0x00000] in <filename unknown>:0

So I thought that it could have something to do with System.Xml checking the xml file with the dtd and calling a method that was inaccessible by the security of the web player. Therefore I just commented out the reference to the DTD file in my xml files and suddenly everything seemed to work again.

So if you have the same issue, try removing your DTD reference in your xml file if it is not necessary. In my case it wasn’t because the xml-files are used for some level logic designed by our game designer in an xml framework. He checks the xml vs. the DTD and it doesn’t change afterwards.