I’m trying to create a data visualization in Unity using kml data. Without worrying too much about what’s in the kml files, I’m trying to figure out the best way to:
- grab a number of large kml (10s to 100s of megs) files from various websites
- Save the files to a datastructure (local files? a data object?) to make parsing more efficient
- Be able to parse / search the kml data sets with different parameters, i.e. to find all instances of a tag (and grab attributes / child tags), search for a tag or attribute value, grab all tags within some distance of a coordinate, etc.
I’m curious if anyone else has worked on this or has ideas about the best way to go around it. The biggest consideration is always speed, of course, and I’m not a programmer by background, so while I could hack something together it would be a far-from-optimal solution.
I poked around on the Unity forums for an xml parser, and I’ve seen the csharp System.Xml.XmlReader (XmlTextReader, etc.) mentioned. It appears that there are different ways to access XML: from a text file or URI, from an xml node, from a dictionary, etc.
What would be the best (quickest) way to store and access the xml data without maxing out memory and bogging Unity down? Save it as a local text or binary file and parse as needed? Save it in memory as a node? Parse straight from the URI? Something else entirely?
Thanks much!
1 Answer
1
It’s been a while since I asked this question, and in the meantime we’ve mostly solved the issue. I’d like to share our solution for anyone else that’s trying to do the same thing.
We ended up using SharpKml (http://sharpkml.codeplex.com/), which is a .NET implementation of the entire KML 2.2 specification.
SharpKml is meant for .NET 4.0, so there are some issues to overcome, things like missing classes (Tuple) and methods (Enum.HasFlag). However, if you go through each of the errors with a google search, you will find ways to fake everything in .NET 3.5. I ended up adding a 2-tuple and 3-tuple implementation and some extension methods to get everything fixed. Once you do so, it works just fine in Unity.
Another issue is that you’ll need the Ionic.Zip library to do kmz, and that library doesn’t run nice in the webplayer - some issues with unmanaged code due to various encoding things. We were able to solve it, but that’s an issue for another post 
Once you get that all set up, then you just have to build a framework around that library that translates between the metadata within the KML (styles, etc.) to your Unity objects, and presto - you have support for KML. It’s not easy, by any stretch of the imagination - it took a pro like me a few days to get it running, let alone create a framework - but it’s doable and supports the entire KML specification.
Hi how's the NoSQL working out . http://forum.unity3d.com/threads/38273-Lightweight-UnityScript-XML-parser I'm about to start a similar project, well, I'm about to start to use unity on an old project where I need to read elevation data from google maps to create terrain - I want to keep the code small and this looks like the only way. I've looked at collecting my map data from other sources but the resolution is too low I'd be really interested in any suggestions you have, thanks for sharing so far.
– droidsculptorAs you guessed not much luck, I was thinking to extract the elevation data and use the numbers not the images... a real head scratcher.... and then I looked a openmaps and wondered if I could spoof the terrain by following their colour key - plenty of coding at work at the minute so I'm going to shoot zombies for a few days and see if I wake up in the middle of the night with some solution.....
– droidsculptorThanks for this, I have a copy of those files too.... you must have a newer version of photoshop than me (I'm still using version 7) but it all sounds cool. So you have a guesstimated higher rez version of the globe? I'd love to have a look at a little section of the mesh, if that's not too invasive, northwest of england near manchester would be cool (so I can recognise it) - I won't need texture in my final project and I'm also hoping to avoid using a mesh too.... at the moment I worrying about unity.... seems expensive if I end up writing all the code :)
– droidsculptorTexture is easy to get - you can use the google maps api to request it. As for the mesh, we're generating it entirely on the fly so I don't have anything to send you there, unfortunately. I can grab that portion of the topo image we created and give you code to turn it into an array of heights, if that works.
– Julien-Lynge