Hello,
I need to organize potentially large quantities of data and I was wondering if you could help me find a good solution.
The data I will be organizing are a series of 2d integer coordinates attached to a list of booleans. As such:
var x:int;
var y:int;
var conditions:boolean[]
I will be saving this data to disk as often as I can. I need to access the information periodically to search for a particular coordinate. The data set will increase indefinitely as the player plays the game.
I know the data I have should be pretty lightweight, but I need the data to be as versatile as possible. This is running on a mobile platform and I will be accessing the data pretty often.
My biggest concern is how to organize the data so it is easy to search through and broken into chunks which can be loaded independently of other chunks.
I’m unfamiliar with Quadtrees but my intuition tells me it might be the solution. If it is, I’m not sure how to implement this solution, especially with how it pertains to data storage.
Any insight into the problem is greatly appreciate. Thanks,
If I’ve understood, all that you want is load 2D Vectors from a file.
I’m developing a Unity3D Plugin for save/load datas (also Vector2 or Integers) from a file and all is encrypted…The method that I’ve used is to load the value from a section…
For example :
Position:Vector2(1,1);
Try with a similar solution…
I’m trying not to use the Vector2 class, since the floating point would unnecessarily increase the storage size (I only need an integer). Thank you for the advice though. I’m not too concerned about encryption either - if players want to hack the game, all they’re doing is robbing themselves of the experience!
It really depends on how hardcore you want to be about this and what your data encodes. Is there anything special about the data that we can exploit? Is there a way to sort is such that you could seek through a large file w/o loading the entire thing? Basically, is there any property of the data you could ‘index’ to allow for fast searching?
An R-tree is an efficient structure for storing and retrieving spatial coordinates and like its cousin the B-tree, it lends itself well to disk storage. However, it is rather complicated to implement. There may be easier solutions that would work just as well unless the data set is enormous.
I can’t say the data set should be enormous. I would estimate that the average player will use 500 - 1000 pieces of data, which need to be saved with the above variables. I haven’t worked out yet how I will actually retrieve the data - that is part of the problem: what is the best way to retrieve the data, and how to organize it?
The above variables are essentially the only way that the data set can be indexed. While most of the time the data will be accessed near a specific coordinate which moves slowly around on the environment (the player), there are times where the data needs to be accessed in disorganized positions.
I’m really a novice when it comes to data organization and management, so some of the questions you’ve all asked I can’t really answer or don’t know what information is impactful. Will 500-1000 instances of this data really be difficult to retrieve? When saving the data it doesn’t need to happen immediately (as in, it can wait a few frames) but reading the data needs to happen during the frame it is requested.
Thanks for your replies. Any additional information would be helpful
A standard integer and a float are both 4 bytes, so the storage size is the same. If the integers are below 32,768 (absolute value), you could use int16, or byte if below 256.
Though I have no experience with mobile platforms, if you’re just talking about 500 - 1000 of these data items, I’d be tempted to store them in memory. In that case, you could forgo the disk-based access all together, unless it’s needed for persistence. In that case, I’d just load and/or save it when it’s convenient (for instance, app start up and shut down).
One other thing. What is the numeric range of the values you’re storing? Could they be stored in something smaller than an int (for instance, a short)? Just trying to keep the data set as lightweight as possible…
The best Data Management tool is UniDDatabase. My product. It can manage most of the basic classes, example:Text,int,float,Sound,Texture,Prefabs. you manage most of you data with one line of code. You can read more about it here. This is the best data management product available on any platform.
Thanks, I may give it a short just having it all loaded at once. However, my game tends to use quite a bit of memory while it’s running and so it may not be an end-all solution.
I’m not very familiar with the standard data types, but the x and y coordinates will never really reach beyond the maximum distance traveled. For example, if there are 500 data sets the highest that X or Y could be is 500 or less.
