I’ve been asked to save “all” statistics from a game “anytime”, “without the user noticing”. Read: Save 5000stats in a blink.
Sounds impossible to me but I’ve got to try so I went for SQLite and made a little test.
Now saving 100*3stats takes an average of 9.8s during which Unity freezes… That is way below expectation and actually seems like poor performance to me:face_with_spiral_eyes:. Retrieving those same entries is next-to-instant.
Now I’d like to hear any feedback from you guys on how you guys save massive amounts of data, what I should take a look at, anything, really
You can always try PlayerPrefs. Loading/Saving 5000 records is practically instant.
private var stats = new int[5000];
function Awake(){
for (i=0;i<5000;i++) stats[i] = i;
}
function OnGUI(){
if (GUILayout.Button("SAVE")){
for (i=0;i<5000;i++){
PlayerPrefs.SetInt(String.Format("stats{0}", i), stats[i]);
}
}
}
I don’t know anything about SQLite, but presumably it can be run in a background thread and thus not freeze your game or even really slow it down at all. Another alternative is to send the data using the WWW class via a POST request and then enter it into the database with server-side code. You could wait for the response from WWW which wont freeze your game as well. I would choose option 2 myself, if only because I’ve never used SQLite. I can’t imagine why option 1 shouldn’t work though
If you want to use SQLite, first see where the slowdown is happening. Output the SQL that Unity is normally sending to SQLite to a text file and populate the DB from the command line. If it’s slow there as well, see how much time is taken opening the table file. The writes may be fast and most of the time is taken opening the table. If that’s the case, then opening the table at game start and keeping it open while the game is running might work.
Hey, thanks everyone for their helpful input. Here’s a few thoughts:
I usually avoided PlayerPrefs like the plague. If feels messy and… unprotected, I dunno. Am I stupid?
I’ve kept the table open for the whole test as I wanted to remove any superfluous operation.
Last but not least, the idea of a background thread is kinda what I’d like to do, though I’m kind of a n00b and can’t seem to pinpoint the process to declare/manage threads.
Do I explicitly initialize another thread and handle it?
And… done! Be sure though that now you’re firmly in multi threaded territory and you need to be aware of reads/writes to ‘shared’ variables between threads.
100*3stats takes an average of 9.8s sounds pretty bad, the only reason for this high duration I see here will be that you do not use transactions and run each command separate, how exactly do you transfer the statistics into the database?
Most databases support binary data, so you could set up a table like this
Columns:
UserID
BinaryBlob
Then, in your game, use System.Runtime.Serialization.Formatters.BinaryFormatters.BinaryFormatter to serialize the data into binary or use System.Xml.Serialization (slower) to save the data is a text string.
Now you can do 1 insert statement per user which will be significantly faster.
To view the data, simply deserialize.
This method would be a little tedious because you want be able to use fancy SQL queries like (SELECT * FROM users Where TotalKills > 1000), because everything is in a custom binary format.
Using Binary data is what slows up a database. In my Native Sqlite Editor, I don’t use Blob. I used a custom field called Texture2d_uni. Do a search for “Native Sqlite For Unity” or UniSqlite for short and you will see pictures of what you can do…It’s very fast and efficient.
I’d say Marrrk is on the right track. I use SQLite in a number of C#-based projects (not Unity-based however), and it can be blazingly fast if used properly. However, if you neglect to wrap your INSERTS within a transaction, it’ll be orders of magnitude slower. Based on your reported insertion time, I’d say that you’re not using a transaction.
A 100 row insertion should be nearly instantaneous…
