How to Read/Write only specific parts of a file?

Unity Engine
,
1

So i was experimenting with making things save and load. I figured that having a large amount of thins to save is not optimal when your loading it as a whole file as, and then using a script to pick the entire chonker apart, every time you need a very small portion of the file. Solution No.1 is to make a bizillion smaller files, and make a method to pick the right one to save and load from when you need it.

Unity/C# makes reading and writing to the full scope of a file not that complicated, but this solution comes with the problem that having a bizillion files as one save solution, is far from optimal.

I then started to look for a method that makes unity only read or write to a specific coordinate of a file, whilst not loading the entire thing into your precious RAM.

It seems like a method seems to exist somewhere, but the documentation and tutorials on how to achieve this are so vague, the closest thing i get to are errors or empty results…

What i have tried so far are thing like trying to stream files (unsuccessfully);

        string path = Application.dataPath + "/saves/" + saveId + ".gameSave";

        int lenthToRead = 5;
        int positionToStartReading = 4;
 
        char[] charArray = new char[lenthToRead];
        string output = new string("");

        StreamReader read = new StreamReader(path);

        read.Read(charArray, positionToStartReading, lenthToRead);

        output = "";
        for (int i = 0; i < lenthToRead; i++)
        {
            output = output + charArray[i];
        }

        Debug.Log(output);

        read.Close();

No matter where i look, every bit of documentation or tutorials, seems to be bent on having me load the entire file into memory before being able to do anything with it…

Does anyone know how to do this properly?

2

Have you looked into System.IO.FileStream and its ReadAsync API already?

FileStream.ReadAsync allows you to read a sequence of bytes from the file stream to memory without blocking the main thread.

3

I can’t advise on your current issue but I can interject and say that having a ‘bazillion’ files isn’t a particularly bad thing.
Games like Minecraft do similar things by storing data in ‘Chunks’.

Given that Minecraft is procedural and seeded, it doesn’t need to make a new save for every single chunk it makes, instead it just saves the differences between the procedurally generated and the player edited versions, preferring the player edited versions of each chunk.

I don’t know much about your implementation but I certainly don’t think having multiple files for this is a bad thing, especially with sufficient optimizations.

4

:open_mouth: xD

So i tried doing it the async way,
and after a embarrassingly low amount of minutes of reading how async read/write operations work in microsofts documentation;

i tried saving something at a coordinate;

            string path = Application.dataPath + "/saves/" + saveId + ".gameSave";

            SeekOrigin whatSideToStartCountingOffsetFrom = SeekOrigin.Begin;
            int offsetPositionToSaveFrom = 5;

            byte[] result = Encoding.Default.GetBytes(" This bit of data was put in lenth 5 of the save file :smile: ");

            using (FileStream SourceStream = File.Open(path, FileMode.OpenOrCreate))
            {
                SourceStream.Seek(offsetPositionToSaveFrom, whatSideToStartCountingOffsetFrom);
                await SourceStream.WriteAsync(result, 0, result.Length);
            }

And to my suprise it works :open_mouth:

Then i tried loading something at a coordinate;

        SeekOrigin whatSideToStartCountingOffsetFrom = SeekOrigin.Begin;
        int offsetPositionToSaveFrom = 5;
        int lenthToRead = 5;

        byte[] bytesToRead = new byte[lenthToRead];

        using (FileStream SourceStream = File.Open(path, FileMode.OpenOrCreate))
        {
            SourceStream.Seek(offsetPositionToSaveFrom, whatSideToStartCountingOffsetFrom);
            await SourceStream.ReadAsync(bytesToRead, 0, bytesToRead.Length);
        }

        string output = Encoding.Default.GetString(bytesToRead);
        Debug.Log(output);

And it all works like a charm :open_mouth:
Thanks a bunch lol

PS:

I know in voxel games you only save generated voxel collections the player touches, but the reason i want to combine all my tiny files into 1 big one is because in my current project i want the player to be able to scramble the entire world reasonable fast, and i have also noticed, that when you instruct a computer to destroy and create files on a wimps notice, it tends to scramble the computers registry full of registry errors, which is also not optimal, especially since not all platform devices have a method of cleaning up their registry (e.g. almost any console device). Also there are many other small benefits to having a save file be 1 file (easy to download, easy find and move, etc.)

5

Reading and writing a file should having nothing to do with the registry. You might be confusing it with how PlayerPrefs works since they do write to the registry but if your just writing bytes into a file yourself, its fine.

1 Like