Read data frequently from a huge csv file

Unity Engine
1

I am making a game that needs to access a few lines (10 lines) of data from a huge (2GB) CSV file frequently (around twice a day). I don’t want to add all the data in the game, so I am thinking of uploading it to Google sheets and reading it from there.

My questions are:

  • What are the best ways to get my task done?
  • If uploading it to Google sheets is ok, how to read data from Google Sheets and use it in the game? (Hoping to get the script to read the data)

Thankful for any help.

2

So you want your user to download 4GB per day? Thats around 120GB per month assuming he/she is playing daily. Thats quite a strain for any internet connection. Don’t you think?

Is the data changing? If it is static just put it in the build. Also if user only needs a small amount (10 lines) of the data let your app only download these. There is no need to load the whole stuff if you discard most of it anyway.

3

I think your needs have evolved beyond a csv file.

This is the realm of relational databases. If you put your data in a table in a SQL database and index it properly, you will be able to very quickly and efficiently query the appropriate small number of rows as needed from the table.