BG Database (inMemory database | Excel/Google Sheets syncing | CodeGen | Save/Load support)

@Duffer123_1 , unfortunately node graph is delayed. We had hard time trying to adopt third party node graph editor and we lost a lot of time. We had to switch to multi threading add-on instead of this feature.
I updated roadmap here: http://www.bansheegz.com/BGDatabase/

1 Like

Ok. Looking forward to it!

1 Like

I just got the invite, I will be sure to try it out. I had a question though about this. I have been working on an RPG but was rethinking my DB setup, my game is somewhat split up so that it has a map server, a login server, etc. I wanted to try and incorporate a ā€œstateā€ server which would essentially act as just an in-memory database and the only thing that actually communicates with the DB is the state server and it would only do it periodically as needed.

The other servers and what not would communicate directly with the state server at all times and the state server would just do something like batch saving every now and then, so that instead of a server having to do queries to a standard db, the info is readily available. I see that this can save and load to google sheets and such, but is it possible to have this act as a state server since it handles the in-memory portion well from what I can see, but then save its data to something a bit more suitable for data storage instead of just google sheets or excel, etc?

@MostHated ,
no, unfortunately, BGDatabase is not meant to be run outside of Unity, so it can not be run in any server environment. Even if it did, it would not make much sense, cause we could not compete with traditional databases. Itā€™s meant to be an alternative for Unityā€™s scriptable objects, csv/xml files or whatever people are using to store their ingame data. The purpose for multithreading support is to allow to use BGDatabase with new Unityā€™s Entity-Component System. Also GoogleSheets export/import works in Editor only, in Runtime you can update database data on players devices with LiveUpdate plugin, but there is no way to update GoogleSheet data from players devices.
As far as I understood, you want to make a fast proxy server for better performance and to reduce db servers load. Probably traditional inMemory databases is a good choice for this, maybe some specialized software exist. Unfortunately, BGDatabase is a no-go for this scenario.

It might still be able to help out with what I am after, or at least make for a good learning experience if nothing else. When I was referring to it being a state server I was meaning that it would still be running as a Unity instance. When I mentioned map server, login server, etc, they are each Unity instances, they are just dedicated to specific tasks, but running in separate binary files so that they could be separated out to run on different physical hardware if need be for expansion and growth, they just run minimal services, such as no physics, no lighting or anything using a customized player loop within the particular Unity instance, so it would still be running within Unity as intended.

As for saving and loading of data, essentially what I was referring to was it would be great to have an instance of Unity running an in memory db which would be the ā€œstate serverā€. My game is an RPG, so say someone logs on, ideally when they log in and select a character, it would tell the state server to load up that particular players data, the way I envision it working is the state server would connect to whatever its ā€œcoldā€ or ā€œinactiveā€ data source happened to be and pulls all the user details for that particular user one time and its then there in memory, as the player plays all interaction is done with the state server only so as not to be making time-consuming database queries throughout the play session if they get a new item or a stat changes, etc, in the event that there are hundreds (or possibly thousands) of players. Periodically the state server would take all recorded changes for all players and save that back to whatever the ā€œinactiveā€ data storage happened to be, which it sounds like in this case is just saved locally, which may not be a bad thing, but ideally in my case it would be nice if the inactive and non in use data was saved outside of Unity so say, I have a website with a user account page that displays a users stats similar to something like WoW Armory, once the user logs off, the in-memory data saves a copy to the inactive (or I guess, cold storage?) so it can be used for other purposes but also backed up / replicated in a more enterprise fashion.

While I have not looked at it yet, so I canā€™t say if it would still work for what I am after. This is more of an ā€œin an ideal world I would likeā€¦ā€ type of thing. I like the idea of the inactive data being housed in a place that can be backed up and replicated using well known and reliable tools. So if BGD was able to grab data from an outside source in some way as needed (when a character logs in) even if it kept its own copy locally that it used as needed for its operation, then was able to periodically save its current state to inactive storage (even if that is locally) then that data that is saved locally within the Unity instance was able to be exported to another storage medium, or something along those lines, that would be awesome.

Again, I am mostly just throwing out what I think would be useful, lol. So if it has no ability to do anything of the sort, that is ok, I am also making a single player game currently in which I am getting to a point in which I would like to start saving data, so it could very well be useful for that as well. : D I just know I hate the idea of using scriptable objects as an actual database, it just feels so, unobtainable I guess, something like a ā€œblack boxā€ in which if something goes wrong you might be out of luck. So anything, even if for peace of mind, ends up being better in my eyes.

@MostHated , your idea makes sense to me, but BGDatabase cannot be used in this scenario- even if it can be run technically, it was not designed for this.
I personally donā€™t know any good alternative to cloud database (or some cache server) which is provided as a service and some service on top of it. Youā€™ll get

  1. Guaranteed high uptime >99.9%
  2. Auto upscaling
  3. almost zero maintenance cost
    I think all these criteria are very crucial to any server and this is not what BGDatabase can provide.
    You still can use it on players devices as a replacement to scriptable objects, but not as a replacement to a cloud database, for sure - its completely different type of software.

The tool is very rigorous, but it is too painful to manually create each field, and a method that can quickly create a large number of fields is strongly needed. I am a SLG gamer, and we have an unimaginable number of fields. Manual setup takes a lot of time.

or you can provide a way for us to build meta from Googlesheetsļ¼Œit must be greatļ¼ This is the normal workflow of mark hight numerical games.

@Tonyzhai , weā€™ll add support for the scripts in the next version.
Iā€™ve sent you a prototype in private message.
Please, let me know if something is missing.

Regarding pulling data from GoogleSheets:
Unfortunately, information about types of the fields is missing in Google Sheets.
We could retrieve field names from GoogleSheets, but you still have to fill in fields types.
If itā€™s more convenient for you, please, let me know, weā€™ll add such option.

itā€™s cool. very thanks, and i have another question.
why not to be able to use ā€œ" as FieldName. I will work with Python, I often use Python pandas for the analysis of the game number data , I usually use format like ā€œSTR_heroNameā€ .So Iā€™m in python can easily call and identifying type.
No matter what, is "
ā€ is commonly used, hope to be able to support.:slight_smile:

@Tonyzhai Iā€™ve sent you an updated package with support of underscores in private message.

@BansheeGz ,

Happy New Year ! Just to say, really loving this Asset - so flexible and so very useful.

Have left a hopefully nice review on the Store.

In terms of enhancements, Iā€™ve had a few ideas for you:-

  • it would be handy if you could move rows around, reorder them, as you can with List elements - when both tinkering with table schema/meta and when actually tinkering with table rows of data.

  • it would be neat if you could quickly re-order rows (both meta/schema and table/data) by clicking on the darker grey column names at the top to sort A-Z or Z-A?

  • insert a row above or below? (as opposed to adding a row at the end only)?

  • using configuration (for large tables) the ability to hide and unhide selected columnsā€¦?

  • conversely it would be useful to always keep on the left the ā€˜nameā€™ column (optionally - configuration) when scrolling to the right to see other columns in a data table or even meta/schema?

  • perhaps an edit box to go to a particular row in a table?

  • also, above tables and metas, an indication of the total number of rows of data?

Just some ideas to enhance productivity really.

1 Like

P.S. when Iā€™m done with some BG Database design, I think I am going to look at an integration with GameCreator by Catsoftā€¦

P.P.S. - when you create a Relationship-Single, there should be the option to have a a selection from the very same table.

  • So imagine you had a table of ā€œElementsā€ - and you created Fire and Ice. One field/column in the meta/schema could be say ā€œOpposing Elementsā€ - and the Relationship - Single should be able to refer to the table ā€œElementsā€ as well as all the other ones - so the ā€œOpposing Elementā€ to Fire would beā€¦ Ice (and vice versa) - I hope that makes senseā€¦

@Duffer123_1 ,
Happy New Year!
Thank you so much for your support, suggestions and review!
I think all your ideas make perfect sense- I scheduled the changes to version 1.3.4 (version 1.3.3 is already in the Store under review)
I will also send you a preview package as soon as itā€™s ready just in case you might be interested in reviewing it before release.

We were thinking of developing out own assets like inventory/crafting, quests/dialogs, character progression on top of the database for quite a while.
I think using database (with save/load support, GoogleSheets/Excel sync, localization etc. ) as Data Layer would be great synergy for such type of assets.
But at the end of the day, we are not sure if it really worth it.

1 Like

@BansheeGz ,

Excellent thanks (re enhancements).

I certainly wouldnā€™t discourage you developing an RPG Kit (or RPG Kit elements) Assets resting on this Asset. Makes good sense really.

1 Like

@BansheeGz ,

Another idea for an enhancement/improvement - in terms of the database Iā€™m not sure if itā€™s just the byte file or the byte file and the settings json file? Either way, it would be good to have a backup folder/files so that when you update to latest version of this Asset you donā€™t (as I just did - Doh!) overwrite all your database work. (it wasnā€™t much, thankfully)ā€¦

Some way to hive off earlier/other versions of the database and settings within the overall project and then pull them back in, in the editor?

@BansheeGz ,

Stupid question but is there a field for a List of a Unity Assets. Of particular use would be a field for a List of Resource folder Sprites ā€¦ and/orā€¦ Resource folder Prefabs ā€¦ and/orā€¦ Scene GameObjects?

@BansheeGz ,

Also wondering about support for Dictionaries? So one of the potential fields/variables with another tied to it - or could you achieve this with lots of mini nested tables? Keys and values.

really useful in gamesā€¦

@Duffer123_1
Sorry for late response, I did not receive any notifications from Unity forums by some reason, I have no idea why

Do you use the default database file for your own data?
If yes, please, create Resources folder somewhere in your project, copy Assets\BansheeGz\BGDatabase\Examples\Resources\bansheegz_database_default.bytes file to this folder using FileExplorer (copy, do not move it) and rename it to bansheegz_database.bytes
This step will create a new file, which would not be overridden
Or, alternatively, follow the steps, which are described here: http://www.bansheegz.com/BGDatabase/Setup/
You can copy database content by coping database file content.

Currently, there is no ā€œassets listā€ field. How such field can be used?
I attached little utility, which can scan Resources folder and insert new rows for each found asset if this asset is not already present in the table.
Not sure if it can be helpful

  1. Place it under Editor folder
  2. In the top menu chose Tools->Asset2Row
  3. Chose table, ā€œUnity assetā€ field and the folder with your assets and click Run
  4. Review the changes, click ā€œSaveā€ to commit or ā€œReloadā€ to revert the changes.

We honestly not sure how to implement it.
We donā€™t support generic fields, all fields have its own predefined type for its value.
And dictionary has 2 generic type parameters(for key and for value). It means we need to create a separate field for each combination and I do not think itā€™s a good solution cause there are a lot of such combinations. And even if we support generic fields, weā€™d need to use reflection to construct the values, which is super-slow.
We could try to use HashTable instead of Dictionary but it introduces new problems, like not type-safe code, producing garbage due to boxing/unboxing. So Iā€™m not sure how we could do it properly.
Yes, a nested table can mimic the dictionary field. You can iterate all nested records and construct your own dictionary with any key/value

4096177ā€“358087ā€“Asset2Row.cs (7.43 KB)

1 Like