BG Database (inMemory database | Excel/Google Sheets syncing | CodeGen | Save/Load support)

Sorry, it was a bug in GUI code :unamused:
Thank you for the screenshot, it helped us to find the bug quickly
Please, upgrade BGDatabase package (I will DM a link to you)
If there are any other issues, please let me know.

Hi, this is because all Unity asset fields are read-only.
We store asset references (addresses) inside the database (not the assets) and in runtime there is no way to figure out the asset’s address, unfortunately.

There are 2 workaround solutions:

  1. If your code is placed in Editor assembly (under Editor folder), you can use field.SetAsset method to assign the asset to a row. Here is the full code
Example 1: setting audio clip in Editor assembly using the asset
using BansheeGz.BGDatabase;
using BansheeGz.BGDatabase.Editor;
using UnityEditor;
using UnityEngine;

public class TestCode 
{
    [MenuItem("Tools/[Test] Set audio clip")]
    public static void Run()
    {
        //[Attention] change values 
        string tableName = "Ability";
        AudioClip testClip = AssetDatabase.LoadAssetAtPath<AudioClip>("Assets/Temp/Resources/Audio/1.wav");
        var id = 0;

        BGMetaEntity testTable = BGRepo.I[tableName];
        Debug.Log($"Got table: {tableName}");
        int entityCount = testTable.CountEntities;
        Debug.Log($"Table {tableName} has {entityCount} entities");

        if (id < entityCount)
        {
            BGEntity testRow = testTable.GetEntity(id);
            Debug.Log($"Got entity at index {id}");

            // Set the value
            //[Attention] modified code
            // testRow.Set("AudioFile", testClip); <- this will not work
            BGFieldUnityAudioClip audioField = (BGFieldUnityAudioClip)testTable.GetField("AudioFile");
            audioField.SetAsset(testRow.Index, testClip);
            
            Debug.Log($"Test: Set AudioFile to {testClip.name} at index {id}");

            // *** Read back the value ***
            AudioClip readBackClip = testRow.Get<AudioClip>("AudioFile");

            if (readBackClip != null)
            {
                Debug.Log($"Test: Read back AudioFile: {readBackClip.name}");
                // Verify if the read clip is the same as the set clip
                if (readBackClip == testClip)
                {
                    Debug.Log("Test: Read back clip matches set clip");
                }
                else
                {
                    Debug.LogError("Test: Read back clip DOES NOT match set clip!");
                    Debug.Log($"Set clip name: {testClip.name}");
                    Debug.Log($"Read back clip name: {readBackClip.name}");
                }
            }
            else
            {
                Debug.LogError("Test: Could not read back AudioFile!");
            }
        }
    }

}

  1. If you know the asset’s address, you can use field.SetStoredValue method to pass this address to a row. This method works in both Editor assembly and in runtime assembly. Here is the full code
Example 2: setting audio clip using asset's address
using BansheeGz.BGDatabase;
using UnityEngine;

public class SetAudioClip : MonoBehaviour
{
    void Start()
    {
        //[Attention] change values 
        string tableName = "Ability";
        string assetAddress = "Audio/1"; // <- path, relative to Resources folder without extension
        AudioClip testClip = Resources.Load<AudioClip>(assetAddress); // <- there is no need to load audioClip here, we load it to make code below compile and run properly 
        var id = 0;

        BGMetaEntity testTable = BGRepo.I[tableName];
        Debug.Log($"Got table: {tableName}");
        int entityCount = testTable.CountEntities;
        Debug.Log($"Table {tableName} has {entityCount} entities");

        if (id < entityCount)
        {
            BGEntity testRow = testTable.GetEntity(id);
            Debug.Log($"Got entity at index {id}");

            // Set the value
            //[Attention] modified code
            // testRow.Set("AudioFile", testClip); <- this will not work
            BGFieldUnityAudioClip audioField = (BGFieldUnityAudioClip)testTable.GetField("AudioFile");
            audioField.SetStoredValue(testRow.Index, assetAddress);
            
            Debug.Log($"Test: Set AudioFile to {testClip.name} at index {id}");

            // *** Read back the value ***
            AudioClip readBackClip = testRow.Get<AudioClip>("AudioFile");

            if (readBackClip != null)
            {
                Debug.Log($"Test: Read back AudioFile: {readBackClip.name}");
                // Verify if the read clip is the same as the set clip
                if (readBackClip == testClip)
                {
                    Debug.Log("Test: Read back clip matches set clip");
                }
                else
                {
                    Debug.LogError("Test: Read back clip DOES NOT match set clip!");
                    Debug.Log($"Set clip name: {testClip.name}");
                    Debug.Log($"Read back clip name: {readBackClip.name}");
                }
            }
            else
            {
                Debug.LogError("Test: Could not read back AudioFile!");
            }
        }

    }
}

Please, let me know if you have any questions

Thanks! That worked.

1 Like

I have more questions and possible bug reports:

  1. is there an API that can find a field (by name) that traverses into nested tables?
  2. how to change the path of the backup directory?
  3. how to change the name of the database path (and file)? NOTE: I tried renaming the file using unity
    but when I started unity the next day BG couldn’t find the database file. I renamed it back
    to the default and BG was able to find it again.
  4. doing a save all or save repro. Once it happens it continues to happen even if unity is restarted.
    happens even if I delete “Instant2” directory.
    [Exception] [FileSystem] [RemoveDirectoryInternal] IOException: Access to the path ‘\?\C:\Users\me\OneDrive\Documents\BGDatabaseAutoBackups\ScifiRpg_534533189\Instant2’ is denied.
    [Exception] IOException: Access to the path ‘\?\C:\Users\me\OneDrive\Documents\BGDatabaseAutoBackups\ScifiRpg_534533189\Instant2’ is denied.
    FileSystem.RemoveDirectoryInternal() at :0
    FileSystem.RemoveDirectoryRecursive() at :0
    FileSystem.RemoveDirectory() at :0
    Directory.Delete() at :0
    BGBackups.BackUpInstant() at :0
    BGBackups.BackUpIfNeeded() at :0
    Debug.LogException()
    BGBackups.BackUpIfNeeded()
    BGRepoSaver.SaveInternal()
    BansheeGz.BGDatabase.Editor.<>c__DisplayClass11_0.b__0()
    BansheeGz.BGDatabase.Editor.<>c__DisplayClass108_0.b__0()
    EditorApplication.Internal_CallUpdateFunctions()
  5. code generated by BG: this could be my issue - I have a field called “Name”. I guess I could the
    provided “name” field instead?
    [CS0114] ‘Characters.Name’ hides inherited member ‘BGEntity.Name’. To make the current
    member override that implementation, add the override keyword. Otherwise add the new keyword.
    Compiler Warning at :39 column 24
    37: set => _name[Index] = value;
    38: }
    –>39: public System.String Name
    40: {
    41: get => _Name[Index];

table.GetField(fieldName)
Here are some code examples of getting nested field/entities/tables

SCREENSHOT: Table structure

Without using CodeGen add-on
using System.Collections.Generic;
using BansheeGz.BGDatabase;
using UnityEngine;

public class Temp : MonoBehaviour
{
    private void Start()
    {
        //get Player table
        BGMetaEntity playerTable = BGRepo.I.GetMeta("Player");

        //for future use
        BGEntity firstPlayerRow = playerTable.GetEntity(0);

        //get nested field
        BGFieldNested playerAbilityField = (BGFieldNested)playerTable.GetField("PlayerAbility");

        //get nested entities for the 1st "Player" row via nested field (Option #1)
        List<BGEntity> playerAbilityEntities1 = playerAbilityField[firstPlayerRow.Index];

        //get nested entities for the 1st "Player" row via parent row (Option #2)
        List<BGEntity> playerAbilityEntities2 = firstPlayerRow.Get<List<BGEntity>>("PlayerAbility");

        //get nested table "PlayerAbility" via nested field
        BGMetaNested playerAbilityTable = playerAbilityField.NestedMeta;

        //get parent table "Player" via nested table
        BGMetaEntity playerTable2 = playerAbilityTable.Owner;
    }
}
With CodeGen add-on (classes prefix=D_)
using System.Collections.Generic;
using BansheeGz.BGDatabase;
using UnityEngine;

public class Temp : MonoBehaviour
{
    private void Start()
    {
        //get Player table
        BGMetaRow playerTable = D_Player.MetaDefault;

        //for future use
        D_Player firstPlayerRow = D_Player.GetEntity(0);

        //get nested field
        BGFieldNested playerAbilityField = D_Player._PlayerAbility;

        //get nested entities for the 1st "Player" row via nested field (Option #1)
        List<BGEntity> playerAbilityEntities1 = playerAbilityField[firstPlayerRow.Index];

        //get nested entities for the 1st "Player" row via parent row (Option #2)
        List<D_PlayerAbility> playerAbilityEntities2 = firstPlayerRow.PlayerAbility;

        //get nested table "PlayerAbility" via nested field
        BGMetaNested playerAbilityTable = playerAbilityField.NestedMeta;

        //get parent table "Player" via nested table
        BGMetaEntity playerTable2 = playerAbilityTable.Owner;
    }
}

Please, update the asset (I will DM the links to you)
Use “Change->Set custom backup folder”

SCREENSHOT: custom backup folder

After setting the folder, click on “Save all” and “Refresh data”, new backup files should appear in the table below (if the number of backups is greater than 0)

SCREENSHOT: making backup

We do not recommend renaming/moving database file
You can use a custom loader to move the database file to a custom location, the latest build also supports custom database file name. When a custom loader is used, database is not loaded automatically in runtime, you need to load database content and pass it to BGRepo class before accessing database.

The simplest database loader implementation
using BansheeGz.BGDatabase;
using UnityEngine;

public class DatabaseLoader : MonoBehaviour
{
    //assign database file to this field
    public TextAsset Database;
    
    //pass database content 
    private void Awake() => BGRepo.SetDefaultRepoContent(Database.bytes);
}

It looks like an issue with permissions
Please, try to assign a custom database backups folder

Both “name” and “Name” are used for system “name” field, Name property is used as a shortcut to “name” field value when code generation is not used. Probably, it’s possible to have both “name” and “Name” fields, but when code generation is used, the generated “Name” property for “Name” field collides with Name property from base class BGEntity. We will add a check in new release to prevent the creation of “Name” field.
Please, rename your “Name” field or use system “name” field instead

regarding #4 (backup error): I forgot to mention that I deleted the contents of “BGDatabaseAutoBackups” and tried again. I also updated the permissions on “BGDatabaseAutoBackups” to give everybody full control. no errors until “instant2”.

Thank you for your help!

You likely have three instant backups (Instant0, Instant1, Instant2), the rotation process tries to delete the oldest backup (Instant2) and access denied exception is thrown. I hope the custom backup folder solved the issue

DB design question (boils down to runtime modifications I think).

a character in my RPG can have many groups of fields:
attributes: strength, intelligence, etc.
equipment: weapons, armor, gadgets, etc.
some equipment can have mods attached.
and more…

some of the data is shared and some is not.

my basic question is how are modifications generally handled (especially with the shared data)?

as a simple example, I have a “Characters” table with an associated table called “Attributes”.
I thought that having “Attributes” be a table would make them easier to manage, etc.
I’ve tried two approaches:
1) “Attributes” table that “Characters” has a relation to.
2) “Attributes” table is a nested table of “Characters”.
problems:
#1 can’t make changes to “Attributes” without effecting other rows.
#2 nested tables not sharable (?). if I want to create another table “Foo”
that also needs an “Attributes” nested table I could not figure out how to do it.

How should my DB be structured for modifications (read-only data isn’t an issue)?
I would like to reduce redundancy as much as possible to make edits and changes easier
in the future.

based on reading through the docs and support site it sounds like creating new rows for the modifications is the way to go? using the example I mention above, new rows would be added to the “Attributes” table for each character?

  1. Indeed, nested tables are not sharable unfortunately, so the current solution is to create 2 nested tables with similar fields
SCREENSHOT: db structure

CharacterAttributes and FooAttributes tables contain information, which is specific to a particular Character and Foo rows and Attributes table contains information, which is specific to a particular attribute and shared among all rows, referencing this attribute.
The view can be created for CharacterAttributes and FooAttributes tables and code generator will generate an interface for this view, the rows from both tables implement this interface, so these rows can be processed using shared code if necessary. The view does not support relationships, but they can be added manually in the C# code very easily.

  1. Actually, a sharable nested-like table can be implemented like so
SCREENSHOT: sharable nested-like table

but editing rows for such table is difficult

SCREENSHOT: sharable nested-like table2

We can add a new column to both “Character” and “Foo” tables with the same behavior as a “nested” field editor, which will allow editing related “AttributesNestedLikeTable” rows in the same way the nested rows are edited

SCREENSHOT: new columns

I think it can be a useful feature, please, let me know if you are interested to receive this update as soon as it’s ready

If we use this example project as a starting point

  1. Most probably you want to handle your mods as usual items, so you need to add “Mods” table and add it to the “Item” view
SCREENSHOT: Mods table

Only required fields are shown, you can add other fields as well.
Instead of a single Mods table, two separate tables can be used if it makes sense, one table for weapon mods and the other one for armor mods. If you need to reference a row from either armor mods or weapon mods table, create a view “ModsView”, include both table to this view and use “viewRelationSingle” field, referencing “ModsView” view.

  1. A row in Mods table, just like a row in other tables, included into “Item” view, represents an item prototype and there are 3 different tables in the example project that represent a physical item. Inventory row represents an item in inventory, ChestItems row represents an item in the chest and TraderItems row represents an item in trader’s inventory. In your particular case, having multiple tables with physical items does not make sense, because you have to add a nested “InstalledMods” field to every table with physical items.
    So maybe the approach, I described above (with nested-like table) will work better.
    Instead of having multiple tables with physical items, you can have one single, regular “PhysicalItems” table and “InstalledMods” nested field added to this table.
    Here is the change list:
    ChestItems, TraderItems and Inventory tables are removed and “PhysicalItems” table is added.
    Chests, Traders and Player tables are added to “ItemsOwner” view and “PhysicalItems” table has viewRelationSingle “owner” field, referencing a row from this view to identify where the item is located. “PhysicalItems.count” field holds the number of items, and “PhysicalItems.item” references an item prototype. “InstalledModes” is a nested table with a list of installed mods
SCREENSHOT:PhysicalItems

Editing “PhysicalItems” rows is currently difficult, but if we add additional columns like I described earlier to Chests, Traders and Player tables it should be similar to editing nested rows. In fact, we can provide you with an example project once new features are added. We will need this example project anyway to make sure that the approach is viable.

Sorry, I have to correct myself

This is not true, an Inventory row represents an inventory slot, there are 20 such slots. If the item reference is not assigned, the slot is empty.
If we want to keep the slots, we need to keep “Inventory” table and include it to “ItemsOwner” view instead of “Player” table and make sure the slot can have only one item

looks like I’ve fallen in the deep end. I need to read you answers a few more times and consult the documentation so I can ask good follow-up questions.
I’ll also write up a more detailed description of what I want in the DB.

point of clarification: when I said “modifications” I wasn’t referring to item “mods” (I will have those as well). my intent was more along the lines of “entity” (generically speaking) creation at runtime and adding rows to tables for it. I’ll try to explain better in a future post.

Ok, we’re currently in the process of adding new columns. Once that’s complete, we’ll add an article to our docs, focused on how data can be organized.

May I ask if view fields do not support the localization type?

Yes, sorry, current implementation does not support adding relationships or localization fields to a view, because of technical difficulties, but there is an easy workaround.
These view fields are used only by code generators, and there is an easy way to add missing fields manually.

For example, for the view with a single name field

View with single field

CodeGen add-on’s code generator with the following settings

CodeGen addon's settings

generates this C# interface

C# interface for the view
public partial interface D_view  : BGAbstractEntityI
{
	System.String name {get; set;}
}

If you create a new C# script, the name does not matter, let’s say GenExtensions.cs and add the following code to it

The code, added manually to generated view interface
public partial interface D_view  
{
    System.String en {get; set;}
    System.String fr {get; set;}
    System.String LocalizedValue {get; set;}
}

These properties will be added to the generated view interface and the code will compile if all tables, included into the view, indeed have such properties. The property names may vary depending on which locales you have. The same trick can be used to add custom properties or methods to generated classes. Please, let me know if you have any questions.

Would it be possible for you to create a simple example file for me to download? I would really appreciate it, as I’d like to see how it’s done.

Wait,No need, I just figured it out. It worked, thanks anyway!

1 Like

Just had a weird error in a build. Game works fine in editor but I get an ENUM not found. Here’s the log dump showing the DB error is the first so presumably not dependent on something else:

Found 1 interfaces on host : 0) 192.168.1.109
Player connection [13120]  Target information:

Player connection [13120]  * "[IP] 192.168.1.109 [Port] 55000 [Flags] 2 [Guid] 1218799568 [EditorId] 1691803282 [Version] 1048832 [Id] WindowsPlayer(2,DESKTOP-GJE8805) [Debug] 0 [PackageName] WindowsPlayer [ProjectName] Adventures" 

Player connection [13120] Started UDP target info broadcast (1) on [225.0.0.222:54997].

[PhysX] Initialized MultithreadedTaskDispatcher with 24 workers.
Initialize engine version: 2022.3.53f1 (df4e529d20d3)
[Subsystems] Discovering subsystems at path D:/_Programming Stuff/Unity/output_Data/Adventures/Adventures_Data/UnitySubsystems
GfxDevice: creating device client; threaded=1; jobified=1
Direct3D:
    Version:  Direct3D 11.0 [level 11.0]
    Renderer: NVIDIA GeForce GTX 750 Ti (ID=0x1380)
    Vendor:   NVIDIA
    VRAM:     4044 MB
    Driver:   32.0.15.6094
<RI> Initializing input.

<RI> Input initialized.

<RI> Initialized touch support.

[PhysX] Initialized MultithreadedTaskDispatcher with 24 workers.
UnloadTime: 0.557800 ms
**** ERROR HERE ****
**BGException: Can not deserialize field Items.Size: enum type WEAPON_SIZE is not found!**
  at BansheeGz.BGDatabase.BGFieldEnumA`1[T].GetEnumType (BansheeGz.BGDatabase.BGField field, System.Type underlyingType, System.String typeName) [0x00000] in <00000000000000000000000000000000>:0 
  at BansheeGz.BGDatabase.BGFieldEnumA`1[T].ConfigFromBytes (System.ArraySegment`1[T] config) [0x00000] in <00000000000000000000000000000000>:0 
  at BansheeGz.BGDatabase.BGField.FromBinary (BansheeGz.BGDatabase.BGBinaryReader binder, BansheeGz.BGDatabase.BGMetaEntity meta) [0x00000] in <00000000000000000000000000000000>:0 
  at BansheeGz.BGDatabase.BGRepoBinaryV8+<>c__DisplayClass5_0.<ReadMeta>b__0 () [0x00000] in <00000000000000000000000000000000>:0 
  at BansheeGz.BGDatabase.BGBinaryReader.ReadArray (System.Action action) [0x00000] in <00000000000000000000000000000000>:0 
  at BansheeGz.BGDatabase.BGRepoBinaryV8.ReadMeta (BansheeGz.BGDatabase.BGBinaryReader binder, System.Boolean multithreaded, BansheeGz.BGDatabase.BGMetaEntity meta, BansheeGz.BGDatabase.BGMultiThreadedLoader multiThreadedLoader) [0x00000] in <00000000000000000000000000000000>:0 
  at BansheeGz.BGDatabase.BGRepoBinaryV8+<>c__DisplayClass4_0.<ReadMetas>b__0 () [0x00000] in <00000000000000000000000000000000>:0 
  at BansheeGz.BGDatabase.BGBinaryReader.ReadArray (System.Action action) [0x00000] in <00000000000000000000000000000000>:0 
  at BansheeGz.BGDatabase.BGRepoBinaryV8.ReadMetas (BansheeGz.BGDatabase.BGBinaryReader binder, BansheeGz.BGDatabase.BGRepo repo, System.Boolean multithreaded) [0x00000] in <00000000000000000000000000000000>:0 
  at BansheeGz.BGDatabase.BGRepoBinaryV8.Read (System.Byte[] dataBytes) [0x00000] in <00000000000000000000000000000000>:0 
  at BansheeGz.BGDatabase.BGRepo.Load (System.Byte[] data) [0x00000] in <00000000000000000000000000000000>:0 
  at BansheeGz.BGDatabase.BGRepo.Load (BansheeGz.BGDatabase.BGRepoLoadingContext context) [0x00000] in <00000000000000000000000000000000>:0 
  at BansheeGz.BGDatabase.BGRepo.get_I () [0x00000] in <00000000000000000000000000000000>:0 
  at BansheeGz.BGDatabase.BGCodeGenUtils.GetMeta[T] (BansheeGz.BGDatabase.BGId metaId, System.Action onUnload) [0x00000] in <00000000000000000000000000000000>:0 
  at DB_Actors.get_MetaDefault () [0x00000] in <00000000000000000000000000000000>:0 
  at DB_Actors.get_CountEntities () [0x00000] in <00000000000000000000000000000000>:0 
  at EntityManager.InitPools () [0x00000] in <00000000000000000000000000000000>:0

Enum is defined in my DataCache.cs script:

public enum WEAPON_SIZE {
	SINGLE,			// one hand, can wield in either
	BOTH,			// visually placed in one hand, occupies two hands
	DOUBLE,			// two handed, visually placed in both hands, occupies both
}

And is added fine to my database:

Edit: It built fine last week with the same DB structure. I’ve been working on replacing my fixed sprites with animated sprites made from body parts using Animancer. As such I’ve changed the database format for my ‘actor’ type to load multiple sprites.

Most probably it happens because of code stripping
If you do not use this enum in your code, the compiler may exclude it from the build
Try to use Preserve attribute on your enum type to prevent the compiler from removing enum type from the build or use link.xml file
Also, there is a player setting that can help

Preserve attribute
[Preserve]
public enum WEAPON_SIZE {
	SINGLE,			// one hand, can wield in either
	BOTH,			// visually placed in one hand, occupies two hands
	DOUBLE,			// two handed, visually placed in both hands, occupies both
}
1 Like

Another reason why this can happen- if you put your enum class in Editor assembly (under Editor folder) and if you do not use code generator
With code generator, there should be compiling error in this case, but without using code generator, it can compile, but the exception is thrown in runtime
The solution is to move enum type to runtime assembly

Thanks for the quick response! I hadn’t got around to using the Enum in code yet so it was being stripped as you said. Your quick reply means I have a working build to test this evening. Muchas gracias!!

1 Like