BinaryFormater is losing saved data on mobile

[ODINKONG]

Hi everyone, I’m really having trouble with this one. We just released our game on Android and IOS several users are reporting that their saved data has been cleared after opening and closing the game over 20 times in some cases. Even though the saving and loading is successful most of the time. I haven’t been able to repeat this issue and I can’t find any commonalities. Some players have never experienced it on the same version of the game after playing for 30 hours.

If anyone has any leads on this or hints on how I would track this down or repeat the issue I’m open to anything. This is really making our release tough, even though the rest of it is going well.
Any help is deeply appreciated.

Here is some of the code I am using for saving. Let me know if anything sticks out at you as being very wrong. Essentailly if the loading script fails in anyway to find the data the game will assume that this is a fresh install and clear all variables to get ready for a new game. So I’m sure whatever is going on is cuasing the save file that is saving correctly to either be missnamed, missplaced or corrupted so that it wont load. My guess is corrupted but again I can’t repeat this even if I force quit in very save heavy parts of the game. I’m at a total loss on what’s going on here. Someone suggested that it might be Android autobackup, but then the IOS auto backup would also need to be causing and issue. Would that be the sort of thing that could clear all of the data for my app or corrupt it?

Saving function

 public void SaveGameStats()
    {
        string saveFileName = "GameStats.dat";
        if (GameManager.instance.hardCoreMode)
        {
            saveFileName = "GameStatsH.dat";
        }
        BinaryFormatter bf = new BinaryFormatter();
        FileStream file = null;
        if (File.Exists(Path.Combine(Application.persistentDataPath, saveFileName)))
        {
            File.Delete(Path.Combine(Application.persistentDataPath, saveFileName));
        }
        savableGameStats.currentSuccesiveMission = currentSuccessiveMission;
        TotalPlayTime += Time.realtimeSinceStartup - lastTimeSinceStartSeconds; // get the number of seconds the game has been running and convert it to a timespan
        lastTimeSinceStartSeconds = Time.realtimeSinceStartup;
        savableGameStats.enemStatListY.Clear(); // clear list to make sure its empty
        for (int y = 0; y < 7; y++)
        { // add seven 0's to enem stat list y so that the enem stat list can use the keys as the x wihtout throwing an out of range error
            savableGameStats.enemStatListY.Add(0);
        }
        for (int i = 0; i < enemyStatAddList.Count; i++)
        {
            savableGameStats.enemStatListY[i] = enemyStatAddList[i].y;
        }
        savableGameStats.hardCoreMode = hardCoreMode;
        if (savableAccountStats.adFreeVersion == true)
        {
            savableAccountStats.adFreeVersion = true;
        }
        savableGameStats.sfxVolume = sfxVolume;
        savableGameStats.musicVolume = musicVolume;
        savableGameStats.HCPPassiveHeal = HCPPassiveHeal;
        savableGameStats.HCPMissionReRolls = HCPMissionReRolls;
        savableGameStats.missionReRolls = missionReRolls;
        savableGameStats.HCPInjuryHeal = HCPInjuryHeal;
        savableGameStats.BCPBuffMirror = BCPBuffMirror;
        savableGameStats.BCPAbilityPower = BCPAbilityPower;
        savableGameStats.BCPAbilityCooldown = BCPAbilityCooldown;
        savableGameStats.HCPAbilityPower = HCPAbilityPower;
        savableGameStats.unitDeadSaveChance = unitDeadSaveChance;
        savableGameStats.itemDeadSaveChance = JunkyardSalvageFromUnits;
        savableGameStats.currentChestType = currentChestType;
        savableGameStats.currentChestMagicFind = currentChestMagicFind;
        savableGameStats.currentChestItemNumber = currentChestItemNumber;
        savableGameStats.currentArmy = currentArmy;
        savableGameStats.arenaBet = CurrentArenaBet;
        savableGameStats.arenaRarity = ArenaRunRarirty;
        savableGameStats.isInArena = isInArena;
        savableGameStats.energy = energy;
        savableGameStats.bloodForgeStartingLevel = bloodForgeStartingLevel;
        savableGameStats.savedBloodForgeItemId = currentBloodForgeItemId;
        savableGameStats.bloodForgeSuccessPercentage = bloodForgeSuccessPercent;
        savableGameStats.isInBloodForge = isInBloodForge;
        savableGameStats.buildingBlueprintMissionNumber = buildingBluePrintMissionNumber;
        savableGameStats.levelReward = levelReward;
        savableGameStats.SCPResurection = SCPResurection;
        savableGameStats.SCPExtraCp = SCPExtraCp;
        savableGameStats.SCPITemPower = SCPITemPower;
        savableGameStats.battleNumber = savedBattleNumber;
        savableGameStats.lastTime = currentTime.ToBinary();
        if (energyEmptyTime != null)
        {
            savableGameStats.energyEmptyTime = energyEmptyTime.ToBinary();
        }
        savableGameStats.totalPlayTime = TotalPlayTime;
        savableGameStats.missionList = missionList;
        savableGameStats.savedArmylist = currentArmyList;
        savableGameStats.skillList = skillList;
        savableGameStats.skillCostList = skillCostList;
        savableGameStats.savedQuestList = currentQuestList;
        savableGameStats.IAttackBuff = IAttackBuff;
        savableGameStats.IdefenseBuff = IdefenseBuff;
        savableGameStats.RattackSpeedBuff = RattackSpeedBuff;
        savableGameStats.RaccuracyBuff = RaccuracyBuff;
        savableGameStats.BhealthBuff = BhealthBuff;
        savableGameStats.HdefenseBuff = HdefenseBuff;
        savableGameStats.SHealthBuff = SHealthBuff;
        savableGameStats.generalPortraitNum = generalPortaitNum;
        savableGameStats.generalIsMale = generalIsMale;
        savableGameStats.generalName = GeneralName;
        savableGameStats.ICPAttackSpeed = ICPAttackSpeed;
        savableGameStats.ICPBleed = ICPBleed;
        savableGameStats.commerceYield = ICPGoldPerMission;
        savableGameStats.HCPSalvage = HCPSalvage;
        //        savableGameStats.BCParmyCost = BCParmyCost;
        savableGameStats.BCPBuildingSlots = BCPBuildingSlots;
        savableGameStats.BCPBuildingCostReduction = BCPBuildingCostReduction;
        savableGameStats.totalKills = totalKills;
        savableGameStats.animatedTotalKills = animatedTotalKills;
        savableGameStats.packItemChance = packItemChance;
        //        savableGameStats.armySizeList = currentArmySizeList;
        savableGameStats.armySize = armySize;
        savableGameStats.bloodLevel = bloodLevel;
        savableGameStats.gold = gold;
        savableGameStats.salvage = salvage;
        savableGameStats.hasAllBuildings = hasAllBuildings;
        savableGameStats.SCPMagicFind = SCPMagicFind;
        savableGameStats.SCPLifeSteal = SCPLifeSteal;
        savableGameStats.BattlesWon = battlesWon;
        savableGameStats.ICp = ICp;
        savableGameStats.totalCpEver = totalCpEver;
        savableGameStats.RCp = RCp;
        savableGameStats.RCPMaxCollectionSize = RCPMaxCollectionSize;
        savableGameStats.BCp = BCp;
        savableGameStats.HCp = HCp;
        savableGameStats.SCp = SCp;
        savableGameStats.Gp = Gp;
        savableGameStats.packCost = packCost;
        savableGameStats.idcount = idCount;
        savableGameStats.firstgame = firstGameEver;
        savableGameStats.fame = fame;
        savableGameStats.maxAct = maxAct;
        savableGameStats.fameTillNext = fameTillNextAct;
        savableGameStats.campaignMission = currentCampaignMission;
        savableGameStats.RCPCritCD = RCPCritCD;
        savableGameStats.goToCustomizer = goToCustomizer;
        savableGameStats.openUnitPanel = openUnitPanel;
        savableGameStats.dragUnitsToArmy = dragUnitsToArmy;
        savableGameStats.goToItems = goToItems;
        savableGameStats.openItemsPanel = openItemsPanel;
        savableGameStats.clickOpenUnit = clickOpenUnit;
        savableGameStats.dragItemsToUnits = dragItemsToUnits;
        savableGameStats.buildingsTip = buildingsTip;
        savableGameStats.civScreenTutTip = civScreenTutTip;
        savableGameStats.wordMapTutTip = wordMapTutTip;
        savableGameStats.shopScreenTutTip = shopScreenTutTip;
        savableGameStats.goToCpTutTip = goToCpTutTip;
        savableGameStats.goToShopTutTip = goToShopTutTip;
        savableGameStats.salvageShopTutTip = salvageShopTutTip;
        savableGameStats.nextActTutTip = nextActTutTip;
        savableGameStats.injuryTutTip = injuryTutTip;
        savableGameStats.retireFarmerTutTip = retireFarmerTutTip;
        file = File.Create(Path.Combine(Application.persistentDataPath, saveFileName));
        bf.Serialize(file, savableGameStats);
        file.Close();
        SaveAccount(); // this is to make sure the account data is saved frequently
    }

Loading Function

 public void LoadGameStats()
    {
        string saveFileName = "GameStats.dat";
        if(GameManager.instance.hardCoreMode)
        {
            saveFileName = "GameStatsH.dat";
        }
        if (File.Exists(Path.Combine(Application.persistentDataPath, saveFileName)))
        {
            BinaryFormatter bf = new BinaryFormatter();
            FileStream file = File.Open(Path.Combine(Application.persistentDataPath, saveFileName), FileMode.Open);
            GameStats loadedStats = (GameStats)bf.Deserialize(file);
            file.Close();
            savableGameStats = loadedStats;
            clearEnemStatList(); // clear list first to make sure we start clean
            for (int i = 0; i < savableGameStats.enemStatListY.Count; i++)
            { // for all the values of the stats
                addToEnemStatList(new Vector2(i + 1, savableGameStats.enemStatListY[i]));
            }
            if (savableGameStats.adFreeVersion == true)
            {
                adFreeVersion = true;
            }
            if (savableAccountStats.adFreeVersion == true) //this is a patch for people who bought the game before account stats exsisted
            {
                adFreeVersion = savableAccountStats.adFreeVersion;
            }
            hardCoreMode = savableGameStats.hardCoreMode;
            energy = savableGameStats.energy;
            sfxVolume = savableGameStats.sfxVolume;
            musicVolume = savableGameStats.musicVolume;
            musicSource.volume = musicVolume;
            HCPPassiveHeal = savableGameStats.HCPPassiveHeal;
            HCPMissionReRolls = savableGameStats.HCPMissionReRolls;
            missionReRolls = savableGameStats.missionReRolls;
            HCPInjuryHeal = savableGameStats.HCPInjuryHeal;
            BCPAbilityPower = savableGameStats.BCPAbilityPower;
            HCPAbilityPower = savableGameStats.HCPAbilityPower;
            BCPAbilityCooldown = savableGameStats.BCPAbilityCooldown;
            BCPBuffMirror = savableGameStats.BCPBuffMirror;
            SCPResurection = savableGameStats.SCPResurection;
            SCPExtraCp = savableGameStats.SCPExtraCp;
            SCPITemPower = savableGameStats.SCPITemPower;
            unitDeadSaveChance = savableGameStats.unitDeadSaveChance;
            JunkyardSalvageFromUnits = savableGameStats.itemDeadSaveChance;
            currentChestType = savableGameStats.currentChestType;
            currentChestMagicFind = savableGameStats.currentChestMagicFind;
            currentChestItemNumber = savableGameStats.currentChestItemNumber;
            currentArmy = savableGameStats.currentArmy;
            bloodForgeStartingLevel = savableGameStats.bloodForgeStartingLevel;
            currentBloodForgeItemId = savableGameStats.savedBloodForgeItemId;
            isInArena = savableGameStats.isInArena;
            buildingBluePrintMissionNumber = savableGameStats.buildingBlueprintMissionNumber;
            isInBloodForge = savableGameStats.isInBloodForge;
            bloodForgeSuccessPercent = savableGameStats.bloodForgeSuccessPercentage;
            CurrentArenaBet = savableGameStats.arenaBet;
            ArenaRunRarirty = savableGameStats.arenaRarity;
            currentSuccessiveMission = savableGameStats.currentSuccesiveMission;
            //            currentSuccessiveMission = null;
            armySize = savableGameStats.armySize;
            levelReward = savableGameStats.levelReward;
            ICPGoldPerMission = savableGameStats.commerceYield;
            HCPSalvage = savableGameStats.HCPSalvage;
            SCPLifeSteal = savableGameStats.SCPLifeSteal;
            SCPMagicFind = savableGameStats.SCPMagicFind;
            ICPAttackSpeed = savableGameStats.ICPAttackSpeed;
            ICPBleed = savableGameStats.ICPBleed;
            battlesWon = savableGameStats.BattlesWon;
            hasAllBuildings = savableGameStats.hasAllBuildings;
            //            BCParmyCost = savableGameStats.BCParmyCost;
            totalKills = savableGameStats.totalKills;
            animatedTotalKills = savableGameStats.animatedTotalKills;
            BCPBuildingCostReduction = savableGameStats.BCPBuildingCostReduction;
            BCPBuildingSlots = savableGameStats.BCPBuildingSlots;
            packItemChance = savableGameStats.packItemChance;
            RCPCritCD = savableGameStats.RCPCritCD;
            RCPMaxCollectionSize = savableGameStats.RCPMaxCollectionSize;
            //            if (loadedStats.armySizeList != null) { // if we load a null list it will break the game and the list
            //                currentArmySizeList = savableGameStats.armySizeList;
            //            }
            if (loadedStats.savedArmylist != null)
            {
                currentArmyList = savableGameStats.savedArmylist;
            }
            //            if (loadedStats.armyTypes != null) { // if we load a null list it will break the game and the list
            //                armyTypes = savableGameStats.armyTypes;
            //            }
            //            if (loadedStats.armyNameList != null) { // if we load a null list it will break the game and the list
            //                armyNames = savableGameStats.armyNameList;
            //            }
            if (loadedStats.skillList != null)
            { // if we load a null list it will break the game and the list
                skillList = savableGameStats.skillList;
            }
            if (loadedStats.skillCostList != null)
            { // if we load a null list it will break the game and the list
                skillCostList = savableGameStats.skillCostList;
            }
            else
            {
                for (int i = 0; i < 25; i++)
                {
                    skillCostList.Add(0);
                }
            }
            if (loadedStats.savedQuestList != null)
            { // if we load a null list it will break the game and the list
                currentQuestList = savableGameStats.savedQuestList;
            }
            if (loadedStats.missionList != null)
            { // if we load a null list it will break the game and the list
                missionList = savableGameStats.missionList;
            }
            else
            {
                missionList.MissionListStart();
            }
            savedBattleNumber = savableGameStats.battleNumber;
            battleNumber = savableGameStats.battleNumber;
            lastTime = currentTime;
            TotalPlayTime = savableGameStats.totalPlayTime;
            IAttackBuff = savableGameStats.IAttackBuff;
            IdefenseBuff = savableGameStats.IdefenseBuff;
            RattackSpeedBuff = savableGameStats.RattackSpeedBuff;
            RaccuracyBuff = savableGameStats.RaccuracyBuff;
            BhealthBuff = savableGameStats.BhealthBuff;
            HdefenseBuff = savableGameStats.HdefenseBuff;
            SHealthBuff = savableGameStats.SHealthBuff;
            generalPortaitNum = savableGameStats.generalPortraitNum;
            generalIsMale = savableGameStats.generalIsMale;
            GeneralName = savableGameStats.generalName;
            bloodLevel = savableGameStats.bloodLevel;
            salvage = savableGameStats.salvage;
            gold = savableGameStats.gold;
            packCost = savableGameStats.packCost;
            ICp = savableGameStats.ICp;
            totalCpEver = savableGameStats.totalCpEver;
            RCp = savableGameStats.RCp;
            BCp = savableGameStats.BCp;
            HCp = savableGameStats.HCp;
            SCp = savableGameStats.SCp;
            Gp = savableGameStats.Gp;
            firstGameEver = savableGameStats.firstgame;
            idCount = savableGameStats.idcount;
            fame = savableGameStats.fame;
            maxAct = savableGameStats.maxAct;
            fameTillNextAct = savableGameStats.fameTillNext;
            currentCampaignMission = savableGameStats.campaignMission;
            goToCustomizer = savableGameStats.goToCustomizer;
            openUnitPanel = savableGameStats.openUnitPanel;
            dragUnitsToArmy = savableGameStats.dragUnitsToArmy;
            goToItems = savableGameStats.goToItems;
            openItemsPanel = savableGameStats.openItemsPanel;
            clickOpenUnit = savableGameStats.clickOpenUnit;
            dragItemsToUnits = savableGameStats.dragItemsToUnits;
            civScreenTutTip = savableGameStats.civScreenTutTip;
            wordMapTutTip = savableGameStats.wordMapTutTip;
            shopScreenTutTip = savableGameStats.shopScreenTutTip;
            goToCpTutTip = savableGameStats.goToCpTutTip;
            goToShopTutTip = savableGameStats.goToShopTutTip;
            salvageShopTutTip = savableGameStats.salvageShopTutTip;
            buildingsTip = savableGameStats.buildingsTip;
            nextActTutTip = savableGameStats.nextActTutTip;
            injuryTutTip = savableGameStats.injuryTutTip;
            retireFarmerTutTip = savableGameStats.retireFarmerTutTip;
            if (savableGameStats.energyEmptyTime != 0) {
            energyEmptyTime = DateTime.FromBinary(savableGameStats.energyEmptyTime);
            }
        }
    }

[Kurt-Dekker]

This is one reason binary formats are hard to deal with. Sure they usually work but when they fail, it’s hard to reason about them. Unless you can get a device in your hands that is failing, it’s hard to replicate a problem.

One possibility I can recommend (and this does NOT help previous users) is to make a JSON-based save system (using a different filename) along side this old legacy one, and then put out an update to your game that reads the binary file, and if it is present, it re-saves it as JSON and then deletes the binary file. From then on it would start using JSON only.

This process is broadly called “migrating data.”

Another thing you could tack onto such a migration system is to see if the binary file exists, try to read it, if you cannot, pop up a warning and tell the users the savefile had a problem, and offer to have them send the binary to you in an email. You would have to write all that code in your app, to extract the binary data that it WAS able to read, MIME- or UU-encode it for transmission via email, and then you could inspect it yourself. But that assumes the file is readable in the first place, i.e., it isn’t a permissions or filename problem, and overall it’s a LOT of work, and a lot of average users just won’t bother.

[PraetorBlue]

One glaring problem I see with your code is a lack of safety around your file streams. A good first step would be to wrap all of your FileStream access with using statements. Otherwise you may be leaking streams and possibly causing issues that cause the inability of your game to save or load.

using statement - ensure the correct use of disposable objects - C# reference

[Kurt-Dekker]

Oooh, good spot @PraetorBlue !! OP, what Praetor is talking about is the use of FileStream, which derives from Stream. Stream implements the IDisposable interface, which means you should wrap its use in using( ...) {} so that it gets disposed at the end.

On iOS you would only be able to save 255 times before it just magically started failing, and it would fail until the game binary was reloaded (force-killed and relaunched), and on cheap flavors of Android, that number might be even lower.

[ODINKONG]

@PraetorBlue

One glaring problem I see with your code is a lack of safety around your file streams. A good first step would be to wrap all of your FileStream access with using statements. Otherwise you may be leaking streams and possibly causing issues that cause the inability of your game to save or load.

https://docs.microsoft.com/en-us/dotnet/csharp/language-reference/keywords/using-statement

Ok first off thank you both so much for giving me all that info. @Kurt-Dekker at this point I will certainty be looking into using JSON or xml formats for saving in the future, but for now I need to patch this thing quickly before it gets to out of hand. If I can’t fix it I may try migration too. I have no traditional background so bear with me here.

I understand now that the way I’m doing this is using extra resources and not freeing them because I am not disposing of the file stream I am creating correct? A using statement would do that for me as well as handle exceptions it sounds like. I think I understand the syntax after reading what you have sent over.

Just for my own sanity is there a way I could replicate this breaking or becoming unstable in unity so that when I make it stable I know it is working? Say if I ran the save function in a for loop 255 times or more, or the load function for that matter would that corrupt the save in the unity editor on my PC? I guess what I’m saying is If I can find a way to break this code on my own device it would be easier for me to learn if what I’m doing is working before I try this using patch out.

I actually just tried saving 255 times in game on my Android but when I opened the application again the save was still there and not corrupted.

[brigas]

Maybe you could create a backup proof system?

When you start saving Im seeing that you immediately delete the old save, if the application somehow gets killed during that time( after you deleted the save and before the new save was created) because of a bug/lowbattery/usererror the save will just be lost. In this case you could at least keep the previous save point if you kept the backups.

[tonemcbride]

I agree with the above posts but for me one potential problem stood out in this code:

        for (int y = 0; y < 7; y++)
        { // add seven 0's to enem stat list y so that the enem stat list can use the keys as the x wihtout throwing an out of range error
            savableGameStats.enemStatListY.Add(0);
        }
        for (int i = 0; i < enemyStatAddList.Count; i++)
        {
            savableGameStats.enemStatListY[i] = enemyStatAddList[i].y;
        }

You allocate 7 entries in ‘savableGameStats’.enemStatListY’ before adding to it. If ‘enemyStatAddList.Count’ is greater than 7 though you would end up with an exception. At that point you’ve already deleted the previous save game so the user would be left with nothing. Might be worth clamping ‘enemyStatAddList.Count’ to a maximum of 7 just to make sure it’s not going to overflow the array. You could also move the ‘delete old save game’ code to just before you serialize the new one.

[Kurt-Dekker]

@tonemcbride

You allocate 7 entries in ‘savableGameStats’.enemStatListY’ before adding to it.

Oh wow, nice spot. Yeah, that would definitely be a problem. I’m not even sure how variable size arrays would serialize in binary… which is another reason to go with JSON.

[ODINKONG]

@brigas

Maybe you could create a backup proof system?

When you start saving Im seeing that you immediately delete the old save, if the application somehow gets killed during that time( after you deleted the save and before the new save was created) because of a bug/lowbattery/usererror the save will just be lost. In this case you could at least keep the previous save point if you kept the backups.

Does it even make sense to delete the old data? I am overwriting it with a new anyway. To be honest I’m not sure when I added that or why this has been a 4 year project so going back through this is a bit confusing. At this point I’m about to

1. add using blocks
2. Get rid of all delete data functions
3. clamp that list

Again though is there anyway to intentionally cause any of these errors in unity to make sure I fix them? I tried doing a break point in the middle of serialization we that I could stop the editor but unity wont let me stop the editor until I allow the break point to continue.

For now I’m going to try these changes and push a new build in the hopes that no more save issues occur. Thanks so much everyone for the help.
Please let me know if anyone sees any other issues or thinks of a way I could replicate the problem of save files being cleared in unity so I can check if my fixes worked.

[ODINKONG]

OK I have applied all the suggestions. I must say you are all very helpful and knowledgeable coders. I really appreciate the help.

After putting the using statements in the correct locations I realized something. I was making use of a variable loadedStats outside of the file stream (after closing it) which I created in the scope of the file stream in the loading script at the top. For some reason I just started using it again about half way down the loading script. I don’t know if this is an issue but it seems like it could be a big problem since I close the file stream before that. When I put the using statements in this actually threw and error so I corrected it.

Here is the new code for anyone still interested I hope it works. So far it hasn’t had any issue saving and loading since I changed although I’ve also never been able to create any issues with saving and loading on my end, but it seems safer to me. I also no longer delete the old saved data and I clamped the for loop.

Saving

 public void SaveGameStats()
    {
        string saveFileName = "GameStats.dat";
        if (GameManager.instance.hardCoreMode)
        {
            saveFileName = "GameStatsH.dat";
        }
        //if (File.Exists(Path.Combine(Application.persistentDataPath, saveFileName)))
        //{
        //    File.Delete(Path.Combine(Application.persistentDataPath, saveFileName));
        //}
        savableGameStats.currentSuccesiveMission = currentSuccessiveMission;

        TotalPlayTime += Time.realtimeSinceStartup - lastTimeSinceStartSeconds; // get the number of seconds the game has been running and convert it to a timespan
        lastTimeSinceStartSeconds = Time.realtimeSinceStartup;

        savableGameStats.enemStatListY.Clear(); // clear list to make sure its empty
        for (int y = 0; y < 7; y++)
        { // add seven 0's to enem stat list y so that the enem stat list can use the keys as the x wihtout throwing an out of range error
            savableGameStats.enemStatListY.Add(0);
        }
        for (int i = 0; i < 7; i++)
        {
            savableGameStats.enemStatListY[i] = enemyStatAddList[i].y;
        }
        savableGameStats.hardCoreMode = hardCoreMode;
        if (savableAccountStats.adFreeVersion == true)
        {
            savableAccountStats.adFreeVersion = true;
        }
        savableGameStats.sfxVolume = sfxVolume;
        savableGameStats.musicVolume = musicVolume;
        savableGameStats.HCPPassiveHeal = HCPPassiveHeal;
        savableGameStats.HCPMissionReRolls = HCPMissionReRolls;
        savableGameStats.missionReRolls = missionReRolls;
        savableGameStats.HCPInjuryHeal = HCPInjuryHeal;
        savableGameStats.BCPBuffMirror = BCPBuffMirror;
        savableGameStats.BCPAbilityPower = BCPAbilityPower;
        savableGameStats.BCPAbilityCooldown = BCPAbilityCooldown;
        savableGameStats.HCPAbilityPower = HCPAbilityPower;
        savableGameStats.unitDeadSaveChance = unitDeadSaveChance;
        savableGameStats.itemDeadSaveChance = JunkyardSalvageFromUnits;
        savableGameStats.currentChestType = currentChestType;
        savableGameStats.currentChestMagicFind = currentChestMagicFind;
        savableGameStats.currentChestItemNumber = currentChestItemNumber;
        savableGameStats.currentArmy = currentArmy;
        savableGameStats.arenaBet = CurrentArenaBet;
        savableGameStats.arenaRarity = ArenaRunRarirty;
        savableGameStats.isInArena = isInArena;
        savableGameStats.energy = energy;
        savableGameStats.bloodForgeStartingLevel = bloodForgeStartingLevel;
        savableGameStats.savedBloodForgeItemId = currentBloodForgeItemId;
        savableGameStats.bloodForgeSuccessPercentage = bloodForgeSuccessPercent;
        savableGameStats.isInBloodForge = isInBloodForge;
        savableGameStats.buildingBlueprintMissionNumber = buildingBluePrintMissionNumber;

        savableGameStats.levelReward = levelReward;
        savableGameStats.SCPResurection = SCPResurection;
        savableGameStats.SCPExtraCp = SCPExtraCp;
        savableGameStats.SCPITemPower = SCPITemPower;
        savableGameStats.battleNumber = savedBattleNumber;
        savableGameStats.lastTime = currentTime.ToBinary();
        if (energyEmptyTime != null)
        {
            savableGameStats.energyEmptyTime = energyEmptyTime.ToBinary();
        }
        savableGameStats.totalPlayTime = TotalPlayTime;
        savableGameStats.missionList = missionList;
        savableGameStats.savedArmylist = currentArmyList;
        savableGameStats.skillList = skillList;
        savableGameStats.skillCostList = skillCostList;
        savableGameStats.savedQuestList = currentQuestList;
        savableGameStats.IAttackBuff = IAttackBuff;
        savableGameStats.IdefenseBuff = IdefenseBuff;
        savableGameStats.RattackSpeedBuff = RattackSpeedBuff;
        savableGameStats.RaccuracyBuff = RaccuracyBuff;
        savableGameStats.BhealthBuff = BhealthBuff;
        savableGameStats.HdefenseBuff = HdefenseBuff;
        savableGameStats.SHealthBuff = SHealthBuff;
        savableGameStats.generalPortraitNum = generalPortaitNum;
        savableGameStats.generalIsMale = generalIsMale;
        savableGameStats.generalName = GeneralName;
        savableGameStats.ICPAttackSpeed = ICPAttackSpeed;
        savableGameStats.ICPBleed = ICPBleed;
        savableGameStats.commerceYield = ICPGoldPerMission;
        savableGameStats.HCPSalvage = HCPSalvage;
        //        savableGameStats.BCParmyCost = BCParmyCost;
        savableGameStats.BCPBuildingSlots = BCPBuildingSlots;
        savableGameStats.BCPBuildingCostReduction = BCPBuildingCostReduction;
        savableGameStats.totalKills = totalKills;
        savableGameStats.animatedTotalKills = animatedTotalKills;
        savableGameStats.packItemChance = packItemChance;
        //        savableGameStats.armySizeList = currentArmySizeList;
        savableGameStats.armySize = armySize;
        savableGameStats.bloodLevel = bloodLevel;
        savableGameStats.gold = gold;
        savableGameStats.salvage = salvage;
        savableGameStats.hasAllBuildings = hasAllBuildings;
        savableGameStats.SCPMagicFind = SCPMagicFind;
        savableGameStats.SCPLifeSteal = SCPLifeSteal;
        savableGameStats.BattlesWon = battlesWon;
        savableGameStats.ICp = ICp;
        savableGameStats.totalCpEver = totalCpEver;
        savableGameStats.RCp = RCp;
        savableGameStats.RCPMaxCollectionSize = RCPMaxCollectionSize;
        savableGameStats.BCp = BCp;
        savableGameStats.HCp = HCp;
        savableGameStats.SCp = SCp;
        savableGameStats.Gp = Gp;
        savableGameStats.packCost = packCost;
        savableGameStats.idcount = idCount;
        savableGameStats.firstgame = firstGameEver;
        savableGameStats.fame = fame;
        savableGameStats.maxAct = maxAct;
        savableGameStats.fameTillNext = fameTillNextAct;
        savableGameStats.campaignMission = currentCampaignMission;
        savableGameStats.RCPCritCD = RCPCritCD;
        savableGameStats.goToCustomizer = goToCustomizer;
        savableGameStats.openUnitPanel = openUnitPanel;
        savableGameStats.dragUnitsToArmy = dragUnitsToArmy;
        savableGameStats.goToItems = goToItems;
        savableGameStats.openItemsPanel = openItemsPanel;
        savableGameStats.clickOpenUnit = clickOpenUnit;
        savableGameStats.dragItemsToUnits = dragItemsToUnits;
        savableGameStats.buildingsTip = buildingsTip;


        savableGameStats.civScreenTutTip = civScreenTutTip;
        savableGameStats.wordMapTutTip = wordMapTutTip;
        savableGameStats.shopScreenTutTip = shopScreenTutTip;
        savableGameStats.goToCpTutTip = goToCpTutTip;
        savableGameStats.goToShopTutTip = goToShopTutTip;
        savableGameStats.salvageShopTutTip = salvageShopTutTip;

        savableGameStats.nextActTutTip = nextActTutTip;
        savableGameStats.injuryTutTip = injuryTutTip;
        savableGameStats.retireFarmerTutTip = retireFarmerTutTip;
        using (FileStream file = File.Create(Path.Combine(Application.persistentDataPath, saveFileName)))
        {
            new BinaryFormatter().Serialize(file, savableGameStats);
        }
        SaveAccount(); // this is to make sure the account data is saved frequently
    }

Loading

 public void LoadGameStats()
    {
        string saveFileName = "GameStats.dat";
        if(GameManager.instance.hardCoreMode)
        {
            saveFileName = "GameStatsH.dat";
        }
        if (File.Exists(Path.Combine(Application.persistentDataPath, saveFileName)))
        {
            using (FileStream file = File.Open(Path.Combine(Application.persistentDataPath, saveFileName), FileMode.Open))
            {
                GameStats loadedStats = (GameStats)new BinaryFormatter().Deserialize(file);
                savableGameStats = loadedStats;
            }
            firstGameEver = savableGameStats.firstgame;
            clearEnemStatList(); // clear list first to make sure we start clean
            for (int i = 0; i < savableGameStats.enemStatListY.Count; i++)
            { // for all the values of the stats
                addToEnemStatList(new Vector2(i + 1, savableGameStats.enemStatListY[i]));
            }
            if (savableGameStats.adFreeVersion == true)
            {
                adFreeVersion = true;
            }
            if (savableAccountStats.adFreeVersion == true) //this is a patch for people who bought the game before account stats exsisted IE my dad
            {
                adFreeVersion = savableAccountStats.adFreeVersion;
            }
            hardCoreMode = savableGameStats.hardCoreMode;
            energy = savableGameStats.energy;
            sfxVolume = savableGameStats.sfxVolume;
            musicVolume = savableGameStats.musicVolume;
            musicSource.volume = musicVolume;
            HCPPassiveHeal = savableGameStats.HCPPassiveHeal;
            HCPMissionReRolls = savableGameStats.HCPMissionReRolls;
            missionReRolls = savableGameStats.missionReRolls;
            HCPInjuryHeal = savableGameStats.HCPInjuryHeal;
            BCPAbilityPower = savableGameStats.BCPAbilityPower;
            HCPAbilityPower = savableGameStats.HCPAbilityPower;
            BCPAbilityCooldown = savableGameStats.BCPAbilityCooldown;
            BCPBuffMirror = savableGameStats.BCPBuffMirror;
            SCPResurection = savableGameStats.SCPResurection;
            SCPExtraCp = savableGameStats.SCPExtraCp;
            SCPITemPower = savableGameStats.SCPITemPower;
            unitDeadSaveChance = savableGameStats.unitDeadSaveChance;
            JunkyardSalvageFromUnits = savableGameStats.itemDeadSaveChance;
            currentChestType = savableGameStats.currentChestType;
            currentChestMagicFind = savableGameStats.currentChestMagicFind;
            currentChestItemNumber = savableGameStats.currentChestItemNumber;
            currentArmy = savableGameStats.currentArmy;
            bloodForgeStartingLevel = savableGameStats.bloodForgeStartingLevel;
            currentBloodForgeItemId = savableGameStats.savedBloodForgeItemId;
            isInArena = savableGameStats.isInArena;
            buildingBluePrintMissionNumber = savableGameStats.buildingBlueprintMissionNumber;
            isInBloodForge = savableGameStats.isInBloodForge;
            bloodForgeSuccessPercent = savableGameStats.bloodForgeSuccessPercentage;
            CurrentArenaBet = savableGameStats.arenaBet;
            ArenaRunRarirty = savableGameStats.arenaRarity;
            currentSuccessiveMission = savableGameStats.currentSuccesiveMission;
            //            currentSuccessiveMission = null;
            armySize = savableGameStats.armySize;
            levelReward = savableGameStats.levelReward;
            ICPGoldPerMission = savableGameStats.commerceYield;
            HCPSalvage = savableGameStats.HCPSalvage;
            SCPLifeSteal = savableGameStats.SCPLifeSteal;
            SCPMagicFind = savableGameStats.SCPMagicFind;
            ICPAttackSpeed = savableGameStats.ICPAttackSpeed;
            ICPBleed = savableGameStats.ICPBleed;
            battlesWon = savableGameStats.BattlesWon;
            hasAllBuildings = savableGameStats.hasAllBuildings;
            //            BCParmyCost = savableGameStats.BCParmyCost;
            totalKills = savableGameStats.totalKills;
            animatedTotalKills = savableGameStats.animatedTotalKills;
            BCPBuildingCostReduction = savableGameStats.BCPBuildingCostReduction;
            BCPBuildingSlots = savableGameStats.BCPBuildingSlots;
            packItemChance = savableGameStats.packItemChance;
            RCPCritCD = savableGameStats.RCPCritCD;
            RCPMaxCollectionSize = savableGameStats.RCPMaxCollectionSize;
            //            if (loadedStats.armySizeList != null) { // if we load a null list it will break the game and the list
            //                currentArmySizeList = savableGameStats.armySizeList;
            //            }
            if (savableGameStats.savedArmylist != null)
            {
                currentArmyList = savableGameStats.savedArmylist;
            }
            //            if (loadedStats.armyTypes != null) { // if we load a null list it will break the game and the list
            //                armyTypes = savableGameStats.armyTypes;
            //            }
            //            if (loadedStats.armyNameList != null) { // if we load a null list it will break the game and the list
            //                armyNames = savableGameStats.armyNameList;
            //            }
            if (savableGameStats.skillList != null)
            { // if we load a null list it will break the game and the list
                skillList = savableGameStats.skillList;
            }
            if (savableGameStats.skillCostList != null)
            { // if we load a null list it will break the game and the list
                skillCostList = savableGameStats.skillCostList;
            }
            else
            {
                for (int i = 0; i < 25; i++)
                {
                    skillCostList.Add(0);
                }
            }
            if (savableGameStats.savedQuestList != null)
            { // if we load a null list it will break the game and the list
                currentQuestList = savableGameStats.savedQuestList;
            }
            if (savableGameStats.missionList != null)
            { // if we load a null list it will break the game and the list
                missionList = savableGameStats.missionList;
            }
            else
            {
                missionList.MissionListStart();
            }
            savedBattleNumber = savableGameStats.battleNumber;
            battleNumber = savableGameStats.battleNumber;
            lastTime = currentTime;
            TotalPlayTime = savableGameStats.totalPlayTime;
            IAttackBuff = savableGameStats.IAttackBuff;
            IdefenseBuff = savableGameStats.IdefenseBuff;
            RattackSpeedBuff = savableGameStats.RattackSpeedBuff;
            RaccuracyBuff = savableGameStats.RaccuracyBuff;
            BhealthBuff = savableGameStats.BhealthBuff;
            HdefenseBuff = savableGameStats.HdefenseBuff;
            SHealthBuff = savableGameStats.SHealthBuff;
            generalPortaitNum = savableGameStats.generalPortraitNum;
            generalIsMale = savableGameStats.generalIsMale;
            GeneralName = savableGameStats.generalName;
            bloodLevel = savableGameStats.bloodLevel;
            salvage = savableGameStats.salvage;
            gold = savableGameStats.gold;
            packCost = savableGameStats.packCost;
            ICp = savableGameStats.ICp;
            totalCpEver = savableGameStats.totalCpEver;
            RCp = savableGameStats.RCp;
            BCp = savableGameStats.BCp;
            HCp = savableGameStats.HCp;
            SCp = savableGameStats.SCp;
            Gp = savableGameStats.Gp;
            idCount = savableGameStats.idcount;
            fame = savableGameStats.fame;
            maxAct = savableGameStats.maxAct;
            fameTillNextAct = savableGameStats.fameTillNext;
            currentCampaignMission = savableGameStats.campaignMission;
            goToCustomizer = savableGameStats.goToCustomizer;
            openUnitPanel = savableGameStats.openUnitPanel;
            dragUnitsToArmy = savableGameStats.dragUnitsToArmy;
            goToItems = savableGameStats.goToItems;
            openItemsPanel = savableGameStats.openItemsPanel;
            clickOpenUnit = savableGameStats.clickOpenUnit;
            dragItemsToUnits = savableGameStats.dragItemsToUnits;

            civScreenTutTip = savableGameStats.civScreenTutTip;
            wordMapTutTip = savableGameStats.wordMapTutTip;
            shopScreenTutTip = savableGameStats.shopScreenTutTip;
            goToCpTutTip = savableGameStats.goToCpTutTip;
            goToShopTutTip = savableGameStats.goToShopTutTip;
            salvageShopTutTip = savableGameStats.salvageShopTutTip;

            buildingsTip = savableGameStats.buildingsTip;

            nextActTutTip = savableGameStats.nextActTutTip;
            injuryTutTip = savableGameStats.injuryTutTip;
            retireFarmerTutTip = savableGameStats.retireFarmerTutTip;
            if (savableGameStats.energyEmptyTime != 0) {
            energyEmptyTime = DateTime.FromBinary(savableGameStats.energyEmptyTime);
            }
        }
    }

[Suddoha]

@ODINKONG

I understand now that the way I’m doing this is using extra resources and not freeing them because I am not disposing of the file stream I am creating correct? A using statement would do that for me as well as handle exceptions it sounds like. I think I understand the syntax after reading what you have sent over.

Closing the stream usually disposes it - but only if the code gets to call it. And that’s where things get tricky without using statement. Of course it’s possible without it, there are some cases in which you have to do it without this handy feature.

Anyway, whenever you can, you should use it, because it’s easy to mess things up.
For example, without using, if your loading or saving procedure fails at some point, it’ll throw and the part with closing & disposing is not run (unless you implemented try, [catch], finally properly).

The using statement ensures that no matter what happens, when the block of code is left, it’ll always dispose what’s been declared in the statement’s parentheses (during compilation the correct try/finally structure will be generated). You can simply leave, return or throw and it’ll dispose the stream.

However, it doesn’t mean it handles exceptions “automagically”. It only ensures that things will be disposed when they need to be. You still have to implement your own exception handling.

It’s also worth noting that you should probably spend a day or two removing all the logic around the streams and the serialization logic. These things are often pulled out and put into a wrapper class, so that you can easily reuse one implementation for whatever data you need to save/load, without caring about details and proper implementation in the future.

@ODINKONG

After putting the using statements in the correct locations I realized something. I was making use of a variable loadedStats outside of the file stream (after closing it) which I created in the scope of the file stream in the loading script at the top. For some reason I just started using it again about half way down the loading script. I don’t know if this is an issue but it seems like it could be a big problem since I close the file stream before that. When I put the using statements in this actually threw and error so I corrected it.

This won’t be an issue. The data returned from reading operations is not bound to the stream in any way.

[brigas]

@ODINKONG

Does it even make sense to delete the old data? I am overwriting it with a new anyway. To be honest I’m not sure when I added that or why this has been a 4 year project so going back through this is a bit confusing. At this point I’m about to

1. add using blocks
2. Get rid of all delete data functions
3. clamp that list

Again though is there anyway to intentionally cause any of these errors in unity to make sure I fix them? I tried doing a break point in the middle of serialization we that I could stop the editor but unity wont let me stop the editor until I allow the break point to continue.

For now I’m going to try these changes and push a new build in the hopes that no more save issues occur. Thanks so much everyone for the help.
Please let me know if anyone sees any other issues or thinks of a way I could replicate the problem of save files being cleared in unity so I can check if my fixes worked.

well to simulate the game crashing during the serialization I would try to serialize a big byte array of maybe 200million entries(200mb) and while unity hangs(binaryformatter takes a long time to serialize 200mb) I ctrl+shift+esc and terminate unity. If you want to make super sure that saves aren’t lost create a copy of the old save before overwriting it and after you are sure the new save was serialized successfully you can delete it.

[ODINKONG]

Ok this all makes a lot of sense by logic around the streams do you mean things like if file exists? Or did you just mean the streams should be isolated in their using block and the rest of the logic for saving and loading should be done entirely outside the stream. Could you give me a quick example of the logic you are referring to, sorry I don’t have a good code foundation, but all this is really helping me strengthen it.

[ODINKONG]

@brigas

well to simulate the game crashing during the serialization I would try to serialize a big byte array of maybe 200million entries(200mb) and while unity hangs(binaryformatter takes a long time to serialize 200mb) I ctrl+shift+esc and terminate unity. If you want to make super sure that saves aren’t lost create a copy of the old save before overwriting it and after you are sure the new save was serialized successfully you can delete it.

Ok that sounds like a great way to test corruption. Thank you. I think I may also try the back up idea.

[ODINKONG]

Do those using statements look right now btw?

[Suddoha]

@ODINKONG

Ok this all makes a lot of sense by logic around the streams do you mean things like if file exists? Or did you just mean the streams should be isolated in their using block and the rest of the logic for saving and loading should be done entirely outside the stream. Could you give me a quick example of the logic you are referring to, sorry I don’t have a good code foundation, but all this is really helping me strengthen it.

Not necessarily just that, though that can be an additional feature.

What I’m talking about goes along these lines:

public class BinaryFileSerializer
{
    public void Serialize<TData>(TData data, string filePath)
    {
        // TODO: check args
        // TODO: handle exceptions or let them bubble up
        using (var stream = new FileStream(filePath, FileMode.Create))
        {
            var formatter = new BinaryFormatter();
            formatter.Serialize(stream, data);
        }
    }

    public TData Deserialize<TData>(string filePath)
    {
        // TODO: check args
        // TODO: handle exceptions or let them bubble up
        using (var stream = new FileStream(filePath, FileMode.Open))
        {
            var formatter = new BinaryFormatter();
            return (TData)formatter.Deserialize(stream);
        }
    }
}

This is not the only way to do that. For example, this solution always assumes that the source/destionation is a filestream, hence a path parameter (which could also be a System.IO.FileInfo, as this does lots of validation already).
You could twist and tweak this in many different ways.

You might ask what’s the actual benefit. Well, first of all, you have those few yet very important lines of code in a single place. It’ll always be something along these lines.

Next, if you’re experienced in OOP languages, you’ll quickly realize that you can actually add an interface and/or base class, and multiple sub-classes, like an XML based serializer, DataContractSerializer, JSONSerializer, CSVSerializer, custom formats whatsoever… and all would share the same public interface.

In this particular case, let’s put the interface:

public interface IFileSerializer
{
    void Serialize<TData>(TData data, string filePath);
    TData Deserialize<TData>(string filePath);
}

That’d allow to easily replace one serialization module by another if there’s ever a need for it (development, debugging, production…) It won’t change the code on the caller’s side because the methods you call are the same.

So let’s say you want to see what’s being written, or you wanna change to a different format. Let’s not be biased towards XML, but it’s the one that’s quickly thrown together:

public class XMLSerializer : IFileSerializer
{
    public void Serialize<TData>(TData data, string filePath)
    {
        // TODO: check args
        // TODO: handle exceptions or let them bubble up
        using (var stream = new FileStream(filePath, FileMode.Create))
        {
            var serializer = new System.Xml.Serialization.XmlSerializer(typeof(TData));
            serializer.Serialize(stream, data);
        }
    }

    public TData Deserialize<TData>(string filePath)
    {
        // TODO: check args
        // TODO: handle exceptions or let them bubble up
        using (var stream = new FileStream(filePath, FileMode.Open))
        {
            var serializer = new System.Xml.Serialization.XmlSerializer(typeof(TData));
            return (TData) serializer.Deserialize(stream);
        }
    }
}

Due to the interface (or base class, if you will) one can quickly replace one with another without changing the code that wants to serialize something, because the interface is all that matters for the caller.

Be aware that this is probably the most trivial abstraction. There’s a lot you can do to make a serialization system and its modules very flexible and powerful. But the most important takeaway: encapsulate it, even if you won’t support multiple serializers, because you don’t wanna repeat yourself, nor want to dig through hundreds of lines of code to find everything related to streams+serializer.

[Kurt-Dekker]

I agree here with @Suddoha as far as splitting up the logic that gathers up all the saved info and the part that writes it to disk file. It’s generally always good practice to separate areas of concern. The file writing doesn’t care where the data came from or is going to and vice versa, so separate it. Logically speaking:

save:

• pull all save state out of your game, wherever it is relevant, putting it into a single blob

• write that blob to disk, making sure it succeeds

load:

• read the blob from disk

• pluck all the save data out

• if it seems good, send it out to the game elements and start the game from there

[ODINKONG]

I’ve been adding a lot of these safety nets, but while I was adding the extra backup file I was testing it to see if it works. If I delete the main file then it falls to the backup and loads fine which is awesome, but I noticed something. If I open the main file in notepad and delete a piece of it intentionally corrupting the file, the game doesn’t clear my save data it just doesn’t start the game. it throws an exception and doesn’t continue to start the game. All these users are reporting their games being cleared and they are starting in the tutorial all of a sudden with a fresh new game that functions.

Some one from unity suggested this issue might have to do with auto backup, on google and I guess on IOS too since it is happening on both?
What are your thoughts on this. The unity moderator explains some of it here. It’s starting to seem like this issue is something on the phone clearing the data while the game is closed, What could possibly cause that?

Persistent data path inconsistent on Android

Yes, I think so. I don’t know how exactly Android’s backup feature works, but since official documentation says that it backups only up to 25MB and then it stops backuping, then imagine you have created 6 save files 5 MB each. Only 5 of them will be backed up. Once that backup is restored, the last (newest) save will be gone. Both the name and the contents of the file matter a lot. The name must be AndroidManifest.xml and it will overwrite the main manifest used by Unity. The best way to make…

[ODINKONG]

I just wanna say thanks everyone! This issue has been fixed. none of our new users have reported it. I don’t know what fixed it, but here is what I ended up doing.

1. fixed an issue around saving game data during an in app purchase.
2. added using wrappers to all serialization statements.
3. instead of deleting the data before writing to it I just overwrite the file now with the new data and I never delete it unless the player clears their save manually.
4. I created a backup that will restore if the first file is deleted.

[Havyx]

Hopefully this isn’t considered necroposting as it could be useful to anyone who finds this via searching on Google.

Deserialization risks in use of BinaryFormatter and related types - .NET

“The BinaryFormatter type is dangerous and is not recommended for data processing. Applications should stop using BinaryFormatter as soon as possible, even if they believe the data they’re processing to be trustworthy. BinaryFormatter is insecure and can’t be made secure.”