Saving System Upgrade: Replacing BinaryFormatter with Json.NET

Important Update: This version of the Json Saving System is Outdated

Please follow the completed tutorial here: SavingSystem

-------------------------------------------------------------------------

Over the last month or so, I’ve been working on a replacement for the BinaryFormatter we currently use in our saving system. As some may be aware, Microsoft has officially declared the BinaryFormatter unsafe as there is a potential for an exploit that could allow a hacker to intercept a save file, inject data, and actually instantiate the hacker’s own classes into our project.

This thread will serve as a replacement model for the existing saving system. It is probably best read on my repo/wiki here: Wiki · Brian K. Trotter / RPGMods · GitLab

Why Convert?

When the RPG Course Saving System was introduced in the first course, Core Combat, we used the BinaryFormatter to serialize and deserialize the needed fields. Unfortunately, Microsoft recently announced that using the BinaryFormatter was not safe, and recommends that it no longer be used. This issue presents a problem for companies around the globe who used this system not knowing the potential threats. I won’t go into the specific threats, and to be clear, while we’re in the prototyping stage (up until we release games to the public using this system) there is little to no risk. It’s when those save files start going over the internet that the trust level declines, and the risk of a hack that can compromise your system could be introduced. Bottom line ***Don’t Panic!*** All of that being said, I’m presenting some modifications to the saving system that will allow us to save in the Json format. As a side effect of teaching how to convert the system, I’ll even be leaving the original saving system more or less in place (you’ll be able to deactivate it and remove it easily once we’re done).

Getting Started

You may want to start by learning how the Saving System works. This is covered in depth in the Core Combat Course, both with a “Here’s an asset Pack” style approach and a “Let’s build this from the ground up approach”. If you’re starting in one of our other courses, you should review [How The Saving System Works](SavingSystemPrimer)

Installing Json.NET

While Unity does come with some rudimentary Json serialization, it’s not sufficient to keep up with the demands of our Saving System. For starters, it doesn’t support Dictionaries, let alone what are essentially nested Dictionaries.

Fortunately, there is another solution. There is a package in the Asset store called Json.NET for Unity. If you’re not familiar with Json.NET, it is a very robust serialization system that is more than powerful enough for our needs. It even works with IC2CPP in Android and iOS. Best of all, it’s free!
Follow this link to the Unity Asset Store Listing and add it to your assets. Then you can install it from the Package Manager in your project under Packages: My Assets.

For this tutorial, I’m going to create a new interface IJsonSaveable, so that at least for now, we can keep the original code intact. At the end of the tutorial, you’ll be able to remove the original save code and simply progress with the new Saving System.

IJsonSaveable

using Newtonsoft.Json.Linq;

namespace GameDevTV.Saving
{
    public interface IJsonSaveable 
    {
        public JToken CaptureAsToken();
        public void RestoreFromToken(JToken state);
    }
}

This is the Interface we will be using to replace the CaptureState/RestoreState from the original save system. You can, if you wish, simply edit ISaveable directly and replace the object in CaptureState and RestoreState with JToken. This will, of course, mean you’ll need to change everything at once to compile the code, which is why I’m taking this approach.
JToken is a special class that can serve the same way as Object for passing information around. Essentially, you can serialize the state to a JToken by using JToken.FromObject(T item), and you can convert it back by using the method ToObject(). I’ll demonstrate this now.

So let’s start with our first IJsonSaveable, we’ll start with Mover.cs. Here’s the original code in Mover.cs

        public object CaptureState()
        {
            return new SerializableVector3(transform.position);
        }

        public void RestoreState(object state)
        {
            SerializableVector3 position = (SerializableVector3)state;
            navMeshAgent.enabled = false;
            transform.position = position.ToVector();
            navMeshAgent.enabled = true;
            GetComponent<ActionScheduler>().CancelCurrentAction();
        }

SerializableVector3 is a special class that was created for the purpose of capturing and restoring Vectors. This is because for the BinaryFormatter, Vector3 is not Serializable. Fortunately, Json.Net has no such trouble. You can directly convert a Vector3 to and from a JToken.

Here’s the code for our IJsonSaveable classes. Be sure to add ,IJsonSaveable to your class declaration (right after ,ISaveable), and to add using NewtonSoft.Json.Linq; to your usings clauses.

// this is the added using declaration
using NewtonSoft.Json.Linq;

//this is the new class declaration
        public class Mover : MonoBehavior, IAction, ISaveable, IJsonSaveable
        
// Add these classes to fulfill the IJsonSaveable
        public JToken CaptureAsToken()
        {
            return JToken.FromObject(transform.position);
        }

        public void RestoreFromToken(JToken state)
        {
            navMeshAgent.enabled=false;
            transform.position=state.ToObject<Vector3>();
            navMeshAgent.enabled=true;
            GetComponent<ActionScheduler>().CancelCurrentAction();
        }

So for CaptureAsToken, we’re simply returning the position as a JToken… Since JToken can manage a Vector3, it’s just a simple matter of using the JToken static method FromObject().
For RestorFromToken, all we need to do is convert the JToken back into a Vector3 .

Ok, time for your first challenge. Head on over to Health.cs, and take a look at the CaptureState and RestoreState. Add IJsonSaveable to the class declaration, the using clause, and implement the the CaptureAsToken() and RestoreFromToken() methods.

5 Likes

How did you do on the challenge?

In the first entry of this thread, I challenged you to write the Health.cs IJsonSaveable code. Were you able to meet the challenge?

Let’s look first at the Health.cs ISaveable code:

        public object CaptureState()
        {
            return healthPoints.value;
        }

        public void RestoreState(object state)
        {
            healthPoints.value = (float) state;        
            UpdateState();
        }

That should be straightforward enough. We’re returning the value portion of our LazyValue healthPoints, and when we restore it, we’re simply changing the state back to a float.
So here’s what we need to do for the IJsonSaveable

//Add this to the using clauses
using NewtonSoft.Json.Linq;

//Modify the class declaration
public class Health : MonoBehaviour, ISaveable, IJsonSaveable

//Implement the missing members
        public JToken CaptureAsToken()
        {
            return healthPoints.value;
        }

        public void RestoreFromToken(JToken state)
        {
            healthPoints.value = state.ToObject<float>();
            UpdateState();
        }
    }

Notice that in this case, all I did was return the float. That’s because many conversions can be done implicitly. Assigning a value like an int, float, or string to JToken will automatically convert the item to JToken. In RestoreFromToken, we still need to explicitly declare the type. This is important, as even if there was an implicit conversion from JToken to our native types, it’s the implicit conversions that are behind much of the danger in the BinaryFormatter.

Let’s try another challenge, this time in Experience.cs:
Looking at Experience, we see the Capture and RestoreState methods look like this:

        public object CaptureState()
        {
            return experiencePoints;
        }

        public void RestoreState(object state)
        {
            experiencePoints = (float)state;
        }

Take a shot at writing the IJsonSaveable version. I’ll hide the solution

Click this to view/hide the solution

//Add this to the using clauses
using NewtonSoft.Json.Linq;

//Modify the class declaration
public class Experience : MonoBehaviour, ISaveable, IJsonSaveable

//Finally the methods
        public JToken CaptureAsToken()
        {
            return experiencePoints;
        }

        public void RestoreFromToken(JToken state)
        {
            experiencePoints = state.ToObject<float>();
        }

Once again, we see that we can return experiencepoints (it is simply a float value) and it is auto converted. You could also have done an explicit declaration

return JToken.FromObject(experiencePoints);

When converting back, we need to explicitly convert the state back to a float using

state.ToObject<float>();

Let’s take care of the last remaining ISaveable from the Core Combat course, Fighter.cs before we jump in to changing the SaveableEntity.

Note that if you have completed the entire course, you’ll likely have removed the ISaveable (as the equipped weapon will really be managed by Equipment anyways). If you look at the commit for this lesson, you won’t see Fighter.cs set up as an ISaveable or IJsonSaveable

Here’s the code from Fighter.cs before the end of Shops and Abilities

public object CaptureState()
{
    return currentWeaponConfig.name;
}
public void RestoreState(object state)
{
    string weaponName = (string)state;
    WeaponConfig weapon = UnityEngine.Resources.Load<WeaponConfig>(weaponName);
    EquipWeapon(weapon);
}

Since string is one of the things that can be implicitly converted as a JToken, we can copy CaptureState’s contents into CaptureAsToken

public JToken CaptureAsToken()
{
    return currentWeaponConfig.name;
}

and all we need to change in RestoreFromToken is how we convert state

public void RestoreFromToken(JToken state)
{
    string weaponName = state.ToObject<string>();
    WeaponConfig weapon = UnityEngine.Resources.Load<WeaponConfig>(weaponName);
    EquipWeapon(weapon);
}

In the next comment, we’ll start making the needed changes to SaveableEntity.cs

1 Like

Converting to JSon Part III

SaveableEntity

In the previous pages, we did some work on the replacements for ISaveables in the components from the Core Combat course. I promise, we’ll get to the components in the rest of the courses, but at this point, it’s a good time to start making the changes to SaveableEntity.cs and the Saving System itself so that we can see this JSON saving system in action.

We’re not going to worry about the extra code in SaveableEntity built around ensuring that every entity has a unique UUID, that’s not important to what we’re doing now, and we’re not going to change any of that code. For a good explanation of how that code works, be sure to study the second saving section in the Core Combat course.

So here is the section we’re interested in changing, and the good news is that there’s really not a lot to change in here:

        public object CaptureState()
        {
            Dictionary<string, object> state = new Dictionary<string, object>();
            foreach (ISaveable saveable in GetComponents<ISaveable>())
            {
                state[saveable.GetType().ToString()] = saveable.CaptureState();
            }
            return state;
        }


        public void RestoreState(object state)
        {
            Dictionary<string, object> stateDict = (Dictionary<string, object>)state;
            foreach (ISaveable saveable in GetComponents<ISaveable>())
            {
                string typeString = saveable.GetType().ToString();
                if (stateDict.ContainsKey(typeString))
                {
                    saveable.RestoreState(stateDict[typeString]);
                }
            }
        }

If you’re not super familiar with how this works, the SaveableEntity gathers all of the ISaveable components attached to the GameObject and asks them for their state. This state is then saved in a Dictionary<string, object> which is returned at the end of the method to the Saving System. When the Saving System restores, RestoreState decodes the dictionary and distributes the states back to the ISaveable components attached to the GameObject. These are the methods that call our Capture and Restore in the ISaveables.

CaptureAsToken

So let’s start with CaptureState… we’re going to make a change and demonstrate a nifty trick in the process:

        public JToken CaptureAsToken()
        {
            JObject state = new JObject();
            IDictionary<string, JToken> stateDict = state;
            foreach (IJsonSaveable saveable in GetComponents<IJsonSaveable>())
            {
                stateDict[saveable.GetType().ToString()] = saveable.CaptureAsToken();
            }
            return state;
        }

So that second line in the code may seem a bit confusing… IDictionary<string, JToken> stateDict = state; To start, we’re using a JObject instead of a JToken. JObject is a special type of JToken that is designed to easily encode collections and Dictionaries of JTokens. That IDictionary<string, JToken> allows us to work on the JObject as if it were a Dictionary. Now the rest of the method can work on stateDict (the IDictionary), and everything we do to it is automatically reflected in the JObject. When we’re done, we don’t have to do any special conversion, because it’s already done. You can just return state;

RestoreFromToken

We’re going to use this JObject approach once again with the RestoreFromToken.

        public void RestoreFromToken(JToken state)
        {
            IDictionary<string, JToken> stateDict = state.ToObject<JObject>();
            foreach (IJsonSaveable saveable in GetComponents<IJsonSaveable>())
            {
                string typeString = saveable.GetType().ToString();
                if (stateDict.ContainsKey(typeString))
                {
                    saveable.RestoreFromToken(stateDict[typeString]);
                }
            }
        }

We’re starting by converting the state to a JObject, which we’re immediately mapping as an IDictionary. We could have done this in two lines, but this is something we should be able to do on one line. Then, because it’s mapped to an IDictionary<string, JToken>, the rest of the method is virtually identical to the original RestoreState, just swapping out the interface name and methods.

It’s really that easy. Don’t be fooled, however, under the hood, a lot of great stuff is happening, but fortunately for us, we’re not doing the heavy lifting.

In the next comment, we’re going to make the changes to SavingSystem.cs. This is when it gets real. We’ll be saving JSON instead of a binary file.

1 Like

Converting to JSon Part IV

SavingSystem Changes

Ok, so far, we’ve converted the ISaveables in the Core Combat course to IJSonSaveables and we’ve modified our SaveableEntity to accept and distribute the JTokens that contained the saved data from each component. It’s time to make the needed changes to the SavingSystem and really start to commit ourselves to saving in JSon instead of the unsafe BinaryFormatter format.

Backup Your Project and Commit Your Repo

First of all, if you’re not using Source Control, then at the very least make a copy of your project. The changes we’re about to make could break the game if done incorrectly, and you definitely want a way back if things go wrong. This isn’t a course on using Git, and this is your big reminder.

Enough preaching, on to modifying the Saving System. Luckily, there’s not a lot to do, but it’s important to get it right.

Replacing CaptureState and RestoreState

As usual, we’ll start with the original code for CaptureState and RestoreState. We’ll leave these in for now, and put in their inevitable replacements.

        private void CaptureState(Dictionary<string, object> state)
        {
            foreach (SaveableEntity saveable in FindObjectsOfType<SaveableEntity>())
            {
                state[saveable.GetUniqueIdentifier()] = saveable.CaptureState();
            }

            state["lastSceneBuildIndex"] = SceneManager.GetActiveScene().buildIndex;
        }

        private void RestoreState(Dictionary<string, object> state)
        {
            foreach (SaveableEntity saveable in FindObjectsOfType<SaveableEntity>())
            {
                string id = saveable.GetUniqueIdentifier();
                if (state.ContainsKey(id))
                {
                    saveable.RestoreState(state[id]);
                }
            }
        }

Much like SaveableEntity, the SavingSystem gathers all of the SaveableEntities and asks them for their state, saving them in a Dictionary<string, object>. RestoreState distributes the states back to their respective SaveableEntities. One big difference is that instead of CaptureState returning a Dictionary, we’re actually passing the Dictionary directly to the method to modify. This allows us to save the data from multiple scenes in the same Dictionary. No need for a separate save file for each scene, it all goes in the one save file. For our replacement methods, instead of passing a Dictionary<string, object> we’re going to pass a JObject. Remember that JObjects are actually a collection of JParameter. A JParameter represents a string and a JToken. Because of this, we can map our JObject to an IDictionary<string, JToken>. Let’s take a look at our new Capture:

        private void CaptureAsToken(JObject state)
        {
            IDictionary<string, JToken> stateDict = state;
            foreach (SaveableEntity saveable in FindObjectsOfType<SaveableEntity>())
            {
                stateDict[saveable.GetUniqueIdentifier()] = saveable.CaptureState();
            }

            stateDict["lastSceneBuildIndex"] = SceneManager.GetActiveScene().buildIndex;
        }

So the JObject that is fed into CaptureAsToken may or may not contain data from other scenes (or even this scene that need to be overwritten). Fortunately, the IDictionary interface comes into play allowing us to treat this JObject as if it were, in fact, a Dictionary(). This means that aside from changing a few method names, we literally don’t have to change anything else in the method! Just the header and that first line to setup the IDictionary.

This leads us to RestoreFromToken() which is also just a straightforward a change:

        private void RestoreFromToken(JObject state)
        {
            IDictionary<string, JToken> stateDict = state;
            foreach (SaveableEntity saveable in FindObjectsOfType<SaveableEntity>())
            {
                string id = saveable.GetUniqueIdentifier();
                if (stateDict.ContainsKey(id))
                {
                    saveable.RestoreFromToken(stateDict[id]);
                }
            }
        }

Just like in SaveableEntity, we’re mapping an IDictionary over the JObject.

SaveFile and LoadFile

This is where the actual serialization happens. Up until now, the JTokens and JObjects are just being collected, they’re not actually JSon yet. Now we’re going to actually write the JSon to disk and read it back. Once again, we’ll start with the original code:

        private void SaveFile(string saveFile, object state)
        {
            string path = GetPathFromSaveFile(saveFile);
            print("Saving to " + path);
            using (FileStream stream = File.Open(path, FileMode.Create))
            {
                BinaryFormatter formatter = new BinaryFormatter();
                formatter.Serialize(stream, state);
            }
        }

        private Dictionary<string, object> LoadFile(string saveFile)
        {
            string path = GetPathFromSaveFile(saveFile);
            if (!File.Exists(path))
            {
                return new Dictionary<string, object>();
            }
            using (FileStream stream = File.Open(path, FileMode.Open))
            {
                BinaryFormatter formatter = new BinaryFormatter();
                return (Dictionary<string, object>)formatter.Deserialize(stream);
            }
        }

This is the part of our saving system that Microsoft has complained about. BinaryFormatter is unsafe, because in the process of Deserializing, any objects within the stream are actually instantiated, and can get their initialization run on them. Not only that, if an object is simply embedded alongside our carefully curated Dictionary, but it’s class doesn’t exist in your project, the object can provide the information to download it into your computer and run it. It means a malicious hacker could inject code into a system by infecting the save file.
While we’ve been prototyping, and the savefile is safely in the PersistentDataPath, not being shared on the web, etc, the danger is virtually zero that it could get infected, but the moment that Save goes out into the world (say you email the save file to me to examine what’s going on, or you put the save file on an AWS or Google Cloud server)… then the trust barrier is broken, and the potential for this hack always exists.

Here’s our new SaveFile method:

        private void SaveFileAsJSon(string saveFile, JObject state)
        {
            string path = GetPathFromSaveFile(saveFile);
            print("Saving to " + path);
            using (var textWriter = File.CreateText(path))
            {
                using (var writer = new JsonTextWriter(textWriter))
                {
                    writer.Formatting = Formatting.Indented;
                    state.WriteTo(writer);
                }
            }
        }

Instead of a binary writer, we’re using a textWriter. We’re setting the formatting to indented so that the file will be readable. This is optional, and you can remove that line if you wish to make the entire JSon go on a single line.

Here’s the LoadFile replacement method:

        private JObject LoadJSonFromFile(string saveFile)
        {
            string path = GetPathFromSaveFile(saveFile);
            if (!File.Exists(path))
            {
                return new JObject();
            }
            
            using (var textReader = File.OpenText(path))
            {
                using (var reader = new JsonTextReader(textReader))
                {
                    reader.FloatParseHandling = FloatParseHandling.Double;

                    return JObject.Load(reader);
                }
            }
        }

This is fairly similar to our original LoadFile method as well. If there is no save file, an empty JObject is returned, which can then essentially become our new save file later on.

This leads us to the moment we’ve been waiting for, the actual Load and Save… Here’s the original:

        public void Save(string saveFile)
        {
            Dictionary<string, object> state = LoadFile(saveFile);
            CaptureState(state);
            SaveFile(saveFile, state);
        }

        public void Load(string saveFile)
        {
            RestoreState(LoadFile(saveFile));
        }

This may shock you, but at this point, the changes are almost minimal… and this is the point of no return, the point at which we commit our saving system to switch from BinaryFormatter to JSON. From here on out, we’re committed. So committed, we’re going to just edit the method directly.

Did you Commit?

Just another reminder, commit your code or at least back it up!

Here’s our modified code. By the end of this post, you’ll be saving and restoring Json

        public void Save(string saveFile)
        {
            JObject state = LoadJSonFromFile(saveFile);
            CaptureAsToken(state);
            SaveFileAsJSon(saveFile, state);
        }

        public void Load(string saveFile)
        {
            RestoreFromToken(LoadJSonFromFile(saveFile));
        }

Finishing touches

There are a few more tidbits in SavingSystem.cs to take care of to make everything work. The first is in the GetPathFromSaveFile method. Right now, we’re using the extension “.sav”. You can leave it at that if you wish, but it might be better to change that extension to “.json” for clarity.

        private string GetPathFromSaveFile(string saveFile)
        {
            return Path.Combine(Application.persistentDataPath, saveFile + ".json");
        }

If you’ve completed the Main Menu section of the course, you’ll also need to change the extension in the ListSaves method to “.json”

Finally, there is one last method to change, and that’s LoadLastScene.

        public IEnumerator LoadLastScene(string saveFile)
        {
            JObject state = LoadJSonFromFile(saveFile);
            IDictionary<string, JToken> stateDict = state;
            int buildIndex = SceneManager.GetActiveScene().buildIndex;
            if (stateDict.ContainsKey("lastSceneBuildIndex"))
            {
                buildIndex = (int)stateDict["lastSceneBuildIndex"];
            }
            yield return SceneManager.LoadSceneAsync(buildIndex);
            RestoreFromToken(state);
        }

We’re starting the method out by getting our state as a JObject. As we’ve already seen through this tutorial, that JObject can be mapped as an IDictionary<string, JToken>. We need this to get the lastSceneBuildIndex, which tells the method which scene to load. Once we’ve loaded the correct scene, we pass the state on to RestoreFromToken() and things are restored.

At this point, you should be able to test the saving system out. Bear in mind, we’re not saving much, just the stuff from the Core Combat course. Start your game, maybe take on an enemy or just move to a new spot and save the game. Then quit the game and return, and you should find yourself in the position you left, not the starting position.

This might be a good time to navigate to the .json file and check out what’s been saved. Unlike the results from BinaryFormatter, the .json file is a human readable document.

So believe it or not, you now have the tools you need to convert the rest of the ISaveables into JsonSaveables. Don’t worry, though, the next comment will go over the remaining sections.

1 Like

Converting to JSon Part V

Converting the rest of the ISaveables

There’s not going to be a great deal of explanation on these. This will be a rundown of all of the ISaveables in the course repo that we haven’t already covered before. You may see classes here you haven’t encountered, depending on what courses you’ve taken/completed. If you do get to those classes in one of the courses, then head here for the code.

Note that if you have made substantive changes to your project, the code here may not work (for example, you may have a Purse with 10 different currency types, so you’d need to capture/restore a collection instead of a single float). I’m thinking that if you’re ready to make those changes, you’ll probably

Remember that for all IJsonSaveables, you’ll need to include using NewtonSoft.Json.Linq; to your using clauses, and add the ,IJsonSaveable to the class declaration.

Inventory.cs

        public JToken CaptureAsToken()
        {
            var slotStrings = new InventorySlotRecord[inventorySize];
            for (int i = 0; i < inventorySize; i++)
            {
                if (slots[i].item != null)
                {
                    slotStrings[i].itemID = slots[i].item.GetItemID();
                    slotStrings[i].number = slots[i].number;
                }
            }
            return JToken.FromObject(slotStrings);
        }

        public void RestoreFromToken(JToken state)
        {
            var slotStrings = state.ToObject<InventorySlotRecord[]>();
            slots = new InventorySlot[inventorySize];
            for (int i = 0; i < inventorySize; i++)
            {
                slots[i].item = InventoryItem.GetFromID(slotStrings[i].itemID);
                slots[i].number = slotStrings[i].number;
            }
            inventoryUpdated?.Invoke();
        }

Equipment.cs

        public JToken CaptureAsToken()
        {
            var equippedItemsForSerialization = new Dictionary<EquipLocation, string>();
            foreach (var pair in equippedItems)
            {
                equippedItemsForSerialization[pair.Key] = pair.Value.GetItemID();
            }
            return JToken.FromObject(equippedItemsForSerialization);
        }

        public void RestoreFromToken(JToken state)
        {
            equippedItems = new Dictionary<EquipLocation, EquipableItem>();

            var equippedItemsForSerialization = state.ToObject<Dictionary<EquipLocation, string>>();

            foreach (var pair in equippedItemsForSerialization)
            {
                var item = (EquipableItem)InventoryItem.GetFromID(pair.Value);
                if (item != null)
                {
                    equippedItems[pair.Key] = item;
                }
            }
        }

ActionStore.cs

        public JToken CaptureAsToken()
        {
            var state = new Dictionary<int, DockedItemRecord>();
            foreach (var pair in dockedItems)
            {
                var record = new DockedItemRecord();
                record.itemID = pair.Value.item.GetItemID();
                record.number = pair.Value.number;
                state[pair.Key] = record;
            }

            return JToken.FromObject(state);
        }

        public void RestoreFromToken(JToken state)
        {
            var stateDict = state.ToObject<Dictionary<int, DockedItemRecord>>();
            foreach (var pair in stateDict)
            {
                AddAction(InventoryItem.GetFromID(pair.Value.itemID), pair.Key, pair.Value.number);
            }
        }

ItemDropper.cs

        [System.Serializable]
        private struct JDropRecord
        {
            public string itemID;
            public Vector3 position; //No need for SerializeableVector3
            public int number;
        }
        
        public JToken CaptureAsToken()
        {
            RemoveDestroyedDrops();
            var droppedItemsList = new JDropRecord[droppedItems.Count];
            for (int i = 0; i < droppedItemsList.Length; i++)
            {
                droppedItemsList[i].itemID = droppedItems[i].GetItem().GetItemID();
                droppedItemsList[i].position = droppedItems[i].transform.position;
                droppedItemsList[i].number = droppedItems[i].GetNumber();
            }
            return JToken.FromObject(droppedItemsList);
        }

        public void RestoreFromToken(JToken state)
        {
            var droppedItemsList = state.ToObject<JDropRecord[]>();
            foreach (var item in droppedItemsList)
            {
                var pickupItem = InventoryItem.GetFromID(item.itemID);
                Vector3 position = item.position;
                int number = item.number;
                SpawnPickup(pickupItem, position, number);
            }
        }

PickupSpawner.cs

        public JToken CaptureAsToken()
        {
            return isCollected(); //implicit conversion on primitive type!
        }

        public void RestoreFromToken(JToken state)
        {
            bool shouldBeCollected = state.ToObject<bool>();

            if (shouldBeCollected && !isCollected())
            {
                DestroyPickup();
            }

            if (!shouldBeCollected && isCollected())
            {
                SpawnPickup();
            }
        }

Mana.cs

        public JToken CaptureAsToken()
        {
            return mana.value; //implicit conversion
        }

        public void RestoreFromToken(JToken state)
        {
            mana.value = state.ToObject<float>();
        }

Purse.cs

        public JToken CaptureAsToken()
        {
            return balance;
        }

        public void RestoreFromToken(JToken state)
        {
            balance = state.ToObject<float>();
        }

QuestStatus

Before QuestList will Capture/Restore correctly, we need to change QuestStatus first. Don’t make QuestStatus an IJSonSaveable, as Questlist will call these methods directly without an interface.

        public QuestStatus(JToken objectState)
        {
            var state = objectState.ToObject<QuestStatusRecord>();
            quest = Quest.GetByName(state.questName);
            completedObjectives = state.completedObjectives;
        }
        public JToken CaptureAsToken()
        {
            QuestStatusRecord state = new QuestStatusRecord();
            state.questName = quest.name;
            state.completedObjectives = completedObjectives;
            return JToken.FromObject(state);
        }

QuestList

Be sure to properly set up QuestList or this won’t work

        public JToken CaptureAsToken()
        {
            JObject state = new JObject();
            IDictionary<string, JToken>  stateList = state;
            foreach (QuestStatus status in statuses)
            {
                //We're adding a key of the instance ID just to ensure unique dictionary entries
                stateList.Add($"{status.GetQuest().GetInstanceID()}", status.CaptureAsToken());
            }
            return state;
        }

        public void RestoreFromToken(JToken state)
        {
            IDictionary<string, JToken> stateList = state.ToObject<JObject>();
            
            statuses.Clear();
            foreach (KeyValuePair<string, JToken> objectState in stateList)
            {
                //We're not concerned with the key, just the value
                statuses.Add(new QuestStatus(objectState.Value));
            }
        }
    }

Shop

        public JToken CaptureAsToken()
        {
            Dictionary<string, int> saveObject = new Dictionary<string, int>();

            foreach (var pair in stockSold)
            {
                saveObject[pair.Key.GetItemID()] = pair.Value;
            }

            return JToken.FromObject(saveObject);
        }

        public void RestoreFromToken(JToken state)
        {
            Dictionary<string, int> saveObject = state.ToObject<Dictionary<string, int>>();
            stockSold.Clear();
            foreach (var pair in saveObject)
            {
                stockSold[InventoryItem.GetFromID(pair.Key)] = pair.Value;
            }
        }

TraitStore

        public JToken CaptureAsToken()
        {
            return JToken.FromObject(assignedPoints);
        }

        public void RestoreFromToken(JToken state)
        {
            assignedPoints = state.ToObject <Dictionary<Trait, int>>();
        }

This completes the basics of converting the saving system to use Json. The sections following this will contain some more advanced topics, including applying the Strategy Pattern to our saving system so that we can implement whatever sort of encoding/encryption we choose.

In the next section, we’ll go over some options to save our files as BSON files, and to save them in an encrypted format.

1 Like

Converting to Json Part VI

The pitfalls of Json

So one of the pitfalls of writing to a Json file is that a Json is a human readable file. Actually, for our purposes during prototyping, I consider this to be a strength. A human readable save file means you can actually see what’s going on in the save file when things don’t seem to be quite right. That doesn’t mean, however, that we want to distribute our games using Json. A player could easily edit the files to give themselves inventory items or other forms of mischief.

Here’s a fantastic example, in just a fragment of a save file:

    "RPG.Stats.Experience": 2929.0,
    "GameDevTV.Inventories.ActionStore": {
      "0": {
        "itemID": "9d0578ce-a8ca-4634-9631-dfcc7002476a",
        "number": 1
      }

What this represents (in my game, it may be different in yours) is the Experience of the player, and a single potion in slot 0 of the ActionStore…
Now… what would happen if I edited this section to read:

    "RPG.Stats.Experience": 100000.0,
    "GameDevTV.Inventories.ActionStore": {
      "0": {
        "itemID": "9d0578ce-a8ca-4634-9631-dfcc7002476a",
        "number": 99
      }

Our player would now have 100,000 experience points (gaining a LOT of levels) and would have 99 health potions.

Let’s explore a few options to make our save files a little bit trickier for the users to edit:

Bson - Binary Json

We’ll start with the most obvious. There is another form of Json called Bson, or Binary Serialized Object Format. The structure is very similar, and is actually somewhat readable. Capture|attachment

Anybody familiar with basic hexadecimal mathematics may find that they can alter many of the same attributes, as all of the classes prefix the values that need to be changed… Ultimately, this probably shouldn’t be our final product, but we’ll use this as the starting point to get to a more secure format.

There are actually only a few changes needed to write to Bson instead of Json. I’ll include these as new methods:

        private void SaveFileAsBSon(string saveFile, JObject state)
        {
            string path = GetPathFromSaveFile(saveFile);
            Debug.Log($"Saving BSon to {path}");
            using (FileStream fileStream = File.Open(path, FileMode.Create))
            {
                using (BsonWriter writer = new BsonWriter(fileStream))
                {
                    state.WriteTo(writer);
                }
            }
        }

        private JObject LoadBSonFromFile(string saveFile)
        {
            string path = GetPathFromSaveFile(saveFile);
            Debug.Log($"Loading BSon from {path}");
            if (!UnityEngine.Windows.File.Exists(path))
            {
                return new JObject();
            }

            using (FileStream fileStream = File.Open(path, FileMode.Open))
            {
                using (BsonReader reader = new BsonReader(fileStream))
                {
                    return (JObject) JToken.ReadFrom(reader);
                }
            }
        }

You’ll need to replace LoadJSonFromFile with LoadBSonFromFile in LoadLastScene(), Save() and Load() You’ll need to replace SaveFileAsJSon in Save() You’ll also need to change the extension to a new extension from .json to avoid confusion. I’ve used .bson If you’ve finished the Menu section of the Shops and Abilities course, you’ll also need to change the extension there.

Once you’re done, you should be able to save and restore the game to a Bson file. You’ve probably noticed, at this point, that it may be a bit cumbersome to switch methods… if you want to save in Json for testing and Bson for production all that switching may get very frustrating.

A Strategy For Saving - The Strategy Pattern

In Dialogues and Quests, Sam introduces a new pattern called the Strategy Pattern. This innovative pattern lets you define a behavior or family of behaviors in the main class, but the actual implementation of that behavior in a separate class. You can swap out the implementation (strategy) class to get a different implementation of the behavior. Setting aside the original BinaryFormatter (really, just don’t use it once this is working), we currently have two different strategies for saving and restoring the data collected by CaptureAsToken… we can save and restore to human readable JSon or we can save and restore to Bson… ultimately, we will want to be able to save to an encrypted format (encode to Bson and encrypt the results before saving). We may also want a pattern to save to an encrypted format that can be saved as a text file. That’s a lot of potential methods to use and swap and edit…

What we’re going to do is introduce a new class SavingStrategy. This will serve as a template class for our actual strategy classes.

using System.IO;
using Newtonsoft.Json.Linq;
using UnityEngine;

namespace GameDevTV.Saving
{
    public abstract class SavingStrategy
    {
        public abstract void SaveToFile(string saveFile, JObject state);
        public abstract JObject LoadFromFile(string saveFile);
        public abstract string GetExtension();
        public string GetPathFromSaveFile(string saveFile)
        {
            return Path.Combine(Application.persistentDataPath, saveFile +GetExtension());
        }
    }
}

This is an abstract class with three abstract methods.
SaveToFile will take a JObject and save it in the format associated with the strategy LoadFromFile will deserialize the save file and return the JObject, using the format associated with the strategy GetExtension will return the correct extension to go with the strategy. Finally, we have the GetPathFromSaveFile() method copied from the SavingSystem. This will give the strategies access to the path information they need to properly save/load the files.

So far, we have two viable strategies for saving… we can save to Json or we can save to Bson. For each of these strategies, we’ll write a new class inherited from SavingStrategy JsonStrategy.cs

using System.IO;
using Newtonsoft.Json;
using Newtonsoft.Json.Linq;
using UnityEngine;


namespace GameDevTV.Saving
{
    public class JsonStrategy : SavingStrategy
    {

        public override void SaveToFile(string saveFile, JObject state)
        {
            string path = GetPathFromSaveFile(saveFile);
            Debug.Log($"Saving to {path} ");
            using (var textWriter = File.CreateText(path))
            {
                using (var writer = new JsonTextWriter(textWriter))
                {
                    JsonSerializer serializer = new JsonSerializer();
                    serializer.Serialize(writer, state);
                }
            }
        }

        public override JObject LoadFromFile(string saveFile)
        {
            string path = GetPathFromSaveFile(saveFile);
            if (!File.Exists(path))
            {
                return new JObject();
            }
            
            using (var textReader = File.OpenText(path))
            {
                using (var reader = new JsonTextReader(textReader))
                {
                    reader.FloatParseHandling = FloatParseHandling.Double;

                    return JObject.Load(reader);
                }
            }
        }

        public override string GetExtension()
        {
            return ".json";
        }
    }
}

BsonStrategy.cs

using System.IO;
using Newtonsoft.Json.Bson;
using Newtonsoft.Json.Linq;
using UnityEngine;

namespace GameDevTV.Saving
{
    public class BsonStrategy : SavingStrategy
    {
        public override void SaveToFile(string saveFile, JObject state)
        {
            string path = GetPathFromSaveFile(saveFile);
            Debug.Log($"Saving BSon to {path}");
            using (FileStream fileStream = File.Open(path, FileMode.Create))
            {
                using (BsonWriter writer = new BsonWriter(fileStream))
                {
                    state.WriteTo(writer);
                }
            }
        }

        public override JObject LoadFromFile(string saveFile)
        {
            string path = GetPathFromSaveFile(saveFile);
            Debug.Log($"Loading BSon from {path}");
            if (!UnityEngine.Windows.File.Exists(path))
            {
                return new JObject();
            }

            using (FileStream fileStream = File.Open(path, FileMode.Open))
            {
                using (BsonReader reader = new BsonReader(fileStream))
                {
                    return (JObject) JToken.ReadFrom(reader);
                }
            }
        }

        public override string GetExtension()
        {
            return ".bson";
        }
    }
}

So each of these strategies use the same basic methods we outlined for saving the Json and Bson, but as overrides to the SavingStrategy abstract class.

The next step is to inject one of these strategies into our saving system.

At the top of the SavingSystem class, add a field for a SavingStrategy

SavingStrategy savingStrategy;

You’ll also need to initialize the saving strategy. For now, we’ll just use Awake(). Create an Awake() method and initialize the savingStrategy with the strategy of your choice:

void Awake()
{
    savingStrategy = new JsonStrategy();
}

Now we need to change the methods for saving and loading… In LoadLastScene():

JObject state = savingStrategy.LoadFromFile(saveFile);

In Save():

JObject state = savingStrategy.LoadFromFile(saveFile);
...
savingStrategy.SaveToFile(saveFile, state);

In Load()

RestoreFromToken(savingStrategy.LoadFromFile(saveFile));

In GetPathFromSaveFile()

return Path.Combine(Application.persistentDataPath, saveFile + savingStrategy.GetExtension());

and finally for those of you who have completed the Menu section of the Shops and Ability section, in IEnumerable ListSaves()

if(Path.GetExtension(path) == savingStrategy.GetExtension())

Here’s the final SavingSystem.cs after the modifications:

using System.Collections;
using System.Collections.Generic;
using System.IO;
using Newtonsoft.Json.Linq;
using UnityEngine;
using UnityEngine.SceneManagement;

namespace GameDevTV.Saving
{
    /// <summary>
    /// This component provides the interface to the saving system. It provides
    /// methods to save and restore a scene.
    ///
    /// This component should be created once and shared between all subsequent scenes.
    /// </summary>
    public class SavingSystem : MonoBehaviour
    {
        private SavingStrategy savingStrategy;

        private void Awake()
        {
            savingStrategy = new JsonStrategy();
        }

        /// <summary>
        /// Will load the last scene that was saved and restore the state. This
        /// must be run as a coroutine.
        /// </summary>
        /// <param name="saveFile">The save file to consult for loading.</param>
        public IEnumerator LoadLastScene(string saveFile)
        {
            JObject state = savingStrategy.LoadFromFile(saveFile);
            IDictionary<string, JToken> stateDict = state;
            int buildIndex = SceneManager.GetActiveScene().buildIndex;
            if (stateDict.ContainsKey("lastSceneBuildIndex"))
            {
                buildIndex = (int)stateDict["lastSceneBuildIndex"];
            }
            yield return SceneManager.LoadSceneAsync(buildIndex);
            RestoreFromToken(state);
        }

        /// <summary>
        /// Save the current scene to the provided save file.
        /// </summary>
        public void Save(string saveFile)
        {
            JObject state = savingStrategy.LoadFromFile(saveFile);
            CaptureAsToken(state);
            savingStrategy.SaveToFile(saveFile, state);
        }

        /// <summary>
        /// Delete the state in the given save file.
        /// </summary>
        public void Delete(string saveFile)
        {
            File.Delete(GetPathFromSaveFile(saveFile));
        }

        public void Load(string saveFile)
        {
            RestoreFromToken(savingStrategy.LoadFromFile(saveFile));
        }

        public IEnumerable<string> ListSaves()
        {
            foreach (string path in Directory.EnumerateFiles(Application.persistentDataPath))
            {
                if (Path.GetExtension(path) == savingStrategy.GetExtension())
                {
                    yield return Path.GetFileNameWithoutExtension(path);
                }
            }
        }
      
        private void CaptureAsToken(JObject state)
        {
            IDictionary<string, JToken> stateDict = state;
            foreach (SaveableEntity saveable in FindObjectsOfType<SaveableEntity>())
            {
                stateDict[saveable.GetUniqueIdentifier()] = saveable.CaptureAsToken();
            }

            stateDict["lastSceneBuildIndex"] = SceneManager.GetActiveScene().buildIndex;
        }

        private void RestoreFromToken(JObject state)
        {
            IDictionary<string, JToken> stateDict = state;
            foreach (SaveableEntity saveable in FindObjectsOfType<SaveableEntity>())
            {
                string id = saveable.GetUniqueIdentifier();
                if (stateDict.ContainsKey(id))
                {
                    saveable.RestoreFromToken(stateDict[id]);
                }
            }
        }

        private string GetPathFromSaveFile(string saveFile)
        {
            return Path.Combine(Application.persistentDataPath, saveFile + savingStrategy.GetExtension());
        }
    }
}

Once these changes are made, you should be able to select saving in Json or Bson simply by changing the strategy we initialize in Awake(). No other modifications to SavingSystem will be necessary, as all changes will be made in the strategies.

Try it first with JsonStrategy(), make sure you can save and load the scene. Then change the strategy in Awake() to BsonStrategy() and do it again. The Json save won’t be available, but you’ll find in a new game you can save and load in Bson.

Now that we’ve established a strategy pattern for saving, in the next section we’ll explore a couple of options for encrypting the save file. One pattern for simple encryption, and another pattern to encrypt the file and convert it to a text file that can be sent to text only cloud systems.

1 Like

So far, we’ve seen how to capture and restore the JToken version of object, and how to serialize and deserialize in both human readable Json and quasi human readable Bson. We’ve also seen how to set up a strategy pattern to allow you to choose at compile time which of these formats we wish to use.

This last section will extend that strategy pattern concept to give us a couple more options when it comes to saving/restoring while still making the file more difficult (but never impossible) to alter.

A quick note on encryption: Aside from implementing NSA level encryption, there is no such thing as a file that cannot be decrypted. Very smart eggheads have literally spent decades coming up with new ways to encrypt messages, and more to the point, coming up with ways to decrypt them. It’s a bit like locking your front door. It will stop honest people from breaking into your home, but it’s not a guarantee.

I’m going to present the simplest of cyphers, the XOR Cypher. This cypher takes a simple key and character by character applies an XOR with the corresponding letter of the key and the text to encrypt… For this example, I’m using a modified version of an example from Codingame.com

using System.IO;
using System.Text;
using Newtonsoft.Json.Linq;
using UnityEngine;

namespace GameDevTV.Saving
{
    public class XorStrategy : SavingStrategy
    {
        private const string key = "TheQuickBrownFoxJumpedOverTheLazyDog"; //This is a TERRIBLE KEY
        
        public string EncryptDecrypt(string szPlainText, string szEncryptionKey)  
        {  
            StringBuilder szInputStringBuild = new StringBuilder(szPlainText);  
            StringBuilder szOutStringBuild = new StringBuilder(szPlainText.Length);  
            char Textch;  
            for (int iCount = 0; iCount < szPlainText.Length; iCount++)
            {
                int stringIndex = iCount % szEncryptionKey.Length;
                Textch = szInputStringBuild[iCount];  
                Textch = (char)(Textch ^ szEncryptionKey[stringIndex]);  
                szOutStringBuild.Append(Textch);  
            }  
            return szOutStringBuild.ToString();  
        } 
        
        public override void SaveToFile(string saveFile, JObject state)
        {
            string path = GetPathFromSaveFile(saveFile);
            Debug.Log($"Saving to {path} ");
            using (var textWriter = File.CreateText(path))
            {
                string json = state.ToString();
                string encoded = EncryptDecrypt(json, key);
                textWriter.Write(encoded);
            }
        }

        public override JObject LoadFromFile(string saveFile)
        {
            string path = GetPathFromSaveFile(saveFile);
            if (!File.Exists(path))
            {
                return new JObject();
            }
            
            using (var textReader = File.OpenText(path))
            {
                string encoded = textReader.ReadToEnd();
                string json = EncryptDecrypt(encoded, key);
                return (JObject)JToken.Parse(json);
            }
        }
        public override string GetExtension()
        {
            return ".xor";
        }
    }
}

So there’s a bit to unpack here, as we can’t serialize and deserialize directly to the filestreams. The first method is our modified encryption cypher. Because this particular cypher uses the key to both encode and decode the file, the same method can be used to flip the bits on the file. It’s imperative that once you’ve chosen a key, you never change it, as it will render any future files unreadable.
If you use a text editor to read the encoded xor file, you’ll note some deficiencies in the algorythm… you’ll find places where you can see sections of the key… even if I were not to have shown you the key, it might not take you very long to figure out that it is the phrase used to teach young typists how to use every key in the alphabet. You’ll see fragments of the phrase in places due to the nature of the algorithm when certain complementary values are encountered. You can minimize this risk by using keys with more random characters. Try copying a UUID from one of your characters and using that as the key. The important thing is once you’ve set a key, you can’t change it or old save files will be unreadable.

The final strategy I’m going to leave you with is what I’m going to call the XorTextStrategy. Base64 encoding is a way of using only “readable” text characters to represent a binary file. It allows sending the file via email or over http as plain text (doing this with the results of Bson or the XorStrategy could cause accidental control characters to be interpreted, so transferring the files has to be done as binary files). While transferring binary files isn’t ordinarily a problem, some 3rd party cloud solutions will only accept pure text files. The XorTextStrategy will add an additional layer of encoding by converting the final product into Base64 encoding.

using System;
using System.IO;
using System.Text;
using Newtonsoft.Json.Linq;
using UnityEngine;

namespace GameDevTV.Saving
{
    public class XorTextStrategy : SavingStrategy
    {
        private const string key = "TheQuickBrownFoxJumpedOverTheLazyDog"; //This is a TERRIBLE KEY
       
        
        public string EncryptDecrypt(string szPlainText, string szEncryptionKey)  
        {  
            StringBuilder szInputStringBuild = new StringBuilder(szPlainText);  
            StringBuilder szOutStringBuild = new StringBuilder(szPlainText.Length);  
            char Textch;  
            for (int iCount = 0; iCount < szPlainText.Length; iCount++)
            {
                int stringIndex = iCount % szEncryptionKey.Length;
                Textch = szInputStringBuild[iCount];  
                Textch = (char)(Textch ^ szEncryptionKey[stringIndex]);  
                szOutStringBuild.Append(Textch);  
            }  
            return szOutStringBuild.ToString();  
        } 

        string EncodeAsBase64String(string source)
        {
            byte[] sourceArray = new byte[source.Length];
            for (int i = 0; i < source.Length; i++)
            {
                sourceArray[i] = (byte)source[i];
            }
            return Convert.ToBase64String(sourceArray);
        }

        string DecodeFromBase64String(string source)
        {
            byte[] sourceArray = Convert.FromBase64String(source);
            StringBuilder builder = new StringBuilder();
            for (int i = 0; i < sourceArray.Length; i++)
            {
                builder.Append((char) sourceArray[i]);
            }

            return builder.ToString();
        }
        
        public override void SaveToFile(string saveFile, JObject state)
        {
            string path = GetPathFromSaveFile(saveFile);
            Debug.Log($"Saving to {path} ");
            using (var textWriter = File.CreateText(path))
            {
                string json = state.ToString();
                string encoded = EncryptDecrypt(json, key);
                string base64 = EncodeAsBase64String(encoded);
                textWriter.Write(base64);
            }
        }

        public override JObject LoadFromFile(string saveFile)
        {
            string path = GetPathFromSaveFile(saveFile);
            if (!File.Exists(path))
            {
                return new JObject();
            }
            
            using (var textReader = File.OpenText(path))
            {
                string encoded = textReader.ReadToEnd();
                string decoded = DecodeFromBase64String(encoded);
                string json = EncryptDecrypt(decoded, key);
                return (JObject)JToken.Parse(json);
            }
        }
        public override string GetExtension()
        {
            return ".xortext";
        }
    }
}

While this will add an extra layer of obscurity to the save file (Base64 encoding uses 6 bits per character meaning that an invididual byte of the original save spans 2 characters, but in different ways depending on it’s position in the file), anybody familiar with Base64 encoding will be able to decode the file back to it’s original format. This is why I’m using it combined with the XOR, to allow an extra layer of security.

This concludes the Converting to Json tutorial. Hopefully, you’ve learned how to adapt your game’s Captures and Restores to be JTokens, and how to save and load your game data to and from Json files.

This tutorial’s final commit

3 Likes

An attempt to replace saving system with XML
I have just posted this. I don’t know if this works better than the Json but it may worth looking into.
Thanks.

Hey Brian, I’m deciding to follow and gut out the current saving system and replace it with your work. I’m a little puzzled, according to your Important Information on the original post it states that this set up isn’t build ready, is that the case still since 21?

Edit:

In the SaveableEntity changes post you have

        private void CaptureAsToken(JObject state)
        {
            IDictionary<string, JToken> stateDict = state;
            foreach (SaveableEntity saveable in FindObjectsOfType<SaveableEntity>())
            {
                stateDict[saveable.GetUniqueIdentifier()] = saveable.CaptureState(); // CaptureAsToken()
            }

            stateDict["lastSceneBuildIndex"] = SceneManager.GetActiveScene().buildIndex;
        }

I had to change the CaptureState() to CaptureAsToken(). Fingers cross that’s what was intended

So I’ve just completed the end of the saving system section, and I did a simple transform catpure for my movement script on the player

public JToken CaptureAsToken()
        {
            return JToken.FromObject(transform.position);
        }

        public void RestoreFromToken(JToken state)
        {
            transform.position = state.ToObject<Vector3>();
        }

But it errors out saying I cannot serialize a normalize path “.” Vector3. Everything else seems to be saving as intended. Here’s the error in relation to the code

JsonSerializationException: Self referencing loop detected for property 'normalized' with type 'UnityEngine.Vector3'. Path ''.
Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.CheckForCircularReference (Newtonsoft.Json.JsonWriter writer, System.Object value, Newtonsoft.Json.Serialization.JsonProperty property, Newtonsoft.Json.Serialization.JsonContract contract, Newtonsoft.Json.Serialization.JsonContainerContract containerContract, Newtonsoft.Json.Serialization.JsonProperty containerProperty) (at <bc3985d37b0241b48fc21474b2de25bd>:0)
Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.CalculatePropertyValues (Newtonsoft.Json.JsonWriter writer, System.Object value, Newtonsoft.Json.Serialization.JsonContainerContract contract, Newtonsoft.Json.Serialization.JsonProperty member, Newtonsoft.Json.Serialization.JsonProperty property, Newtonsoft.Json.Serialization.JsonContract& memberContract, System.Object& memberValue) (at <bc3985d37b0241b48fc21474b2de25bd>:0)
Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.SerializeObject (Newtonsoft.Json.JsonWriter writer, System.Object value, Newtonsoft.Json.Serialization.JsonObjectContract contract, Newtonsoft.Json.Serialization.JsonProperty member, Newtonsoft.Json.Serialization.JsonContainerContract collectionContract, Newtonsoft.Json.Serialization.JsonProperty containerProperty) (at <bc3985d37b0241b48fc21474b2de25bd>:0)
Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.SerializeValue (Newtonsoft.Json.JsonWriter writer, System.Object value, Newtonsoft.Json.Serialization.JsonContract valueContract, Newtonsoft.Json.Serialization.JsonProperty member, Newtonsoft.Json.Serialization.JsonContainerContract containerContract, Newtonsoft.Json.Serialization.JsonProperty containerProperty) (at <bc3985d37b0241b48fc21474b2de25bd>:0)
Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.Serialize (Newtonsoft.Json.JsonWriter jsonWriter, System.Object value, System.Type objectType) (at <bc3985d37b0241b48fc21474b2de25bd>:0)
Newtonsoft.Json.JsonSerializer.SerializeInternal (Newtonsoft.Json.JsonWriter jsonWriter, System.Object value, System.Type objectType) (at <bc3985d37b0241b48fc21474b2de25bd>:0)
Newtonsoft.Json.JsonSerializer.Serialize (Newtonsoft.Json.JsonWriter jsonWriter, System.Object value) (at <bc3985d37b0241b48fc21474b2de25bd>:0)
Newtonsoft.Json.Linq.JToken.FromObjectInternal (System.Object o, Newtonsoft.Json.JsonSerializer jsonSerializer) (at <bc3985d37b0241b48fc21474b2de25bd>:0)
Newtonsoft.Json.Linq.JToken.FromObject (System.Object o) (at <bc3985d37b0241b48fc21474b2de25bd>:0)
Game.Control.WorldMovement.CaptureAsToken () (at Assets/Scripts/Control/WorldMovement.cs:148)
Game.Saving.JsonSaveableEntity.CaptureAsJtoken () (at Assets/Scripts/Core/Saving/JsonSaveableEntity.cs:30)
Game.Saving.SavingSystem.CaptureAsToken (Newtonsoft.Json.Linq.JObject state) (at Assets/Scripts/Core/Saving/SavingSystem.cs:226)
Game.Saving.SavingSystem.Save (System.String saveFile) (at Assets/Scripts/Core/Saving/SavingSystem.cs:70)
Game.SceneManagement.SavingWrapper.Save () (at Assets/Scripts/Core/Saving/SavingWrapper.cs:117)
Game.SceneManagement.SavingWrapper.Update () (at Assets/Scripts/Core/Saving/SavingWrapper.cs:70)

No idea why it’s only doing it when trying to serialize my player’s transform position.

Hopefully, you’re using the version of the saving system from my Wiki mentioned at the top of this thread: SavingSystem as opposed to what’s in this post.

In that system, the issues with Vector3 are discussed, along with a solution: First, we need a quick Static class to add some extensions methods.

public static class JsonStatics
    {

        public static JToken ToToken(this Vector3 vector)
        {
            JObject state = new JObject();
            IDictionary<string, JToken> stateDict = state;
            stateDict["x"] = vector.x;
            stateDict["y"] = vector.y;
            stateDict["z"] = vector.z;
            return state;
        }

        public static Vector3 ToVector3(this JToken state)
        {
            Vector3 vector = new Vector3();
            if (state is JObject jObject)
            {
                IDictionary<string, JToken> stateDict = jObject;

                if (stateDict.TryGetValue("x", out JToken x))
                {
                    vector.x = x.ToObject<float>();
                }

                if (stateDict.TryGetValue("y", out JToken y))
                {
                    vector.y = y.ToObject<float>();
                }

                if (stateDict.TryGetValue("z", out JToken z))
                {
                    vector.z = z.ToObject<float>();
                }
            }
            return vector;
        }
    }

Now our new CaptureAsJToken and RestoreAsJToken becomes

public JToken CaptureAsJToken()
{
    return transform.position.ToToken();
}
public void RestoreFromJToken(JToken state)
{
    transform.position = state.ToVector3();
}
1 Like

I think I’m getting the hang of the new syntax with JSON.

Had to try to go over your code and get a feel of it but I managed to get all my skill information saved. Here’s what I did code wise in the past with the original object capture/restore states.

Old Captrure/RestoreState – SkillTree.cs

[System.Serializable]
struct SaveData
{ 
     public List<bool> skillUnlocked;
     public List<bool> skillInUse;
     public List<int> levelCap;
     public List<int> level;
     public int points;
     public string pointsText;
}

public object CaptureState()
{
    SaveData data = new SaveData();

    data.level = new List<int>();
    data.levelCap = new List<int>();
    data.skillUnlocked = new List<bool>();
    data.skillInUse = new List<bool>();
            
    for (int i = 0; i < skillsList.Count; i++)
    {
         data.skillUnlocked.Add(skillsDictionary[i].Unlocked);
         data.skillInUse.Add(skillsDictionary[i].SkillInUse);
         data.level.Add(skillsDictionary[i].SkillLevel);
         data.levelCap.Add(skillsDictionary[i].SkillCap);
    }
    data.points = currentPoints;
    return data;
}

public void RestoreState(object state)
{
     SaveData loadData = (SaveData)state;
            
     for (int i = 0; i < skillsList.Count; i++)
     {
          skillsDictionary[i].SkillLevel = loadData.level[i];
          skillsDictionary[i].SkillCap = loadData.levelCap[i];
          skillsDictionary[i].Unlocked = loadData.skillUnlocked[i];
          skillsDictionary[i].SkillInUse = loadData.skillInUse[i];
          skillsUnlocked[i] = loadData.skillUnlocked[i];
     }
     currentPoints = loadData.points;
}

To now the new Capture/Restore JToken states – (I took the same approach as you done with the Inventory Brian) and trying to learn how to apply this for other custom scripts that will need saving in the future. :slight_smile:

public JToken CaptureAsJToken()
{
      JObject state = new JObject();
      IDictionary<string, JToken> stateDict = state;

      for (int i = 0; i < skillsList.Count; i++)
      {
           if (skillsList[i].SkillLevel > 0)
           {
               JObject skillState = new JObject();
               IDictionary<string, JToken> skillStateDict = skillState;
               skillState["currentPoints"] = JToken.FromObject(currentPoints);
               skillState["skillLevel"] = JToken.FromObject(skillsDictionary[i].SkillLevel);
               skillState["skillUnlocked"] = JToken.FromObject(skillsDictionary[i].Unlocked);
               skillState["skillInUse"] = JToken.FromObject(skillsDictionary[i].SkillInUse);
               stateDict[i.ToString()] = skillState;
           }
      }
      return state;
}

public void RestoreFromJToken(JToken state)
{
      if (state is JObject stateObject)
      {
           IDictionary<string, JToken> stateDict = stateObject;

           for (int i = 0; i < skillsList.Count; i++)
           {
                if (stateDict.ContainsKey(i.ToString()) && stateDict[i.ToString()] is JObject skillState)
                {
                      IDictionary<string, JToken> skillStateDict = skillState;
                      currentPoints = skillStateDict["currentPoints"].ToObject<int>();
                      skillsDictionary[i].SkillLevel = skillStateDict["skillLevel"].ToObject<int>();
                      skillsDictionary[i].Unlocked = skillStateDict["skillUnlocked"].ToObject<bool>();
                      skillsDictionary[i].SkillInUse = skillStateDict["skillInUse"].ToObject<bool>();
                      skillsUnlocked[i] = skillsList[i].Unlocked;
                }
           }
      }
}

And then I reused the same TraitStore.cs changes to my AbilityStore.cs and it works wonderfully. Now it’s time to move on to the section to encrypt the data. Looking forward to learning more about this topic. Thanks a bunch Brian!

Is this system still on hold/incomplete/not working?? I dont understand. Its 1.5 years later and no update? The link to the wiki is broken, gone??

Hmmm I thought I had put up the link to the current version of the system:

2 Likes

Thank you! So this version is fully operational then? The one in the wiki?

Yep

1 Like

Awesome thank you!!!

I have not read through any of this (sorry) but I have implemented my own JToken version of the saving system. I’m not going to put it here because @Brian_Trotter’s version here is sure to be excellent.

What I have done (I didn’t see it in a quick scan of the post) was the ability to compress the json before storing it (I don’t have a bson version). I have the compression/decompression in a compiler directive so that the json remains uncompressed in editor that will allow me to check and verify my save data during development, but compress it in a build.

The compression code

using System.IO;
using System.IO.Compression;

public static class Utilities
{
    public static byte[] Compress(byte[] inputBytes)
    {
        using var outputStream = new MemoryStream();
        using var gzipStream = new GZipStream(outputStream, CompressionLevel.Fastest);

        gzipStream.Write(inputBytes, 0, inputBytes.Length);
        return outputStream.ToArray();

    }
    public static byte[] Decompress(byte[] inputBytes)
    {
        using var inputStream = new MemoryStream(inputBytes);
        using var gzipStream = new GZipStream(inputStream, CompressionMode.Decompress);
        using var outputStream = new MemoryStream();

        gzipStream.CopyTo(outputStream);
        return outputStream.ToArray();
    }
}

In my saving system, I convert the json into a UTF8 encoded byte array and then send it to be compressed, or I decompress it, and then decode the UTF8 bytes back to a string.

private byte[] Compress(string json) => Compress(json, Encoding.UTF8);
private byte[] Compress(string json, Encoding encoding)
{
    var decompressedBytes = encoding.GetBytes(json);
    return Utility.Utilities.Compress(decompressedBytes);
}

private string Decompress(byte[] compressed) => Decompress(compressed, Encoding.UTF8);
private string Decompress(byte[] compressed, Encoding encoding)
{
    var decompressedBytes = Utility.Utilities.Decompress(compressed);
    return encoding.GetString(decompressedBytes);
}

The result is a gzipped file that is not human readable and quite a bit smaller than the uncompressed save. It’s not encrypted and anyone that figures out that it is compressed can decompress it, but like someone recently mentioned to me; It doesn’t really matter if a person playing a single-player game can alter the save file. They’re only affecting their own gameplay and do so at their own peril. It’s with multiplayer games where you want to be certain this doesn’t happen in which case you’d probably not store the save on the local machine but in the cloud, away from prying eyes.
That being said, with a small tweak it would be possible to ‘XOR Cypher’ the compressed bytes before writing it to file.

1 Like
        public JToken CaptureAsJToken()
        {
            JObject state = new JObject();
            IDictionary<string, JToken> stateDict = state;
            foreach (var pair in dockedItems)
            {
                JObject dockedState = new JObject();
                IDictionary<string, JToken> dockedStateDict = dockedState;
                dockedStateDict["item"] = JToken.FromObject(pair.Value.item.GetItemID());
                dockedStateDict["number"] = JToken.FromObject(pair.Value.number);
                stateDict[pair.Key.ToString()] = dockedState;
            }
            return state;
        }

        public void RestoreFromJToken(JToken state)
        {
            if (state is JObject stateObject)
            {
                IDictionary<string, JToken> stateDict = stateObject;
                foreach (var pair in stateDict)
                {
                    if(pair.Value is JObject dockedState)
                    {
                        int key = Int32.Parse(pair.Key);
                        IDictionary<string, JToken> dockedStateDict= dockedState;
                        var item = InventoryItem.GetFromID(dockedStateDict["item"].ToObject<string>());
                        int number = dockedStateDict["number"].ToObject<int>();
                        AddAction(item, key, number);
                    }
                }
            }
        }

Hi apologies for the reply just wanted to say that I followed the implementation json from the actual course and in ActionStore, Restore function: AddAction(item, key, number); found this line duplicating the objects in my action bar when reloading. Now if anyone else is finding this I solved it by calling storeUpdated?.Invoke(); instead of this line. Now I have noticed that in this post the Capture and Restore are differently slightly from the git repo so perhaps this is build with it in mind. Just wanted to mention in case anyone else found the same as me. Also on my testing thus far it seems to work. Still need to do more testing though.

Upon testing I just realized that this doesn’t save the problem of loading between scenes it actually doesn’t keep the items through the scenes in your action bar. I was able to find a solution to mitigate this problem.

public void AddAction(InventoryItem item, int index, int number)
        {
            if (item == null || number <= 0)
            {
                // Wrong item or quantity, do nothing
                return;
            }

            if (dockedItems.ContainsKey(index))
            {
                // Slot already contains an item
                if (object.ReferenceEquals(item, dockedItems[index].item))
                {
                    // Item already exists in the slot, update quantity
                    dockedItems[index].number += number;
                }
                else
                {
                    // (prevent duplicates)
                    return;
                }
            }
            else
            {
                // Slot is empty, add the item
                var slot = new DockedItemSlot();
                slot.item = item as ActionItem;
                slot.number = number;
                dockedItems[index] = slot;
            }

            if (storeUpdated != null)
            {
                storeUpdated();
            }
        }

Made changes to the restore to take in consideration if items are already there and when to add the item. Upon some tests this seems to work. Sorry for the many messages. Thank you for the awesome community here. :slight_smile:

Privacy & Terms