That pesky Duplicate Bug... (InventoryItem)

Notice: Since Unity 2021, this check will result in Unity throwing an error, do not use this

So I discovered a potential “bug” in the code for randomly generating the UUIDs in InventoryItem… If you Duplicate an InventoryItem in the inspector, the UUID for both items will be the same. When you go to retrieve an item, the GetFromID() method will complain about two (or possibly many more) items with the same UUID, and you’ll only be able to retrieve one of the items… even if all three items were saved somewhere…
Fortunately, there’s a solution, though it will require the help of Linq

Add this to your usings clause: using System.Linq;

Then in the ISerializationCallbackReceiver.OnBeforeSerialization() change your method to look like this:

void ISerializationCallbackReceiver.OnBeforeSerialize()
        {
            // Generate and save a new UUID if this is blank.
            if (string.IsNullOrWhiteSpace(itemId))
            {
                itemID = Guid.NewGuid().ToString();
            }
            // Test for multiple objects with the same UUID
            var items = Resources.LoadAll<InventoryItem>(""). //continues below
                       Where(p => p.GetItemID() == itemID).ToList();
            if (items.Count > 1)
            {
                itemID = Guid.NewGuid().ToString();
            }
        }

What this does, in a nutshell, is query the Resources, basically “give me a list of every InventoryItem with this itemID and put it in a List for me”. It’s a magical Query device that System.Linq provides.
The object you’re editing is one of those objects. If there is a duplicate object, then items.Count will be greater than 1. If this is the case, then we change the UUID. If the count is 1, then there is no need to change the UUID of the item.

6 Likes

I love how often you use LINQ, it really helps connect the dots with my day job SQL query brain.

This would be an excellent 'Brian’s tips and tricks" candidate I think.

Takes again!

1 Like

I’m definitely a big fan of LINQ. As long as you’re not doing big sorts every frame (like in an update loop), it’s a very powerful tool.

1 Like

I know I’m late to the party, but as I wait for your upcoming 3rd person <-> RPG tutorial, I decided to see how I can improve my project to make it more robust. This is definitely going to save me from a truck load of headaches down the line (because this bug absolutely bothered me once). Thank you again Brian :slight_smile:

Hi again, sorry if I’m being annoying, but… There’s a bug coming out of this code, in my main project:

Objects are trying to be loaded during a domain backup. This is not allowed as it will lead to undefined behaviour!

UnityEngine.Resources:LoadAll<GameDevTV.Inventories.InventoryItem> (string)
GameDevTV.Inventories.InventoryItem:UnityEngine.ISerializationCallbackReceiver.OnBeforeSerialize () (at Assets/GameDev.tv Assets/Scripts/Inventories/InventoryItem.cs:139)

This is line 139 it’s referring to:

var items = Resources.LoadAll<InventoryItem>("").Where(p => p.GetItemID() == itemID).ToList();

Do I ignore this?

I started getting these messages when the jobs system came out. This check is now obsolete.

2 Likes

No clue what the jobs system is, but okay I guess… :slight_smile:

To make the Editor more performant, it now uses true multitasking under the hood when dealing with serialization, object loading and saving, etc. The long and short of it is that the right hand doesn’t know what the left hand is doing and while the Linq expression is busy trying to sort and test for duplicates, another thread is actually saving the asset, and the two things create a merge conflict.

So one is searching duplicates, and the other is saving them… is Unity trying to destroy itself by accident? :stuck_out_tongue_winking_eye:

1 Like

[I WILL NEED TO DELETE THE CODE FOR THIS POST IN MY PROJECT, BUT ANYWAY…]

Umm… @Brian_Trotter - I have a bit of a funky problem. By now, you know I’m working on a pirate system for almost 3 months now, but I ran into a funny saving and restoring issue that has to do with the JSON Saving system. To be clear, I got the same NPC on more than one boat in the game, and they both have the same Unique ID. So if I save and quit the game and one NPC is dead, if he’s on an entirely different boat as well, that doppelganger will die as well. Any idea how to fix this? I suspect it has something to do with ‘JSONSavingSystem.Update()’:

        private void Update()
        {
            if (Application.IsPlaying(gameObject)) return;
            if (string.IsNullOrEmpty(gameObject.scene.path)) return;

            SerializedObject serializedObject = new SerializedObject(this);
            SerializedProperty property = serializedObject.FindProperty("uniqueIdentifier");

            if (string.IsNullOrEmpty(property.stringValue) || !IsUnique(property.stringValue))
            {
                property.stringValue = System.Guid.NewGuid().ToString();
                serializedObject.ApplyModifiedProperties();
            }

            globalLookup[property.stringValue] = this;
        }

And to narrow it down even further, here’s my ‘IsUnique’:

        private bool IsUnique(string candidate) 
        {
            if (!globalLookup.ContainsKey(candidate)) return true;

            if (globalLookup[candidate] == this) return true;

            if (globalLookup[candidate] == null) 
            {
                globalLookup.Remove(candidate);
                return true;
            }

            if (globalLookup[candidate].GetUniqueIdentifier() != candidate) 
            {
                globalLookup.Remove(candidate);
                return true;
            }

            return false;

        }

I still have to figure out why they float in a completely different position when I return to the game, but for now this is a priority bug

And here’s my entire script, if it helps:

using System;
using System.Collections.Generic;
using Newtonsoft.Json.Linq;
using UnityEditor;
using UnityEngine;

namespace GameDevTV.Saving 
{
    [ExecuteAlways]
    public class JSONSaveableEntity : MonoBehaviour, ISerializationCallbackReceiver 
    {
        [SerializeField] string uniqueIdentifier = "";

        // CACHED STATE
        static Dictionary <string, JSONSaveableEntity> globalLookup = new Dictionary<string, JSONSaveableEntity>();

        public string GetUniqueIdentifier() 
        {
            return uniqueIdentifier;
        }

        public JToken CaptureAsJToken()
        {
            JObject state = new JObject();
            IDictionary<string, JToken> stateDict = state;

            foreach (IJsonSaveable JSONSaveable in GetComponents<IJsonSaveable>())
            {
                JToken token = JSONSaveable.CaptureAsJToken();
                string component = JSONSaveable.GetType().ToString();
                Debug.Log($"{name} Capture {component} = {token.ToString()}");
                stateDict[JSONSaveable.GetType().ToString()] = token;
            }

            return state;
        }

        public void RestoreFromJToken(JToken s) 
        {
            JObject state = s.ToObject<JObject>();
            IDictionary<string, JToken> stateDict = state;

            foreach (IJsonSaveable jsonSaveable in GetComponents<IJsonSaveable>()) 
            {
                string component = jsonSaveable.GetType().ToString();
                if (stateDict.ContainsKey(component)) 
                {
                    Debug.Log($"{name} Restore {component} => {stateDict[component].ToString()}");
                    jsonSaveable.RestoreFromJToken(stateDict[component]);
                }
            }
        }

#if UNITY_EDITOR

        private void Update()
        {
            if (Application.IsPlaying(gameObject)) return;
            if (string.IsNullOrEmpty(gameObject.scene.path)) return;

            SerializedObject serializedObject = new SerializedObject(this);
            SerializedProperty property = serializedObject.FindProperty("uniqueIdentifier");

            if (string.IsNullOrEmpty(property.stringValue) || !IsUnique(property.stringValue))
            {
                property.stringValue = System.Guid.NewGuid().ToString();
                serializedObject.ApplyModifiedProperties();
            }

            globalLookup[property.stringValue] = this;
        }

#endif

        private bool IsUnique(string candidate) 
        {
            if (!globalLookup.ContainsKey(candidate)) return true;

            if (globalLookup[candidate] == this) return true;

            if (globalLookup[candidate] == null) 
            {
                globalLookup.Remove(candidate);
                return true;
            }

            if (globalLookup[candidate].GetUniqueIdentifier() != candidate) 
            {
                globalLookup.Remove(candidate);
                return true;
            }

            return false;

        }

        public void OnBeforeSerialize()
        {
            if (string.IsNullOrEmpty(uniqueIdentifier) || !IsUnique(uniqueIdentifier))
            {
                uniqueIdentifier = Guid.NewGuid().ToString();
            }
        }

        public void OnAfterDeserialize()
        {
            // Function to silence the 'ISerializationCallbackReceiver' interface from complaining
        }
    }
}

You can always change the id manually by clearing it in the inspector… If the character is in different scenes, there is no way to ensure an overlap does not exist, because each scene is treated separately.

Let’s make a radical change to the JsonSaveableEntity…

  • Remove using UnityEditor;
  • Remove [ExecuteAlways]
  • Remove the ,ISerializationCallbackReceiver from the class declaration.
  • Remove the static Dictionary<string, JSONSaveableEntity> globalLookup;
  • Completely remove the entire #if UNITY_EDITOR block as well as the OnBeforeSerialize() and OnAfterSerialize() methods.

Next, create a folder in the same directory as your JSONSaveableEntity script. Name it Editor. Unity automatically removes this folder from any builds.

Finally, add this script (which is the finalized version of the editor script we’ll be using in the updated Saving System) to the new Editor folder:

JsonSaveableEntityEditor.cs
using GamedevTV.Saving.Json;
using UnityEditor;
using UnityEngine;

namespace GameDevTV.Saving.Json.Editors
{
    /// <summary>
    /// Custom editor for the <see cref="JSONSaveableEntity"/> component, providing a
    /// user interface in the Unity Editor for managing the entity's identifier.
    /// </summary>
    [UnityEditor.CustomEditor(typeof(JSONSaveableEntity))]
    public class JsonSaveableEntityEditor : Editor
    {
        
        /// <summary>
        /// A serialized property representing a unique identifier for the JSON saveable entity.
        /// The identifier is used to ensure that each entity can be uniquely identified and managed.
        /// A note on SerializedProperty.  For best results, always make sure that the property's name
        /// is exactly what it is in the class (case sensitive!).  This will ensure that it's easy
        /// to look up the property in the SerializedObject using the nameof() feature.
        /// </summary>
        private SerializedProperty uniqueIdentifier; 
        
        /// <summary>
        /// Holds a reference to the JSONSaveableEntity instance for the editor script.
        /// </summary>
        private JSONSaveableEntity entity;

        /// <summary>
        /// A boolean variable indicating whether the identifier of the JsonSaveableEntity must be changed.
        /// </summary>
        private bool mustChange;

        /// <summary>
        /// This method is called when the editor becomes enabled and active.
        /// It initializes the serialized property and references to the associated JsonSaveableEntity,
        /// and checks if the entity's identifier needs to be changed to ensure uniqueness
        /// and avoid conflicts in the Unity Editor.
        /// </summary>
        private void OnEnable()
        {
            uniqueIdentifier = serializedObject.FindProperty(nameof(uniqueIdentifier));
            entity = target as JSONSaveableEntity;
            if (!IsPrefab() && !IsUnique(uniqueIdentifier.stringValue))
            {
                mustChange = true;
            }
        }

        /// <summary>
        /// Custom inspector GUI for the JsonSaveableEntity component. This method handles the display and management
        /// of the identifier field in Unity's inspector. It ensures that the identifier field is unique and not empty.
        /// If the component is a prefab, it clears the identifier. If it's an instance in the scene, it provides a
        /// text field for user input and a mechanism to ensure uniqueness and validity of the identifier.
        /// </summary>
        public override void OnInspectorGUI()
        {
            if (IsPrefab())
            {
                if (uniqueIdentifier.stringValue != "")
                {
                    uniqueIdentifier.stringValue = "";
                }
            }
            else
            {
                EditorGUI.BeginChangeCheck();
                string newValue = EditorGUILayout.TextField("ID", uniqueIdentifier.stringValue);
                if (EditorGUI.EndChangeCheck() || string.IsNullOrWhiteSpace(uniqueIdentifier.stringValue) || mustChange)
                {
                    if (IsUnique(newValue))
                    {
                        uniqueIdentifier.stringValue = newValue;
                    }
                    else
                    {
                        uniqueIdentifier.stringValue = System.Guid.NewGuid().ToString();
                    }
                    mustChange = false;
                }
            }
            serializedObject.ApplyModifiedProperties();
        }

        /// <summary>
        /// Determines if the current <see cref="JSONSaveableEntity"/> is a prefab.
        /// </summary>
        /// <returns>True if the entity is a prefab; otherwise, false.</returns>
        bool IsPrefab()
        {
            return !entity.gameObject.scene.IsValid();
        }

        /// <summary>
        /// Checks if the provided identifier is unique among all instances of JsonSaveableEntity within the scene.
        /// Note that if you copy and paste a character from one scene to another, this method may not catch the
        /// change.  It's best instead to prefab your characters and bring them in from the assets, which would
        /// always ensure that a new identifier is generated.
        /// </summary>
        /// <param name="value">The identifier string to be checked for uniqueness.</param>
        /// <returns>True if the identifier is unique; otherwise, false.</returns>
        bool IsUnique(string value)
        {
            if (string.IsNullOrWhiteSpace(value)) return false;
            foreach (JsonSaveableEntity saveableEntity in FindObjectsByType<JsonSaveableEntity>(FindObjectsSortMode.None))
            {
                if (saveableEntity.GetIdentifier() == value)
                {
                    if (object.ReferenceEquals(entity, saveableEntity)) continue;
                    return false;
                }
            }
            return true;
        }
    }
}

This editor is a little more robust, but there are a few features of which you should be aware:

  • If the gameObject the SaveableEntity is on is tagged “Player”, the editor automagically serializes the uniqueIdentifier to be “Player” and instead of a field to edit, a notification is shown that this is the player. One of the most common issues with the saving system is people using Player in one scene and player in another… this utterly and completely eliminates this issue.
  • If the entity is in a prefab, an empty string is serialized with no way to edit. This ensures that every time a character is dragged in from the assets, it’s forced to receive a new UniqueID.
  • Like the original setup, this will check to determine if the current identifier clashes with any other identifier name, and if it does, it will automatically replace it with a new identifier.
  • There is no setup possible to prevent the same identifier in two different scenes.

Yeah, the Linq-based solution identifies and corrects duplicate UUIDs. However, loading all Resources might impact performance in large projects maybe a more targeted approach?

The entire ‘IsUnique’ function depends on that, though, so I just deleted it as well. That aside, I’ll continue anyway with the instructions

We don’t throw the new script anywhere in the scene, right?

That aside, the Doppelgangers still die with the death of the NPC I just killed, and I suspect its because the change of the Unique ID happens in ‘OnEnable’ in your new Editor script (although to my big surprise, this problem doesn’t exist for NPCs on land), whereas the respawning of these guys when returning from the save happens in RestoreFromJToken, which is before start, which is before OnEnable, so everything that matters happens before the Unique IDs are checked for, and that’s some twisted logic

So now I have a serious problem. Any idea how to solve this? @Brian_Trotter (Unfortunately I can’t ignore this one, as its a game-breaking bug)

The bigger problem is, this doesn’t happen with NPCs spawned on the ground, and I am guessing its because Boats Respawn the AggroGroup, RespawnManager and the enemy (this had to be done to avoid a game-breaking bug) alltogether, whereas the ground NPCs have AggroGroup and RespawnManager static, but the enemies are the ones being respawned


Edit: OK I barely got something to work for a moment before I got confused again, so here’s what I’m trying to do:

  1. Ensure the boat allies (NPCs with boat state machines as their parents) have unique Identifiers. I got that covered when they respawn

Here’s how I check for Unique Identifiers the moment these boat allies respawn (because that’s the only way to get this check to work before ‘Start()’):

// in 'JSONSaveableEntity.cs':

        // TEST 22/11/2024

        /// <summary>
        /// Determines if a given unique identifier is unique in the scene.
        /// </summary>
        public bool IsUnique(string id)
        {
            if (string.IsNullOrWhiteSpace(id)) return false;

            var allEntities = FindObjectsOfType<JSONSaveableEntity>();
            return allEntities.All(entity => entity.GetUniqueIdentifier() != id);
        }

        // TEST 22/11/2024
        public void SetUniqueIdentifier(string id) => uniqueIdentifier = id;

// in 'RespawnManager.BoatAllyRespawn()' (the function responsible for respawning boat allies):
// This code runs the moment the NPC is instantiated in the game scene

            // TEST 22/11/2024 - Make sure the Unique Identifier is Unique
            if (spawnedEnemy.GetComponent<JSONSaveableEntity>() != null)
            {
                if (spawnedEnemy.GetComponent<JSONSaveableEntity>().GetUniqueIdentifier() != null)
                {
                    var jsonSaveableEntity = spawnedEnemy.GetComponent<JSONSaveableEntity>();
                    var uniqueIdentifier = spawnedEnemy.GetComponent<JSONSaveableEntity>().GetUniqueIdentifier();
                    if (!jsonSaveableEntity.IsUnique(uniqueIdentifier))
                    {
                        var newUniqueID = System.Guid.NewGuid().ToString();
                        jsonSaveableEntity.SetUniqueIdentifier(newUniqueID);
                    }
                }
            }

  1. Block the Capture and Restore (I don’t know how to block a Capture, returning a null messes up the game drastically) for the RespawnManager from working for NPCs that are on a boat, and call that in the boat respawn manager as well. The problem is, I have absolutely no idea how to do that efficiently

For step 2, here’s my current attempt with that so far:

// In 'CaptureAsJToken():

                // TEST 23/11/2024
                foreach (IJsonSaveable JSONSaveable in spawnedBoat.GetComponentsInChildren<IJsonSaveable>()) 
                {
                    JToken token = JSONSaveable.CaptureAsJToken();
                    string component = JSONSaveable.GetType().ToString();
                    stateDict[component] = token;
                    Debug.Log($"Component Saved: {component}, Token: {token}");
                }

// in 'LoadEnemyState(called in 'RestoreFromJToken())':

            // TEST 23/11/2024
            foreach (IJsonSaveable jsonSaveable in spawnedBoat.GetComponentsInChildren<IJsonSaveable>()) 
            {
                string component = jsonSaveable.GetType().ToString();
                if (stateDict.ContainsKey(component)) 
                {
                    Debug.Log($"Restoring component: {component}, Token: {stateDict[component]}");
                    jsonSaveable.RestoreFromJToken(stateDict[component]);
                    Debug.Log($"{spawnedBoat.gameObject.name} has been restored");
                }
                else 
                {
                    Debug.Log($"Component not found in stateDict: {component}");
                }
            }

I also tried messing with the script execution order, where I try to get the Enemy’s RespawnManager to work before the Boat’s Respawn Manager (as a way to ensure that the restoring for the boat has the higher hand in the end, which was theoretically supposed to ensure that the boat allies don’t go crazy), but that failed too


Edit 2: Why do I have a funny feeling that this line in ‘BoatRespawnManager.cs’:

                    spawnedEnemyRespawnManager.BoatAllyRespawn();

Which spawns enemies, just like this line in ‘RespawnManager.cs’:

                Respawn();

Overwrites the ‘Respawn()’ and hence it causes the nightmare I’m dealing with? (Problem is, testing this theory is very difficult because the fix code is insanely complicated…)

I’m so lost on your boat ally thing right now, it’s really run away from you (sailed away)…

Assigning new UniqueIDs at runtime is rubbish, utterly meaningless as they won’t match up when you start the game again. The UniqueIDs need to be managed on the GameObject that is the RespawnManager. In other words, the RespawnManager’s Gameobject should get the SaveableEntity component, and that’s the UniqueID that will be used.

Except that we’re pulling the responsibility from the SaveableEntity class into the SaveableEntityEditor class, and it’s got a better way of handling this than Sam’s original code. Hint: the Update() code in SaveableEntity is the reason why the very act of opening a scene causes it to be “dirty”. Ever notice if you just open the scene and then close it, it asks you if you want to save?

OnEnable in an Editor class occurs when you click on the GameObject with the class in question. This is the standard way of hooking up the properties and setting things up in Editor code.

This is your problem. Only the RespawnManager should be spawning enemies. If the enemy is on a boat, then the RespawnManager should determine that the enemy is on a boat and save that information so it can put the enemy back on the boat.

It’s so complicated (and you’ve left so much clutter in the comments in most of your scripts because you won’t use source control) that it’s gotten into nightmare territory. It’s not the first time I’ve said you’re in a bit too deep on the boat thing. It’s not even something I would attempt, especially with other outstanding issues on your plate.

can’t blame you one bit

I deleted that UniqueID at runtime thing entirely

OK Maybe it’s because I just woke up, but I am dead confused about what’s going on here :sweat_smile:

Right now the RespawnManager has two variants of the Respawn function, mainly because the second one has the sole purpose of ensuring boat allies don’t respawn on the boat unless the boat itself respawns, otherwise that’s another game-breaking bug. The only problem is, the Boat Ally Respawn function is hooked to the Boat Respawn function, and I can’t think of anyway of turning that function off if the player just came back from a game restore… Even I am confused as of what’s going on right now :sweat_smile: (this is a terrible position to be in)

This is the very very last step of this system. I had plans to expand it a tad bit, but considering the nightmare I’m dealing with, I want to shut that system down as quickly as possible now

trust me, its doable, but its a nightmare in a ton of ways

(Just the fact that respawning NPCs on the boat means Respawning their AggroGroups, so their AggroGroups and their RespawnManagers can’t be saved on that one if I’m not mistaken, is what makes this incredibly hard)

For now I am just stuck with a nightmare that I have absolutely no clue how to fix…

Edit: Solved my nightmare in the most complicated way possible, lol. Learned a few things along the way as well

Privacy & Terms