Enemies moving around Obstacles

[SOLVED, Check the last comment below]

OK so… Now that I’m done with my personal Dodging State, I ran into a brand new (OK kinda old, because I asked about this when I was still using a Point-and-click system, but the problem got nullified with the new Third-Person System) problem.

My problem, this time, is that my enemies can’t move around Obstacles. Before I came to ask about it, I went to ‘Mover.cs’ (and ‘Fighter.cs’), copied a little bit of the work in there, and tried tweaking and working on something new to get it to work, BUT… It failed. Here’s what I tried doing in ‘EnemyChasingState.cs’ to try and combat this problem (everything new is labelled with ‘TEST - 27/5/2024’ for convenience):


```using UnityEngine;
using UnityEngine.AI;

namespace RPG.States.Enemies
{
    public class EnemyChasingState : EnemyBaseState
    {
        public EnemyChasingState(EnemyStateMachine stateMachine) : base(stateMachine) {}

        private float attackingRange;
        public readonly int TargetingForwardSpeedHash = Animator.StringToHash("TargetingForwardSpeed");

        public override void Enter()
        {
            Debug.Log($"{stateMachine.gameObject.name} has entered chasing state");
            stateMachine.Animator.CrossFadeInFixedTime(FreeLookBlendTreeHash, stateMachine.CrossFadeDuration);
            attackingRange = stateMachine.Fighter.GetAttackingRange();
            attackingRange *= attackingRange;
        }

        public override void Tick(float deltaTime)
        {

        if (!IsInChaseRange() && !IsAggrevated())
        {
            stateMachine.TriggerOnPlayerOutOfChaseRange();
            stateMachine.SwitchState(new EnemyIdleState(stateMachine));
            return;
        }

        Vector3 lastPosition = stateMachine.transform.position;

        if (IsInAttackingRange())
        {
            if (!stateMachine.CooldownTokenManager.HasCooldown("Attack"))
            {
                // you're not on Attack Cooldown, so you're quite aggressive, so switch to attacking state
                Debug.Log($"{stateMachine.gameObject.name} is exiting chase state to attack state");
                stateMachine.SwitchState(new EnemyAttackingState(stateMachine));
                return;
            }

            else
            {
                Move(deltaTime);
                stateMachine.Animator.SetFloat(FreeLookSpeedHash, 0);
                // TEST (SUCCESS): get the enemies to point at each other, instead of the player, if they're fighting one another:
                if (stateMachine.LastAttacker == null) stateMachine.SwitchState(new EnemyIdleState(stateMachine)); // ensures the enemy knows what to do if the enemy is dead (if this isn't here, NREs will be unleashed like bombs...!)
                else FaceTarget(stateMachine.LastAttacker.transform.position, deltaTime);

                // ORIGINAL:
                // FaceTarget(stateMachine.Player.transform.position, deltaTime);                    
                return;
            }
        }

            // else MoveToTarget(deltaTime); (Replaced with Temporary 'TEST - 27/5/2024' Below):

            // TEST - 27/5/2024 (THE ENTIRE 'else' STATEMENT BELOW IS A TEST):
            else 
            {
                Vector3 targetPosition = stateMachine.LastAttacker != null ? stateMachine.LastAttacker.transform.position : stateMachine.Player.transform.position;
                if (!HasLineOfSight(targetPosition)) 
                {
                    MoveAroundObstacles(targetPosition, deltaTime);
                }
                else MoveToTarget(deltaTime);
            }

            Vector3 deltaMovement = lastPosition - stateMachine.transform.position;
            float deltaMagnitude = deltaMovement.magnitude;

            if (deltaMagnitude > 0) 
            {
                // if the game is not paused:
                FaceTarget(stateMachine.transform.position - deltaMovement, deltaTime);
                float grossSpeed = deltaMagnitude / deltaTime;
                stateMachine.Animator.SetFloat(FreeLookSpeedHash, grossSpeed / stateMachine.MovementSpeed, stateMachine.AnimatorDampTime, deltaTime);
            }

            else 
            {
                // if the game is paused:
                FaceTarget(stateMachine.Player.transform.position, deltaTime);
                stateMachine.Animator.SetFloat(FreeLookSpeedHash, 0f);
            }
        }

        public override void Exit()
        {
            stateMachine.Agent.ResetPath();
            stateMachine.Agent.velocity = Vector3.zero;
        }

        private bool IsInAttackingRange()
        {
            if (stateMachine.LastAttacker == null)
            {
                return Vector3.SqrMagnitude(stateMachine.Player.transform.position - stateMachine.transform.position) <= attackingRange;
            }

            else return Vector3.SqrMagnitude(stateMachine.LastAttacker.transform.position - stateMachine.transform.position) <= attackingRange;
        }

        public void MoveToTarget(float deltaTime)
        {
            if (!stateMachine.Agent.enabled) stateMachine.Agent.enabled = true; // turn on the NavMeshAgent, otherwise the enemy won't be able to chase you down...
            
            // stateMachine.Agent.destination = stateMachine.Player.transform.position;
            // TEST: Instead of just hunting the player down everytime this enemy is attacked, hunt down the LastAttacker (it can be the player or another NPC...). 
            // If you found none, and this function is called, hunt the player down (FIX THIS, BECAUSE IT SOMETIMES GETS NPCs TO ATTACK THE PLAYER FOR NO REASON!):
            stateMachine.Agent.destination = stateMachine.LastAttacker != null ? stateMachine.LastAttacker.transform.position : stateMachine.Player.transform.position;

            Vector3 desiredVelocity = stateMachine.Agent.desiredVelocity.normalized; // Normalized Desired Speed of the NPC
            Move(desiredVelocity * stateMachine.MovementSpeed, deltaTime); // Go to the player, at that desired speed
            stateMachine.Agent.velocity = stateMachine.CharacterController.velocity; // The velocity of the player
            stateMachine.Agent.nextPosition = stateMachine.transform.position; // The next position to aim for
        }

        // TEST - 27/5/2024 (ALL FUNCTIONS BELOW ARE TEMPORARY TESTS, AND MAY OR MAY NOT STAY IN THE END):
        private bool HasLineOfSight(Vector3 targetPosition) 
        {
            Vector3 directionToTarget = targetPosition - stateMachine.transform.position;
            return !Physics.Raycast(stateMachine.transform.position, directionToTarget.normalized, LayerMask.GetMask("Obstacle"));
        }

        private void MoveAroundObstacles(Vector3 targetPosition, float deltaTime) 
        {
            while (!HasLineOfSight(targetPosition) && CanMoveTo(targetPosition)) 
            {
                MoveToTarget(deltaTime);
                return;
            }
            stateMachine.SwitchState(new EnemyIdleState(stateMachine));
        }

        private bool CanMoveTo(Vector3 destination) 
        {
            NavMeshPath path = new NavMeshPath();
            bool hasPath = NavMesh.CalculatePath(stateMachine.transform.position, destination, NavMesh.AllAreas, path);
            if (!hasPath) return false;
            if (path.status != NavMeshPathStatus.PathComplete) return false;
            return true;
        }

    }
}

SO… Like I said, the goal is that if there’s an obstacle between the player and the enemy, and there’s a path (preferably under 100 meters) that can be walked by the enemy to get to the player, then he’ll go ahead and take that path. If not, and you’re a melee warrior, just give up already… (if the enemy is a ranger or a wizard, based on the skill associated with the weapon, you can fight from a distance if the player is within your chase range, and you’re hostile (either by default nature, or because the player attacked you), but out of your physical range… I recall we covered that at the end of the first Core Combat Creator course back in the day, but I want to clean it up for the Third-Person version as well)

It’s really built into the navmesh. I just did this same movement for my enemy and it walks around the obstacle to get to the player. No weird MoveAroundObstacle method. Just tell it to go here, and it follows the navmesh to get there, avoiding obstacles as it goes.

protected void MoveTo(Vector3 position, float speed, float deltaTime)
{
    if (_stateMachine.Agent.isOnNavMesh)
    {
        _stateMachine.Agent.destination = position;
        Move(_stateMachine.Agent.desiredVelocity * speed, deltaTime);
    }
    _stateMachine.Agent.nextPosition = _stateMachine.transform.position;
}
protected void Move(Vector3 motion, float deltaTime)
{
    var movement = motion + _stateMachine.ForceReceiver.GetMovement();
    _stateMachine.Controller.Move(movement * deltaTime);
}

public override Tick(float deltaTime)
{
    var playerPosition = PlayerStateMachine.GetPosition();
    MoveTo(playerPosition, _stateMachine.RunSpeed, deltaTime);
}

OK I may not be fully up to date, but… (I’m assuming I’m expected to copy-paste this script, or is this just a demo?)

  1. What does your ‘GetMovement()’ function in ‘ForceReceiver.cs’ contain? My script is unable to identify it

  2. Is your tick method just 2 lines, or did you only keep what’s necessary for the question?

Apologies, I’m a little confused here

Yeah, as usual I’m not going to make your game for you. This is what I have and all that is needed to make the enemies avoid obstacles.

See the 3rd Person Combat & Traversal Course. It’s here. I used a getter method, while the code in the repo is a property. Same thing

My Tick is a couple of lines more because it checks conditions to decide if it should change state. This is only what’s needed to get the enemy to move

OK one last question, just to be clear (I’ll go through this again in the morning), your functions are in ‘EnemyChasingState.cs’?

Tick is in my EnemyChasingState. The others are in the EnemyBaseState because my enemy also roams around and has targets that are not necessarily the player. So, other states can also use that

Got it, thanks @bixarrio - I’ll give it another attempt in the morning and see if I can reach something this time

For tonight, I’m just glad I pulled off the Dodging state :smiley: (I’m trying to fix a bit of a rare bug that’s been annoying me for a while right now)

Well… I have no idea how to do this, especially with the new NavMesh that I’m using (I’m still trying as we speak, but because I haven’t touched this topic in a while I genuinely am confused)

If anything, I accidentally got my enemies to run to chase the player down by accident, and it was kinda funny (sorry, I got a little side-tracked)

But… I have no idea how to get obstacle avoidance to work here :sweat_smile:

I also have to mention that, although the new NavMesh on the Terrain has “Obstacles” unticked so as to avoid them, and the fence is labelled as “Static” and the LayerMask is correctly set to “Obstacles”, I still can’t get the code to work:

In the image, the NavMesh is meant to stop around the Obstacle, but it’s not (P.S: This is the NEW NavMesh Agent system)

You don’t have to. It works automagically.

Your fence is not an obstacle so the enemies won’t try to avoid it. The table over there is an obstacle (you can see how it has carved holes in the navmesh). Enemies will try to avoid it (although in this case, they may try to walk through the middle of it) For cases like the table, you’d want to create a box collider on the table that will serve as the obstacle so that the navmesh bake will cut the middle bits out, too. NavMeshAgents will walk on anything that’s blue. If your obstacle doesn’t cut the navmesh, they will not try to avoid it. You’ve also set the navmesh to think your agents are half-a-meter tall…

That’s… already in there, and the big surprise that I just noticed is that although it’s Labelled as “Static” BUT the LayerMask is NOT Labelled “Obstacles” (in fact, it’s currently “static”), somehow it carved into the NavMeshAgent

I changed that, still no luck (i.e: Set it to “Static”, gave them a “Nav Mesh Obstacle” component of type “Box”, and the LayerMask is set to “Obstacles”)

And for ranged attacks, this’ll just get worse, because they don’t know how to avoid attacking something with an obstacle around it

I will reverse this to 1.6 meters or so. I did that back then because the walking distance seemed to be the most accessible at 0.5 meters, and I thought I can get away with that :sweat_smile:

ANYWAY, I accidentally found a ‘Carve’ option in ‘NavMeshObstacle’, and I’m about to bombard the entire game obstacles that need to be dealt with , with accurate carves (don’t ask me how, but somehow, the houses and other obstacles without a ‘Nav Mesh Obstacle’ have no problems avoiding them. I will eventually switch out the old system entirely for the new one)… Let’s hope it works (I removed the Box Collider on my Crafting Table and replaced it with a ‘Nav Mesh obstacle’ and it seems to be doing a good job so far)

It’s always these nice conversations that flicker something in the right direction… Thank you

But I still need help coding it :sweat_smile:

I was just going to tell you that. I don’t think the new nav system cares about the static thing anymore. Not sure, though. I haven’t marked any obstacles or static and my navmesh is cutting out the obstacles. But I also haven’t mangled the navmesh data beyond recognition. I added a NavMeshSurface and pressed the bake button

There’s no coding. I said this 3 times now. The agents avoid obstacles. That’s the whole point of the navmesh and agents.

This has nothing to do with navigation. This is something you have to build yourself.

OK so… I got them to start walking around and finding directions around problems, for now. I’m still testing a few things out, BUT…

For this one, I’m thinking of Raycasts, but isn’t that a little too computationally expensive? I honestly don’t see a way around having that run in an Update function, BUT having countless enemies do that kind of operation sounds like a computational performance nightmare. Do you have any better ideas?

That’s why you optimise. For example, enemies don’t have to do that if they’re not targeting anyone. They also don’t have to do that all the time.

ahh, events… Again :sweat_smile: (I’ll see what I can come up with)

Ahh… OK I am struggling to develop this one. This was my idea:

In ‘EnemyIdleState.cs’, how about we get a reference from ‘EnemyStateMachine.cs’ for fighter (which exists as ‘stateMachine.Fighter’) and from there, we can get a reference to the ‘currentWeaponConfig’ variable (i.e: What weapon is the enemy holding), and get the skill on that? If the skill is a ranged or magic skill, and in ‘EnemyStateMachine’ the ‘LastAttacker’ variable is not null, then we shoot a raycast that aims for the player, and if there’s an obstacle, then we get the position of the player and find our way around it?

but… let’s wait and see if Brian has any better ideas (I don’t know of anyway that the NavMesh can automatically figure this one out tbh) regarding that one when he’s back. For now I’m still trying to think of a way to actually implement this, assuming that it’s actually a good idea to implement (if anyone reading this has a better idea, please share it)

Bahaa,
I think I see something here for you to check. Concider these things together:

Screenshot 2024-05-27 091949

Are you using both navmesh systems at the same time? This might be why enemies don’t move around obstacles automatically.

nope, I eliminated/cleared the old one (otherwise my swimming system ground limit wouldn’t work for my AI, xD). The reason they won’t move around obstacles, was because they didn’t have a NavMesh Obstacle component, which I eventually added

and I put the box collider on the crafting table back, otherwise it won’t work

ANYWAY… Here’s what I tried coming up with, mixing a little bit of ‘Mover.cs’ and ‘Fighter.cs’ into ‘EnemyChasingState.cs’, but that still failed. First, I tried introducing a LayerMask and a maxNavPathLength into ‘EnemyStateMachine.cs’, as follows:

    [field: Tooltip("Which LayerMask will this enemy use to avoid whilst hunting down his LastAttacker, in 'EnemyChasingState.cs'?")]
    [field: SerializeField] public LayerMask LayerMask {get; private set;} = LayerMask.GetMask("Obstacles");
    [field: SerializeField] public float maxNavPathLength {get; private set;} = 100.0f;

Next, I tried introducing a checker function, for the type of skill in the NPC’s hand (the Debugger works), in ‘EnemyChasingState.Tick().IsInAttackingRange()’:

            // (TEST - 27/5/2024): Solution to ensure the enemy can move around Obstacles:
            if (stateMachine.LastAttacker != null)
            {
                if (stateMachine.Fighter.GetCurrentWeaponConfig().GetSkill() == Skill.Ranged || stateMachine.Fighter.GetCurrentWeaponConfig().GetSkill() == Skill.Magic)
                {
                    Debug.Log($"{stateMachine.gameObject.name} is wielding a ranged weapon, and LastAttacker is {stateMachine.LastAttacker.gameObject.name}");
                    if (!HasLineOfSight()) 
                    {
                        while (!HasLineOfSight() && CanMoveTo(stateMachine.LastAttacker.transform.position)) 
                        {
                            MoveToTarget(deltaTime);
                            return;
                        }
                    }
                }
            }

Then, I tried copy-pasting the function from ‘Mover.cs’ to ‘EnemyChasingState.cs’, as follows:

        bool HasLineOfSight() 
        {
            Ray ray = new Ray(stateMachine.transform.position + Vector3.up, stateMachine.LastAttacker.transform.position - stateMachine.transform.position);
            if (Physics.Raycast(ray, out RaycastHit hit, Vector3.Distance(stateMachine.transform.position, stateMachine.LastAttacker.transform.position - stateMachine.transform.position), stateMachine.LayerMask))
            {
                return false;
            }
            return true;
        }

        public bool CanMoveTo(Vector3 destination) 
        {
            var path = new NavMeshPath();
            var hasPath = NavMesh.CalculatePath(stateMachine.transform.position, destination, NavMesh.AllAreas, path);
            if (!hasPath) return false;
            if (path.status != NavMeshPathStatus.PathComplete) return false;
            if (GetPathLength(path) > stateMachine.maxNavPathLength) return false;
            return true;
        }

        public float GetPathLength(NavMeshPath path) 
        {
            float total = 0;
            if (path.corners.Length < 2) return total;

            for (int i = 0; i < path.corners.Length - 1; i++) 
            {
                total += Vector3.Distance(path.corners[i], path.corners[i + 1]);
            }
            return total;
        }

but that failed, and I’m still trying to find out a way to make it work… In the meanwhile, anyone knows of any better approach?

AND… AS IT TURNED OUT, This does work. I just didn’t set the LayerMask and ‘maxNavPathLength’ variables properly in the test enemy’s Inspector

So, for Melee weapons, we are essentially assigning NavMeshObstacle components to the obstacles in the scene, and they’ll automatically avoid those obstacles and move around them.

As for Ranged and Magic warriors (and this is an extremely dedicated solution to my project, as it uses my own Skilling System. Unless you followed the tutorial where @Brian_Trotter helped me out with that, it won’t make sense for ya), we first check if the enemy is holding a ranged or magic weapon, and then we are using the code I wrote above to get the line of sight of the enemy, and then using that (assuming the player is out of sight), we are getting the enemy to walk around the obstacles to find the player, and the moment he sees us, he attacks us.

Lovely stuff :smiley:

OK umm… Last question. The math works, and the system does what I need it to do for my needs, but… what should I set my ‘Animator.SetFloat()’ to be to get the animations to act properly…?! (He’s currently just floating his way to the player on the ground)

This is my current code where I want to throw the Animator.SetFloat() into…:

        if (IsInAttackingRange())
        {
            // (TEST - 27/5/2024): Solution to ensure the enemy can move around Obstacles, if his enemy (Player/another NPC) is behind a wall of some sort:
            if (stateMachine.LastAttacker != null)
            {
                if (stateMachine.Fighter.GetCurrentWeaponConfig().GetSkill() == Skill.Ranged || stateMachine.Fighter.GetCurrentWeaponConfig().GetSkill() == Skill.Magic)
                {
                    Debug.Log($"{stateMachine.gameObject.name} is wielding a ranged weapon, and LastAttacker is {stateMachine.LastAttacker.gameObject.name}");
                    if (!HasLineOfSight())
                    {
                        Debug.Log($"{stateMachine.gameObject.name} has no line of sight of {stateMachine.LastAttacker.gameObject.name}");
                        while (!HasLineOfSight() && CanMoveTo(stateMachine.LastAttacker.transform.position))
                        {
                            Debug.Log($"{stateMachine.gameObject.name} is attempting to move to {stateMachine.LastAttacker.gameObject.name}");
                            MoveToTarget(deltaTime);
                            return;
                        }
                    }
                }
            }

Privacy & Terms