EnemyPatrolState for Unity Third Person

If you’re like me, you like your enemies to do more than stand around waiting to kill you. These guards are making 3 sheckles a day to keep that campfire safe, they should patrol!

Most of the groundwork for this has already been laid out through the course, leading up to the end of the AI section, but we’ll need a couple of helper classes to help move things along, before we even get to creating a patrol state. (Actually, we’ll be creating two new states, EnemyPatrolState() and EnemyDwellState())

Patrol Points

We can't do anything without some points to patrol in the scene. With this in mind, I created a class PatrolPoint.cs. We've used Patrol Points in other courses, but there are some pitfalls that have snagged students in the RPG course, in that you can't always tell on a terrain if you've placed the patrol point on the ground or not. We'll make that automagically happen.

It’s very important that waypoints be on the ground, not sitting a few feet above or below the ground.

Why is this important, exactly?

The waypoint system is set up to move towards a waypoint until you are within a certain radius of the waypoint. This simply doesn’t work if you’re already up in the air 2 meters because that Y component will be part of the distance equation.

First, if you haven’t done so already, create a layer in your layer indexes called Ground, and assign the terrain to the Ground layer.

Now for the PatrolPoint class:

PatrolPoint.cs
using UnityEngine;


public class PatrolPoint : MonoBehaviour
{
    [SerializeField] private LayerMask floor;
   
    void SetOnFloor()
    {
        Ray ray = new Ray(transform.position + Vector3.up * 100, Vector3.down);
        if (Physics.Raycast(ray, out RaycastHit hit, 1000, floor))
        {
            transform.position = hit.point;
        }
    }

    private void OnValidate()
    {
        
        SetOnFloor();
    }

    private void OnDrawGizmos()
    {
        Gizmos.color = Color.yellow;
        Gizmos.DrawSphere(transform.position, .2f);
    }
}

First, we’re setting up a layer mask. Once you’ve created an empty GameObject with the PatrolPoint in it, you’ll be able to set that layer mask. You want it to only be things that can be walked on. In general, that’s the Ground layer.

Next is a method that moves above the current position of the PatrolPoint and uses Raycasting to locate the ground beneath it, automagically fixing the point onto the ground.

Next is OnValidate() that just lurks there until you try to move the waypoint… Wherever you move it, unless you’ve got a monster drop going on (or the wrong items on the Ground layer), the point will automagically be at ground level.

Finally, to help us visualize, a simple Gizmos draw call to put a sphere at the chosen location. That makes it easy to see at a glance where your friendly guard will be patrolling.

Patrol Path

Now that we have Patrol Points out of the way, we need a Patrol Path. Here, we need a class that will allows to easily
  • Locate the nearest patrol point
  • Locate the next patrol point once we’ve arrived.

Create an empty GameObject and add this script:

Summary
using System.Collections.Generic;
using System.Linq;
using UnityEngine;

public class PatrolPath : MonoBehaviour
{
    private readonly List<PatrolPoint> patrolPoints = new List<PatrolPoint>();
    private void Awake()
    {
        OnValidate();
    }

    private void OnValidate()
    {
        patrolPoints.Clear();
        foreach (PatrolPoint patrolPoint in GetComponentsInChildren<PatrolPoint>())
        {
            patrolPoints.Add(patrolPoint);
        }
    }

    public int GetNearestWaypointIndex(Vector3 position)
    {
        PatrolPoint closest =
            patrolPoints.OrderBy(t => (t.transform.position - position).sqrMagnitude).FirstOrDefault();
        return patrolPoints.IndexOf(closest);
    }

    public int GetNextWaypointIndex(int index)
    {
        return (index + 1) % patrolPoints.Count;
    }

    public PatrolPoint this[int i] => patrolPoints[i];
    
    private void OnDrawGizmos()
    {
        OnValidate();
        if (patrolPoints.Count < 2) return;
        for (int i = 1; i < patrolPoints.Count; i++)
        {
            Debug.DrawLine(patrolPoints[i-1].transform.position, patrolPoints[i].transform.position);    
        }
        if (patrolPoints.Count > 2)
        {
            Debug.DrawLine(patrolPoints[0].transform.position, patrolPoints[patrolPoints.Count-1].transform.position);
        }
    }
}

There’s a lot going on under the hood on this one. We’re starting with a List<PatrolPoint>, which we’re going to build using OnValidate(). Basically, every time add or remove a patrol point, OnValidate will rebuild the list of patrol points. OnDrawGizmos will draw lines between each (assuming you have at least 2.

We have three distinct functions that will be exposed to our EnemyPatrolState.

GetNearestWaypointIndex(Vector3 position) will return the index of the nearest waypoint to the character’s current position. This will help us prevent the enemy from always returning to the first waypoint whenever something takes him away from his route (like chasing a player, for example).

GetNextWaypointIndex(int index) will return the next index in line, using the % modulo operator to ensure that (as long as the index is positive), it will always wrap around to the start of the list when it gets too big.

Finally, I thought this would be a good time to show off how to build an Indexer.

public PatrolPoint this[int i] => patrolPoints[i];

This property declaration gives us read only access to a specific element in the List. It means that if you have a reference to PatrolPoint, you can get whatever patrol point you want by the index, for example:

[SerializeField] PatrolPath path;

void Start()
{
    PatrolPoint firstPoint = path[0];
}

Ok, we’ve got a PatrolPath, with PatrolPoints in it… Make sure to add a PatrolPath reference in your EnemyStateMachine and drag the PatrolPath into an enemy’s StateMachine.

[field: SerializeField] public PatrolPath  {get; private set;}

Now we need the EnemyPatrolState(). This state needs to inherit from EnemyBaseState, and we’ll actually be generating two constructors, one constructor with just the stateMachine, and another with an integer for an index.

EnemyPatrolState.cs
using UnityEngine;

    public class EnemyPatrolState : EnemyBaseState
    {
        public EnemyPatrolState(EnemyStateMachine machine) : base(machine)
        {
        }

        public EnemyPatrolState(EnemyStateMachine machine, int lastWaypoint) : base(machine)
        {
            currentWaypoint = lastWaypoint;
        }
        
        private int currentWaypoint = -1;
        private Vector3 currentTarget;
        
        public override void Enter()
        {
            stateMachine.Animator.CrossFadeInFixedTime(LocomotionHash, .25f);
            if(currentWaypoint<0)
            {
                currentWaypoint = stateMachine.PatrolPath.GetNearestWaypointIndex(stateMachine.transform.position);
            }
            else
            {
                currentWaypoint = stateMachine.PatrolPath.GetNextWaypointIndex(currentWaypoint);
            }
            currentTarget = stateMachine.PatrolPath[currentWaypoint].transform.position;
            Debug.Log($"{stateMachine.name} Current Waypoint: {currentWaypoint} - {currentTarget}");
            stateMachine.NavMeshAgent.destination = currentTarget;
            
        }

        public override void Tick(float deltaTime)
        {
            if (CanSeePlayer())
            {
                Debug.Log($"{stateMachine.name} can see player, switching to chase state.");
                stateMachine.SwitchState(new EnemyChaseState(stateMachine));
                return;
            }
            Debug.Log($"{stateMachine.name} PatrolState.Tick() Distance To Waypoint = {Vector3.Distance(stateMachine.transform.position, currentTarget)}");
            stateMachine.Animator.SetFloat(Speed, 1f, AnimatorDampTime, deltaTime);
            if (Vector3.Distance(stateMachine.transform.position, currentTarget) < 2)
            {
                stateMachine.SwitchState(new EnemyDwellState(stateMachine, currentWaypoint));
            }
            MoveToWaypoint(deltaTime);

        }

        public override void Exit()
        {
        }
        
        void MoveToWaypoint(float deltaTime)
        {
            if (!stateMachine.NavMeshAgent.enabled) stateMachine.NavMeshAgent.enabled = true;
            Vector3 velocity = stateMachine.NavMeshAgent.desiredVelocity.normalized * stateMachine.MovementSpeed;
            stateMachine.transform.forward = stateMachine.NavMeshAgent.desiredVelocity;
            Move(velocity, deltaTime);
            stateMachine.NavMeshAgent.nextPosition = stateMachine.transform.position;
        }
    }

Much of this is similar to what we doin in ChaseState… In Tick, we’re checking to see if we’re in chase range of the player, like we do in the Idle state. Next, we check to see if we’re in range of the waypoint. If we are, then we call a new DwellState, which will be below. Other than that, like EnemyChaseState, we’re moving to the goal, only the goal in this case is the waypoint location, not the Player.

What I did was put a check in the EnemyIdleState, if the character has a PatrolPath, it immediately switches to a new EnemyPatrolState(stateMachine); Otherwise, Idle carries on like normal.

Finally, we have the EnemyDwellState… so that at each waypoint, the character will stop. This would be a great place to put an animation for looking around, etc… Alternatively,

EnemyDwellState.cs
using UnityEngine;


    public class EnemyDwellState : EnemyBaseState
    {
        public EnemyDwellState(EnemyStateMachine machine) : base(machine)
        {
        }

        public EnemyDwellState(EnemyStateMachine machine, int currentWaypoint) : base(machine)
        {
            this.currentWaypoint = currentWaypoint;
        }

        private static int InvestigateHash = Animator.StringToHash("Investigate");
        private float dwellDuration = 5;
        private int currentWaypoint;
        
        public override void Enter()
        {
            stateMachine.Animator.CrossFadeInFixedTime(InvestigateHash, .25f);
        }

        public override void Tick(float deltaTime)
        {
            if (CanSeePlayer())
            {
                stateMachine.SwitchState(new EnemyChaseState(stateMachine));
            }
            dwellDuration -= deltaTime;
            if (dwellDuration < 0)
            {
                stateMachine.SwitchState(new EnemyPatrolState(stateMachine, currentWaypoint));
                return;
            }
        }

        public override void Exit()
        {
            
        }
    }

You’ll note this looks very much like the Idle State, except that we have a countdown timer. You’ll need to put an animation into your animator named Investigation. Maybe something where the character is looking around.

Note that we have a special constructor for this class (and PatrolState) with an int for the waypoint… Here’s how this works:

When you enter IdleState(), and you determine you have a PatrolPath assigned, then you SwitchState(new EnemyPatrolState(stateMachine);, but when the enemy reaches a destination and dwells, the Patrol State calls SwitchState(new EnemyDwellState(stateMachine, currentWaypoint)). The whole purpose of passing the waypoint is so that we can pass it right back when the time is up and we call a new EnemyPatrolState. That’s the purpose of the 2nd constructor.

If we use the first constructor, from Idle, then currentWaypoint is left at -1, which triggers the Enter state to ask the Patrol Path for the nearest patrol point to it’s current location. This will occur at the beginning of the game and after any chase/attack states that the character may be in. If, however, the second constructor is used, then the Enter gets the NEXT waypoint and sets it as the current waypoint and begins it’s travels. Left alone with no encounters with a player, the enemy will cycle back and forth between the two states, continuing his loop around the area.

Possible improvements

Right now, the PatrolPoint class serves just the purpose of ensuring that the waypoint will always be on the ground instead of several feet off the ground. It could be expanded... you could have a number of differing animations, and then have a string stored in each PatrolPoint with the name of the animation state that should run, and perhaps even a duration.

You could create reverse logic… so that the enemy could go to one end of the patrol route, and then walk back to the beginning.

You might even create other actions that can be performed, perhaps referenced by an Enum on the PatrolPoint. A wolf, for example, might stop at one waypoint and howl, then move on to another waypoint and attack the nearest rabbit.

The possibilities are endless. Thanks for reading this far!

4 Likes

Excellent work, as usual, @Brian_Trotter. Thanks for taking the time to post this in-depth guide and framework.

This is great! @Brian_Trotter I added it to my project and really enjoying it. One thing I noticed is that the Enemy doesnt rotate smoothly when changing way point or if the Enemy needs to curve in order to get to the next waypoint, it snaps into rotation instead. Is there a way of fixing this?

Probably would be best to add the same Rotation code we use with the Player:

        //Edited, as I copied the original method from the wrong project
        protected void FaceMovementDirection( Vector3 movement, float deltaTime)
        {
            stateMachine.transform.rotation = Quaternion.Lerp(
                stateMachine.transform.rotation,
                Quaternion.LookRotation(movement),
                deltaTime * stateMachine.RotationDamping);
        }
1 Like

Thank you! This worked great when patrolling, The character is still snapping when going from dwell state to patrolling new path, also stares in a different direction than the direction the character was facing when arriving at the patrol point.

Hmmm I think the same issue may be occurring, we need to rotate the character towards the state, taking it out of the hands of the NavMeshAgent. Make sure that agent.updateRotation is set to false (usually in the StateMachine.Awake() or Start().

Checked the stateMachine and I do have updateRotation = false in Start().

I noticed another bug while playing around with this where it seems like the enemy rotates to Zero for a few seconds. I checked the console and I dont see an error but I do have a log with the following message:
Look rotation viewing vector is zero

Seems to be happening in the MoveToWaypoint function in the following line:

stateMachine.transform.forward = stateMachine.NavMeshAgent.desiredVelocity;

That would seem to indicate that the agent has reached it’s destination (or SetDestination hasn’t calculated a path yet)… perhaps a test

if(statemachine.NavMeshagent.desiredVelocity.sqrMagnitude>0)
{
    FaceMovementDirection(stateMachine.NavMeshagent.desiredVelocity, deltaTime);
}
1 Like

That works!, I did have to add it around both lines of code, the FaceMovementDirection and the
stateMachine.transform.forward = stateMachine.NavMeshAgent.desiredVelocity;
Thank you!
I am seeing that the enemy is getting stuck when leaving a patrol point for a few seconds. Have you experienced the same? Any ideas on how to debug?

No, I’m not seeing that at all if I don’t go to a dwell state. (in my current project, that dwell state could be anything from waiting idle for 10 seconds to laying down and taking a nap).

Privacy & Terms