Unit and integration testing in Unreal Engine

Unreal Courses Show
1

Wanted to share how Unreal Engine supports automated testing directly from its toolset. You don’t have to use it to do unit testing in your game, but this makes things easier. Automated testing is an important aspect of large software projects as manual “acceptance” testing doesn’t scale well as you continually add more features to your game. It continues to take longer and longer as new features are added, and it’s also difficult to cover all cases leading to bugs that pop up that are hard to reproduce. I feel games and real-time systems in general are different than web applications from a testing perspective as some features can only be play tested and automation is not possible. For other pure technical aspects though you can assert the expected behavior, especially if you decouple the visual components from pure logic.

I’ve used Cocos2dx on a few projects and for that I used the Catch2 unit testing framework for C++. Most unit testing frameworks in C++ rely on macros heavily which makes sense. Unreal has a similar framework and can even be used for integration testing visual components and used in Unreal extensively itself. I’ve only scratched the surface. Setting it up though is not well documented, and the internet seems to have a dearth of info on it, but I found a blog post that was helpful:

As an example, I’ve included a test of my GunAmmo class that is the guts of the ammo management in my Simple Shooter game from the Unreal C++ course:

#include "CoreTypes.h"
#include "Containers/UnrealString.h"
#include "Misc/AutomationTest.h"

#include "GunAmmo.h"

#if WITH_DEV_AUTOMATION_TESTS

BEGIN_DEFINE_SPEC(GunAmmoTest, "GunAmmo", EAutomationTestFlags::ApplicationContextMask | EAutomationTestFlags::ProductFilter)
END_DEFINE_SPEC(GunAmmoTest)

void GunAmmoTest::Define() {
	Describe("FireRound", [this]() {
		It("Should reduce current magazine and total ammo", [this]() {
			FGunAmmo GunAmmo(EAmmoType::Pistol, 10, 100, 10);

			TestTrue(TEXT("FireRound succeeds when non-empty magazine"), GunAmmo.FireRound());
			TestEqual(TEXT("Current magazine decrements by one"), GunAmmo.GetCurrentMagazineAmmo(), 9);
			TestEqual(TEXT("total ammo decrements by one"), GunAmmo.GetTotalAmmo(), 99);
		});


		It("Should return false when current magazine out of ammo", [this]() {
			FGunAmmo GunAmmo(EAmmoType::Pistol, 10, 100, 0);

			TestFalse(TEXT("FireRound fails when empty magazine"), GunAmmo.FireRound());
			TestEqual(TEXT("Current magazine decrements by one"), GunAmmo.GetCurrentMagazineAmmo(), 0);
			TestEqual(TEXT("total ammo decrements by one"), GunAmmo.GetTotalAmmo(), 100);
		});
	}); // Describe FireRound

	Describe("Reload", [this]() {
		It("Should remove ammo from current magazine and total when not on last partial reload", [this]() {
			FGunAmmo GunAmmo(EAmmoType::Pistol, 10, 88, 3);

			TestTrue(TEXT("Reload succeeds when have extra magazines"), GunAmmo.Reload());
			TestEqual(TEXT("Current magazine resets to full magazine"), GunAmmo.GetCurrentMagazineAmmo(), 10);
			TestEqual(TEXT("total ammo decrements by remaining ammo in ejected magazine"), GunAmmo.GetTotalAmmo(), 85);
		});

		It("Should not reduce total ammo if reloading on empty and have remaining magazines", [this]() {
			FGunAmmo GunAmmo(EAmmoType::Pistol, 10, 88, 0);

			TestTrue(TEXT("Reload succeeds when have extra magazines"), GunAmmo.Reload());
			TestEqual(TEXT("Current magazine resets to full magazine"), GunAmmo.GetCurrentMagazineAmmo(), 10);
			TestEqual(TEXT("total ammo decrements by remaining ammo in ejected magazine"), GunAmmo.GetTotalAmmo(), 88);
		});

		It("Should remove ammo from current magazine and total when last reload is a full reload", [this]() {
			FGunAmmo GunAmmo(EAmmoType::Pistol, 10, 12, 2);

			TestTrue(TEXT("Reload succeeds when have extra magazines"), GunAmmo.Reload());
			TestEqual(TEXT("Current magazine resets to full magazine"), GunAmmo.GetCurrentMagazineAmmo(), 10);
			TestEqual(TEXT("total ammo decrements by remaining ammo in ejected magazine"), GunAmmo.GetTotalAmmo(), 10);
		});

		It("Should not remove ammo from current magazine and total when last reload is a partial reload to full", [this]() {
			FGunAmmo GunAmmo(EAmmoType::Pistol, 10, 10, 2);

			TestTrue(TEXT("Reload succeeds when have extra magazines"), GunAmmo.Reload());
			TestEqual(TEXT("Current magazine resets to full magazine"), GunAmmo.GetCurrentMagazineAmmo(), 10);
			TestEqual(TEXT("total ammo does not decrement when last magazine"), GunAmmo.GetTotalAmmo(), 10);
		});

		It("Should not remove ammo from total when last reload is a full reload and no ammo in current", [this]() {
			FGunAmmo GunAmmo(EAmmoType::Pistol, 10, 10, 0);

			TestTrue(TEXT("Reload succeeds when have extra magazines"), GunAmmo.Reload());
			TestEqual(TEXT("Current magazine resets to full magazine"), GunAmmo.GetCurrentMagazineAmmo(), 10);
			TestEqual(TEXT("total ammo does not decrement when last magazine"), GunAmmo.GetTotalAmmo(), 10);
		});

		It("Should not remove ammo from total when last reload is a partial reload and no ammo in current", [this]() {
			FGunAmmo GunAmmo(EAmmoType::Pistol, 10, 9, 0);

			TestTrue(TEXT("Reload succeeds when have extra magazines"), GunAmmo.Reload());
			TestEqual(TEXT("Current magazine resets to full magazine"), GunAmmo.GetCurrentMagazineAmmo(), 9);
			TestEqual(TEXT("total ammo does not decrement when last magazine"), GunAmmo.GetTotalAmmo(), 9);
		});

		It("Should not remove ammo from current magazine and total when last reload is a partial reload to non-full", [this]() {
			FGunAmmo GunAmmo(EAmmoType::Pistol, 10, 7, 2);

			TestTrue(TEXT("Reload succeeds when have extra magazines"), GunAmmo.Reload());
			TestEqual(TEXT("Current magazine resets to remaining ammo"), GunAmmo.GetCurrentMagazineAmmo(), 7);
			TestEqual(TEXT("total ammo does not decrement when last magazine"), GunAmmo.GetTotalAmmo(), 7);
		});

		It("Should not reload when no remaining magazines", [this]() {
			FGunAmmo GunAmmo(EAmmoType::Pistol, 10, 3, 3);

			TestFalse(TEXT("Reload fails when no remaining spare ammo or magazines"), GunAmmo.Reload());
			TestEqual(TEXT("Current magazine resets to full magazine"), GunAmmo.GetCurrentMagazineAmmo(), 3);
			TestEqual(TEXT("total ammo decrements by remaining ammo in ejected magazine"), GunAmmo.GetTotalAmmo(), 3);
		});

		It("Should not reload when no remaining ammo", [this]() {
			FGunAmmo GunAmmo(EAmmoType::Pistol, 10, 0, 0);

			TestFalse(TEXT("Reload fails when no remaining ammo"), GunAmmo.Reload());
			TestEqual(TEXT("Current magazine does not change when no more ammo"), GunAmmo.GetCurrentMagazineAmmo(), 0);
			TestEqual(TEXT("total ammo does not change when no more ammo"), GunAmmo.GetTotalAmmo(), 0);
		});

		It("Should not reload when magazine is already full", [this]() {
			FGunAmmo GunAmmo(EAmmoType::Pistol, 10, 100, 10);

			TestFalse(TEXT("Reload fails when magazine is already full"), GunAmmo.Reload());
		});
	}); // Describe Reload

	Describe("IsEmpty", [this]() {
		It("Should return true when no more ammo", [this]() {
			FGunAmmo GunAmmo(EAmmoType::Pistol, 10, 0, 0);

			TestTrue(TEXT("No more ammo"), GunAmmo.IsEmpty());
		});

		It("Should return false when magazine empty but there is remaining ammo", [this]() {
			FGunAmmo GunAmmo(EAmmoType::Pistol, 10, 23, 0);

			TestFalse(TEXT("Ammo remains if reload"), GunAmmo.IsEmpty());
		});

		It("Should return false when ammo remains in current magazine", [this]() {
			FGunAmmo GunAmmo(EAmmoType::Pistol, 10, 31, 3);

			TestFalse(TEXT("Ammo remains in current magazine"), GunAmmo.IsEmpty());

		});
	}); // Describe IsEmpty

	Describe("CanReload", [this]() {
		It("Should return true when magazines remain", [this]() {
			FGunAmmo GunAmmo(EAmmoType::Pistol, 10, 34, 5);

			TestTrue(TEXT("Magazines remain"), GunAmmo.CanReload());
		});

		It("Should return true when last magazine is a partial", [this]() {
			FGunAmmo GunAmmo(EAmmoType::Pistol, 10, 6, 3);

			TestTrue(TEXT("Partial magazine remains"), GunAmmo.CanReload());
		});

		It("Should return false when on last full magazine", [this]() {
			FGunAmmo GunAmmo(EAmmoType::Pistol, 10, 10, 10);

			TestFalse(TEXT("Last full magazine"), GunAmmo.CanReload());
		});

		It("Should return false when on last partial magazine", [this]() {
			FGunAmmo GunAmmo(EAmmoType::Pistol, 10, 7, 7);

			TestFalse(TEXT("Last partial magazine"), GunAmmo.CanReload());
		});

		It("Should return false when no more ammo", [this]() {
			FGunAmmo GunAmmo(EAmmoType::Pistol, 10, 0, 0);

			TestFalse(TEXT("No ammo"), GunAmmo.CanReload());
		});

		It("Should return false when magazine full", [this]() {
			FGunAmmo GunAmmo(EAmmoType::Pistol, 10, 100, 10);

			TestFalse(TEXT("Magazine already full"), GunAmmo.CanReload());
		});
	}); // Describe CanReload

	Describe("GetReloadsRemaining", [this]() {
		It("Should calculate based on current magazines", [this]() {
			FGunAmmo GunAmmo(EAmmoType::Pistol, 10, 77, 8);
			bool bLastIsPartial;
			auto ReloadsRemaining = GunAmmo.GetReloadsRemaining(&bLastIsPartial);

			TestEqual(TEXT("Expected magazines match"),ReloadsRemaining, 7);
			TestFalse(TEXT("Not on last partial magazine"), bLastIsPartial);
		});

		It("Should consider last full magazine not a partial", [this]() {
			FGunAmmo GunAmmo(EAmmoType::Pistol, 10, 10, 0);

			bool bLastIsPartial;
			auto ReloadsRemaining = GunAmmo.GetReloadsRemaining(&bLastIsPartial);

			TestEqual(TEXT("Expected magazines match"), ReloadsRemaining, 1);
			TestFalse(TEXT("Last magazine is not a partial"), bLastIsPartial);
		});

		It("Should consider last non-full magazine a partial", [this]() {
			FGunAmmo GunAmmo(EAmmoType::Pistol, 10, 8, 0);

			bool bLastIsPartial;
			auto ReloadsRemaining = GunAmmo.GetReloadsRemaining(&bLastIsPartial);

			TestEqual(TEXT("Expected magazines match"), ReloadsRemaining, 1);
			TestTrue(TEXT("Last magazine is not a partial"), bLastIsPartial);
		});

		It("Should consider last split magazine a partial even if full", [this]() {
			FGunAmmo GunAmmo(EAmmoType::Pistol, 10, 10, 2);

			bool bLastIsPartial;
			auto ReloadsRemaining = GunAmmo.GetReloadsRemaining(&bLastIsPartial);

			TestEqual(TEXT("Expected magazines match"), ReloadsRemaining, 1);
			TestTrue(TEXT("Last magazine is a partial if it's split between two"), bLastIsPartial);
		});

		It("Should consider last partial split magazine a partial", [this]() {
			FGunAmmo GunAmmo(EAmmoType::Pistol, 10, 9, 1);

			bool bLastIsPartial;
			auto ReloadsRemaining = GunAmmo.GetReloadsRemaining(&bLastIsPartial);

			TestEqual(TEXT("Expected magazines match"), ReloadsRemaining, 1);
			TestTrue(TEXT("Last magazine is a partial"), bLastIsPartial);
		});

		It("Should consider no reloads remaining not a partial", [this]() {
			FGunAmmo GunAmmo(EAmmoType::Pistol, 10, 3, 3);

			bool bLastIsPartial;
			auto ReloadsRemaining = GunAmmo.GetReloadsRemaining(&bLastIsPartial);

			TestEqual(TEXT("Expected magazines match"), ReloadsRemaining, 0);
			TestFalse(TEXT("No reloads remaining means no partial reload"), bLastIsPartial);
		});

		It("Should consider no ammo remaining not a partial", [this]() {
			FGunAmmo GunAmmo(EAmmoType::Pistol, 10, 0, 0);

			bool bLastIsPartial;
			auto ReloadsRemaining = GunAmmo.GetReloadsRemaining(&bLastIsPartial);

			TestEqual(TEXT("Expected magazines match"), ReloadsRemaining, 0);
			TestFalse(TEXT("No ammo remaining means no partial reload"), bLastIsPartial);
		});

		It("Should Ignore output parameter if null and magazines remain", [this]() {
			FGunAmmo GunAmmo(EAmmoType::Pistol, 10, 77, 8);

			TestEqual(TEXT("Expected magazines match"), GunAmmo.GetReloadsRemaining(), 7);
		});

		It("Should Ignore output parameter if null and last is partial", [this]() {
			FGunAmmo GunAmmo(EAmmoType::Pistol, 10, 7, 2);

			TestEqual(TEXT("Expected magazines match"), GunAmmo.GetReloadsRemaining(), 1);
		});

		It("Should Ignore output parameter if null and no reloads remain", [this]() {
			FGunAmmo GunAmmo(EAmmoType::Pistol, 10, 3, 3);

			TestEqual(TEXT("Expected magazines match"), GunAmmo.GetReloadsRemaining(), 0);
		});

		It("Should Ignore output parameter if null and out of ammo", [this]() {
			FGunAmmo GunAmmo(EAmmoType::Pistol, 10, 0, 0);

			TestEqual(TEXT("Expected magazines match"), GunAmmo.GetReloadsRemaining(), 0);
		});
	}); // Describe GetReloadsRemaining

	Describe("AddAmmo", [this]() {
		It("Should increase total ammo if greater than zero", [this]() {
			FGunAmmo GunAmmo(EAmmoType::Pistol, 10, 100, 5);

			TestTrue(TEXT("Ammo added"), GunAmmo.AddAmmo(11));

			TestEqual(TEXT("Current magazine unaffected"), GunAmmo.GetCurrentMagazineAmmo(), 5);
			TestEqual(TEXT("total ammo increases by amount"), GunAmmo.GetTotalAmmo(), 111);
		});

		It("Should be a no-op if amount is zero", [this]() {
			FGunAmmo GunAmmo(EAmmoType::Pistol, 10, 100, 5);

			TestFalse(TEXT("Ammo not added"), GunAmmo.AddAmmo(0));

			TestEqual(TEXT("Current magazine unaffected"), GunAmmo.GetCurrentMagazineAmmo(), 5);
			TestEqual(TEXT("total ammo unaffected"), GunAmmo.GetTotalAmmo(), 100);
		});

		It("Should be a no-op if amount is less than zero", [this]() {
			FGunAmmo GunAmmo(EAmmoType::Pistol, 10, 100, 5);

			TestFalse(TEXT("Ammo not added"), GunAmmo.AddAmmo(-1));

			TestEqual(TEXT("Current magazine unaffected"), GunAmmo.GetCurrentMagazineAmmo(), 5);
			TestEqual(TEXT("total ammo unaffected"), GunAmmo.GetTotalAmmo(), 100);
		});

		It("Should clamp to max ammo when partially exceeding", [this]() {
			FGunAmmo GunAmmo(EAmmoType::Pistol, 10, 100, 5, 150);

			TestTrue(TEXT("Ammo added"), GunAmmo.AddAmmo(100));

			TestEqual(TEXT("Current magazine unaffected"), GunAmmo.GetCurrentMagazineAmmo(), 5);
			TestEqual(TEXT("total ammo increases up to max ammount"), GunAmmo.GetTotalAmmo(), 150);
		});

		It("Should be a no-op if already at maximum ammo", [this]() {
			FGunAmmo GunAmmo(EAmmoType::Pistol, 10, 100, 5, 100);

			TestFalse(TEXT("Ammo not added"), GunAmmo.AddAmmo(40));

			TestEqual(TEXT("Current magazine unaffected"), GunAmmo.GetCurrentMagazineAmmo(), 5);
			TestEqual(TEXT("total ammo unaffected"), GunAmmo.GetTotalAmmo(), 100);
		});
	}); // Describe AddAmmo

	Describe("Add", [this]() {
		It("Should increase total ammo if matching type and greater than zero", [this]() {
			FGunAmmo GunAmmo(EAmmoType::Pistol, 10, 100, 5);

			const FGunAmmo AmmoToAdd(EAmmoType::Pistol, 11);
			TestTrue(TEXT("Ammo added"), GunAmmo.Add(AmmoToAdd));

			TestEqual(TEXT("Current magazine unaffected"), GunAmmo.GetCurrentMagazineAmmo(), 5);
			TestEqual(TEXT("total ammo increases by amount"), GunAmmo.GetTotalAmmo(), 111);
		});

		It("Should not increase total ammo if not matching type and greater than zero", [this]() {
			FGunAmmo GunAmmo(EAmmoType::Pistol, 10, 100, 5);

			const FGunAmmo AmmoToAdd(EAmmoType::BoltActionRifle, 11);
			TestFalse(TEXT("Ammo not added"), GunAmmo.Add(AmmoToAdd));

			TestEqual(TEXT("Current magazine unaffected"), GunAmmo.GetCurrentMagazineAmmo(), 5);
			TestEqual(TEXT("total ammo not changed"), GunAmmo.GetTotalAmmo(), 100);
		});

		It("Should be a no-op if amount is zero", [this]() {
			FGunAmmo GunAmmo(EAmmoType::Pistol, 10, 100, 5);

			const FGunAmmo AmmoToAdd(EAmmoType::Pistol, 0);
			TestFalse(TEXT("Ammo not added"), GunAmmo.Add(AmmoToAdd));

			TestEqual(TEXT("Current magazine unaffected"), GunAmmo.GetCurrentMagazineAmmo(), 5);
			TestEqual(TEXT("total ammo unaffected"), GunAmmo.GetTotalAmmo(), 100);
		});

		It("Should be a no-op if amount is less than zero", [this]() {
			FGunAmmo GunAmmo(EAmmoType::Pistol, 10, 100, 5);

			const FGunAmmo AmmoToAdd(EAmmoType::Pistol, -1);
			TestFalse(TEXT("Ammo not added"), GunAmmo.Add(AmmoToAdd));

			TestEqual(TEXT("Current magazine unaffected"), GunAmmo.GetCurrentMagazineAmmo(), 5);
			TestEqual(TEXT("total ammo unaffected"), GunAmmo.GetTotalAmmo(), 100);
		});

		It("Should clamp to max ammo if matching type and when partially exceeding", [this]() {
			FGunAmmo GunAmmo(EAmmoType::Pistol, 10, 100, 5, 150);

			const FGunAmmo AmmoToAdd(EAmmoType::Pistol, 100);
			TestTrue(TEXT("Ammo added"), GunAmmo.Add(AmmoToAdd));

			TestEqual(TEXT("Current magazine unaffected"), GunAmmo.GetCurrentMagazineAmmo(), 5);
			TestEqual(TEXT("total ammo increases up to max ammount"), GunAmmo.GetTotalAmmo(), 150);
		});

		It("Should be a no-op if non-matching type and when partially exceeding", [this]() {
			FGunAmmo GunAmmo(EAmmoType::Pistol, 10, 100, 5, 150);

			const FGunAmmo AmmoToAdd(EAmmoType::BoltActionRifle, 100);
			TestFalse(TEXT("Ammo added"), GunAmmo.Add(AmmoToAdd));

			TestEqual(TEXT("Current magazine unaffected"), GunAmmo.GetCurrentMagazineAmmo(), 5);
			TestEqual(TEXT("total ammo should not increase"), GunAmmo.GetTotalAmmo(), 100);
		});

		It("Should be a no-op if already at maximum ammo", [this]() {
			FGunAmmo GunAmmo(EAmmoType::Pistol, 10, 100, 5, 100);

			const FGunAmmo AmmoToAdd(EAmmoType::Pistol, 40);
			TestFalse(TEXT("Ammo not added"), GunAmmo.Add(AmmoToAdd));

			TestEqual(TEXT("Current magazine unaffected"), GunAmmo.GetCurrentMagazineAmmo(), 5);
			TestEqual(TEXT("total ammo unaffected"), GunAmmo.GetTotalAmmo(), 100);
		});
	}); // Describe Add

	Describe("IsCurrentMagazineEmpty", [this]() {
		It("Should return true when no more ammo", [this]() {
			FGunAmmo GunAmmo(EAmmoType::Pistol, 10, 0, 0);

			TestTrue(TEXT("No more ammo"), GunAmmo.IsCurrentMagazineEmpty());
		});

		It("Should return true when magazine empty but there is remaining ammo", [this]() {
			FGunAmmo GunAmmo(EAmmoType::Pistol, 10, 23, 0);

			TestTrue(TEXT("Ammo remains if reload"), GunAmmo.IsCurrentMagazineEmpty());
		});

		It("Should return false when ammo remains in current magazine", [this]() {
			FGunAmmo GunAmmo(EAmmoType::Pistol, 10, 31, 3);

			TestFalse(TEXT("Ammo remains in current magazine"), GunAmmo.IsCurrentMagazineEmpty());
		});
	}); // Describe IsCurrentMagazineEmpty

	Describe("DiscardSpareMagazines", [this]() {
		It("Should discard spare ammo not in current magazine", [this]() {
			FGunAmmo GunAmmo(EAmmoType::Pistol, 10, 97, 7);

			GunAmmo.DiscardSpareMagazines();

			TestEqual(TEXT("Current magazine unaffected"), GunAmmo.GetCurrentMagazineAmmo(), 7);
			TestEqual(TEXT("total ammo only current magazine"), GunAmmo.GetTotalAmmo(), 7);
		});

		It("Should be a no-op if out of ammo", [this]() {
			FGunAmmo GunAmmo(EAmmoType::Pistol, 10, 0, 0);

			GunAmmo.DiscardSpareMagazines();

			TestEqual(TEXT("Current magazine unaffected"), GunAmmo.GetCurrentMagazineAmmo(), 0);
			TestEqual(TEXT("total ammo only current magazine"), GunAmmo.GetTotalAmmo(), 0);
		});

		It("Should result in empty gun if current magazine is empty", [this]() {
			FGunAmmo GunAmmo(EAmmoType::Pistol, 10, 100, 0);

			GunAmmo.DiscardSpareMagazines();

			TestEqual(TEXT("Current magazine unaffected"), GunAmmo.GetCurrentMagazineAmmo(), 0);
			TestEqual(TEXT("total ammo only current magazine"), GunAmmo.GetTotalAmmo(), 0);
		});
	});
}


#endif

You just add the file directly into your project Source folder and in development mode the macro will be active and compile your test into the game module DLL and then you can go into the editor and go to Window->“Test Automation” and it opens the “Session Front End” window.

The groupings follow typical behavior driven development style (BDD) with “Describe” being a way to organize the tests - I usually use the function names here. And then “It” being each individual test. These should read in plain English so they are easy to understand from the results. You should only test one thing at a time and try to cover all the use cases of how clients of your code would use it in expected and unexpected ways. Writing tests is also a good way of discovering holes in your logic and making you think deeper about the problem. It’s also fun to try and break your own code :slight_smile: There are many useful assertion functions in Misc/AutomationTest.h I use AssertTrue, AssertFalse, and AssertEquals the most. If you’ve used any unit testing framework before, the function names follow familiar patterns.

Then under Product Tests you can select the tests identified from the name in the test source and click “Run Tests”. You then get a window with feedback on any failing tests and are looking for all green if they pass:

image
image1348×268 44.9 KB

image
image1262×392 56.2 KB

One bug I found is if you add any new tests, you have to restart the editor as even “Refresh Tests” doesn’t seem to discover them after a hot reload or compile from the editor.

1 Like
2

Oh wow. You really went in-depth. Thank you for making this post

1 Like