At Mercuna we make extensive use of automated testing including continuous integration. Our AI navigation middleware is used on multiple game engines (Unreal Engine and Unity), multiple versions of engines, on multiple platforms (Windows, Mac, Linux) and for multiple platforms (the above plus consoles and mobile devices). As such it’s essential that we “fail early”, that is to say that errors introduced during development of a particular feature that don’t occur on all configurations are resolved during the development phase, rather than us trying to resolve them all just before shipping (or even worse, just after!).
Covering how we automate all of those is definitely outside the scope of a single blog post, so here I’m just going to talk about one of our mainstays: Compiling against multiple UE4 Linux Installed Builds on Buildbot.
There are a couple of reasons that this configuration is attractive to us, including
- Linux – Linux usually has the strictest compiler and the fastest build times.
- Unreal Engine Installed Builds – Binary builds allow us to have more versions, and don’t need to be recompiled when the OS headers change.
- Buildbot – An open source continuous integration framework that allows easy control with python scripts
In the following I’m going to describe some of the steps we needed to get this working. You’ll want a linux machine, I’d recommend with an SSD, with > 100GB of free disk space (though a final version may be <25GB), and I’m using Ubuntu 20.04, although we’ve had this working on older versions of Ubuntu before.
Get the Unreal Engine 4 Source from GitHub
The first thing you’ll want to do is checkout the UE source for the version you want from Epic’s GitHub repository,
git clone [email protected]:EpicGames/UnrealEngine.git -b 4.XX --depth 1
where
4.XX – refers to the UE4 version (e.g. 4.24)
--depth 1 means we just want the latest version (to save disk space we don’t need the whole history)
The next step is to setup this repository with the usual
./Setup.sh
This is going to take several minutes and take about 80GB of your hard drive, so now might be a good time for a cup of tea…
Make an installed Linux UE4 Build
We could use the above for a source build of the engine (see SettingUpAnUnrealWorkflow[unrealengine.com]) but since we’re going to have multiple installed builds we don’t really want 80GB for every UE version we have, and we don’t really want to recompile if any headers change. As such we’re going to create binaries (an installed build).
The output of this command is going to create a build in LocalBuilds/Engine/Linux relative to the engine path. You can change this by editing the /Engine/Build/InstalledEngineBuild.xml but to keep this simple we’ll just use the vanilla version and move it to where we want afterwards. This is going to take over an hour to compile, so you might want to consider lunch…
Engine/Build/BatchFiles/./RunUAT.sh BuildGraph -target="Make Installed Build Linux" -script=Engine/Build/InstalledEngineBuild.xml -clean -set:HostPlatformOnly=true -set:WithDDC=false -set:GameConfigurations="DebugGame;Shipping"
The first few options are self explanatory,
-set:HostPlatformOnly=true
i.e. we don’t want e.g. console builds, even if we had earlier set the engine up for them,
-set:WithDDC=false
Don’t build a stand-alone derived-data cache (speeds up your build if you intend only to compile, rather than run the build)
-set:GameConfigurations="DebugGame;Shipping"
This is a bit deceptive, since you can compile in development or debug configurations against either an installed development or debug engine (but for shipping you must build shipping).
Depending on your settings this is a 15-25GB resultant package (depending on engine version). In our workflow we move the result to a separate path and delete the source checkout, though you can of course fetch a different engine branch and repeat.
Build your project on the command line
Before we try and get Buildbot to automate building our project, we’re going to manually make sure it works (usually a good idea). Building your project on the command line usually involves the following two steps:
1 Generate project files
To generate the project files you can run
ENGINE_FOLDER/Engine/Build/BatchFiles/Linux/./GenerateProjectFiles.sh PROJ_FOLDER/ExampleProject/ExampleProject.uproject -NoIntelliSense
For some reason UE4 can be a little picky about absolute (rather than relative) paths, so if it’s not working for you then it might be that. IntelliSense is the code completion tool from Microsoft’s Visual Studio (which we aren’t using here) so we’ve omitted it to speed up the generation somewhat.
2 Build the project
To build the project you’ll need
ENGINE_FOLDER/Engine/Build/BatchFiles/Linux/Build.sh ExampleProjectEditor Linux DebugGame -project=PROJ_FOLDER/ExampleProject/ExampleProject.uproject -WaitMutex
Note that for Editor builds you have the “Editor” suffix after the project name, and “DebugGame” can be replaced with “Development” (you can’t build the Editor for Shipping, that would have to be a non-editor build).
Optionally, you may also want to run your project (not just test it compiles) with
ENGINE_FOLDER/Engine/Binaries/Linux/UE4Editor-Linux-DebugGame PROJ_FOLDER/ExampleProject/ExampleProject.uproject
(running the editor or game engine binary as necessary).
Automate the build with Buildbot
We now replicate the two building steps above with buildbot (buildbot.net). If you’re not familiar with buildbot, essentially you create a series of “build steps”, with properties either set as strings, or using the util.Interpolate closure which lets paths be finalised (i.e. relative to the build) at build time.
For running GenerateProjectFiles.sh we’re just going to use a simple ShellCommand step:
from buildbot.plugins import steps, util
generate_project_files = engine_dir + "/Engine/Build/BatchFiles/Linux/./GenerateProjectFiles.sh"
proj_dir = util.Interpolate('%(prop:builddir)s/ExampleProject')
project = util.Interpolate('%(prop:builddir)s/ExampleProject/ExampleProject.uproject')
command = [generate_project_files, project, "-NoIntelliSense"]
gpf_step = steps.ShellCommand(name="Generate Project Files",
command=command,
workdir=proj_dir,
haltOnFailure=True,
description="Generate Project Files")
For the build step we use the step from https://github.com/pampersrocker/buildbot-UnrealEngine.git
from buildbot_UnrealEngine.BuildTool import Build
build_step = UEBuild(name="Build Example",
engine_path=engine_dir, project_path=project ,
target="ExampleProjectEditor", target_config="DebugGame",
target_platform="Linux", build_platform="Linux",
engine_type="Installed", timeout=10000, haltOnFailure=True,
description="Build Example")
These steps can be added to a util.BuildFactory() in the normal way.
Thoughts
A continuous integration server can take a while to set up, but can save you a lot of pain down the line. We’ve found this approach works well for us, though it’s certainly not the only way to get your projects up and tested. We know a lot of others who’ve been using Jenkins for their continuous integration, and some of you are mostly building on Windows machines (where you can just download the binary builds from Epic). We hope you found this useful!