Disclaimer: the knowledge contained in this post was largely gained by spelunking through the source code of UnrealTournament, which is available on GitHub for all holders of an Epic Games account. I hope this article will save other people much of the pain of learning to use these (almost entirely undocumented) capabilities of UE4.
In this blog post I will give a high-level overview how to cook and package Unreal assets such as blueprints and meshes for loading at runtime. This is necessary if you ever want to:
- Publish modular Downloadable Content (DLC) for your game
- Allow users to create and share new content
- Save having to rebuild and redistribute your entire game every time you want to add new content
I will focus on the second requirement, creating and sharing new content, as it’s what first motivated me to investigate this. I may later write a follow-up article on how to write an editor plugin that largely automates this process. This post will be light on specific implementation details, but I may provide example code in a future post.
Contrary to popular opinion, it is entirely possible to build a working DLC / modding system without modifying the engine source. A small amount of C++ programming is required, but if there’s enough interest I will try to release a code plugin for use in Blueprints.
There are two main topics that are important to understand in order to achieve this: the Content Pipeline and the Asset Registry.
The Content Pipeline
The Unreal Editor is, at the time of writing, generally set up to only allow monolithic builds of your game. The encouraged workflow is to build all your content into a single application that you ship to your users. But that’s not we want to do here – we want to be able to distribute additional content at a later date without having to do a full rebuild.
So how can we do that? Well, first we must understand what is involved in that build process.
You probably already know the three steps1 that all content must take before it can be loaded and used by a game:
- Cooking - the process of preparing assets for use at runtime by stripping out editor data and optimising them for the chosen platform.
- Packaging - the process of storing the cooked assets into a
.pakfile, which allows bundles of related assets to be compressed and distributed as a single file.
- Loading - the process of actually loading the assets during the game’s runtime, for example to instantiate them in the level.
Luckily for us, the first two steps can be achieved very easily with the editor.
Cooking & Packaging
You may already be aware that Unreal has some support for creating extra “DLC” (Downloadable Content) by hijacking the plugin system, as detailed by Tom Looman in his great tutorial on creating a modding system.
You should read it, but to summarise, the editor can be told to distinguish between a core “full game” build and a build of DLC content by simply ticking a couple of boxes in the launch profile. The produced DLC .pak file can then simply be placed in the runtime application’s directory and it will be automagically loaded on startup. Hooray, problem solved… right?
Unfortunately what is not at all obvious to the casual reader is that this alone
is, to a large extent, totally useless. The reason for this is simple: if the
core build is not executed with the DLC plugin present in the Plugins folder,
the content within the produced
.pak will not be accessible at runtime.
Clearly this negates the whole point of being able to produce extra DLC content,
if it must be known about at the time of the full release.
So why is this case? Why can’t we access the DLC content at runtime? To understand that, we must first understand the Asset Registry.
The Asset Registry
The Asset Registry is essentially a central index of all assets available to the game. That means blueprints, meshes, widgets – anything that you would want to load.
The advantage of having this central index is that it allows the game to quickly search for assets that satisfy some criteria. In the case of UnrealTournament, for example, it is used to populate the drop down menu of available hats. It also provides functionality for finding dependencies, which is important because a blueprint might reference a mesh, which references a material, which references a texture… and so on.
Every running instance of the game has a singleton Asset Registry sitting in
memory, which is loaded from a
AssetRegistry.bin file that is distributed with
the content. This file is an artefact of the cook process – it lets the game
know what content was cooked and where in the Unreal Filesystem (UFS) it can be
found. If you’ve tried cooking DLC in the editor, you’ll know that the way it
works is by taking the difference between the release AssetRegistry of the
“release build” and the current one, therefore only cooking and packaging the
Remember that our aim is to allow multiple packages of content to be loaded; that means multiple cooks, which in turn means multiple Asset Registry files.
Clearly it is necessary to merge the contents of all these separate files into
the runtime Asset Registry to allow the game to find all the content. It turns
out that this is the only part of the process which the engine won’t do for
you. The best way I’ve found to do this is simply to iterate over all the
files found at runtime and call
IAssetRegistry::Serialize() on them2.
Here’s a diagram showing how that works:
Note that in reality you’ll want all your
.bin files to have different names
to avoid naming clashes when you load them. The convention UnrealTournament uses
is to simply add the name of the containing
.pak file as a prefix. The
advantage of this is that it allows you to search the content folder for all
.pak files and then load the corresponding
sidestepping the chicken-egg problem of knowing where to find them.
You’ll probably want to reserve
AssetRegistry.bin for all your core release
content in any case, as this is the special case that is merged in
Loading at Runtime
I mentioned before that your distributed game will automatically load any
<PathToGame>/Content/Paks. As in many other game engines, the Unreal
Filesystem (UFS) is actually an abstraction over the system filesystem for the
sake of portability. It’s important to know about this because otherwise it can
be very confusing to see paths like
/Game/MyContentFolder/Asset that don’t
seem to correspond to files in the Windows (or other system) filesystem.
The interaction between
.pak files and the UFS is called mounting. Much like
an OS might mount a volume, when the game starts it will mount each
Content/Paks folder (and all subfolders). This maps the location of the
assets within to the UFS, allowing them to be loaded.
.pak file is created using the UnrealPAK utility, the mount path of
each file is specified. This is then used when the
.pak file is loaded to
determine where the file should be mapped to in the UFS. It is vitally
important that the asset paths in an
AssetRegistry.bin file match the mount
paths used to create the
The general convention seems to be that a file in
<PathToGame>/Content/MyFolder/MyFile.uasset is mapped to
/Game/MyFolder/MyFile.uasset in the UFS.
Assuming you’ve merged in the
AssetRegistry.bin from your
.pak file and the
mount paths were correct, you should be able to load the assets within just as
you normally would, e.g. using the
SpawnActor blueprint node.
This was a whistle-stop tour of getting Unreal Engine 4 to work with modularised content. Let me know your feedback on the forum thread.