Building Arbitrary Video Game Projects on Azure
About the author
Seth Denner has been building web and mobile based technology solutions for Seattle area start-up companies since 2007. Seth is currently implementing and deploying automated build systems at PlayEveryWare. PlayEveryWare is a video game development company based in Seattle that specializes in providing porting support to companies bringing their games to console platforms.
Setting up an automated build process for console video game development can pose quite a challenge. Most web service based solutions on the market have a very narrow toolchain support and even less have any kind of support for targeting current generation consoles. Here at PlayEveryWare we take on many projects where we have to make a PC game run on console platforms and we have no control over what engine framework our clients are handing over for us to work on. There is no off the shelf one-size-fits-all service that can handle the diversity of projects that come through our door. Luckily by leveraging a few Azure services and a bit of elbow grease there is a solution to this problem of never knowing what kind of game project you are going to build next.
All developer interactions and automated triggers go through an Azure serverless function first. Serverless functions are the glue that bind the rest of the service architecture together.
Creating and deploying a serverless function is quite easy using the Azure CLI or Azure Powershell and is itself is fully automatable so you can have automated builds and deployments for your automated build and deployment functions. Here is an example of how to set up your own function app.
First let's do a sanity check to see if your environment is set up appropriately:
Functions Core Tools should be version 3 or greater and Azure CLI should be version 2.4 or greater. If that all looks good log into your Azure account if you haven't already:
After satisfying these prerequisites creating a new function project and app can be achieved with the following commands:
This function starter is very helpful in getting your new app up and running quickly. The req object contains the HTTP request data that you can consume in your function and creating the HTTP response is as easy as assigning an object to the context.res member.
An example of a useful serverless function we use is a bot that integrates with our internal chat channels. The bot as you may have guessed is an integration with a popular team chat service many are probably familiar with using. Implementing a bot utilizing the API provided by our chat vendor is a great use case for serverless architecture. When automated tasks are complete they can notify the serverless function and the bot will in turn figure out the appropriate channels to post notifications to. What if you need to make an ad-hoc build of a project? No problem the bot can render you a form to fill out in the chat UI and kick off the specified build for you.
Azure Functions are a great solution for any kind of stateless operation that needs to be handled such as triggering builds, rendering UI or implementing a full-fledged REST API. The on-demand nature of serverless functions makes handling periods of higher usage with ease while not incurring unnecessary cost during periods of low usage which is great for our use case as much of the time the daily timer triggered build requests and their corresponding result notifications are the only functions that run on a given day.
While serverless functions are great sometimes you need to there are some limitations of the kinds of things you can achieve.
Say you need to deliver the ROM package that your automated video game build system just produced and to do that you need to run a command line tool that uploads a completed build package to the cloud storage provider used by our customer. Although it is possible to deploy executables to your Azure function it is not desirable to have this kind of package or functionality bloat in our serverless functions.
This is where Docker containers take over the workload. Using Azure Container Instances we can implement custom functionality for a project that is easy to develop, deploy and scale. The atomicity of the containers make it a low risk way to integrate tasks that may be specific to just one or two projects. Another benefit is that these containers can run asynchronously from the main build job so that your builds can focus on cranking out packages while any other publishing or validation tasks can run in their own time.
There are many great tutorials out there for creating and deploying containers on Azure so I will not cover that in this post but for more information see: Deploying Docker containers on Azure.
Windows 10 Virtual Machines
As would be expected from being a part of the Microsoft ecosystem, Azure provides great support for Windows 10 Virtual Machines including graphics driver support which is often a necessity when building games. Being able to create a custom image for each project with the environment preinstalled makes it easy for a CI system such as Jenkins to spin up build nodes on demand.
In our case we use the Azure VM Agents Plugin to manage our windows 10 build instances. An example configuration can be seen here:
To set this up you simply need to create a Service Principal with the appropriate permission scope to create Virtual machines and add the credentials to the Jenkins credentials manager by clicking the add button. Then select an appropriate resource group or have Jenkins create a new one for you (requires additional permissions).
With the authentication part out of the way you can now configure a new instance template that Jenkins will use to create new instances to build your project on.
Most of the options here are pretty straight forward and can be tailored to your specific needs. The interesting bit is the Image Configuration section:
If one of the base images provided by azure serves your needs then by all means select one of those. For us however we have a lot of prerequisite packages to install to support the Unity projects we build so we use a Custom Managed Image. Select the option and put the Image ID of the image you want Jenkins to spin up to build this project type. Then select the appropriate OS type and a launch method that works with your network topology and Jenkins will take care of the rest including deallocating your instances once builds are complete.
Most projects we build get their own dedicated image. This makes troubleshooting any build issues we run into much easier as there aren't a bunch of unnecessary packages clouding the environment and possibly causing incompatibility issues. This isolation allows us to preserve the state of working build environments.
Many projects require several build passes to produce packages for each target platform and for creating debug and release builds. Our CI system can easily scale up the number of build nodes using the appropriate VM image for each project and run these build jobs in parallel. This can be especially important for large games that can take hours to build which would otherwise take the better part of a work day or more to run in series. This would lead to delays in delivering packages with the current changes to our QA team and in turn lead to delays in reporting any issues back to our development team creating slow turn around times on even small issues. Keeping the QA team loaded with packages to test is key here to ensuring that bugs are iterated on and squashed in a timely fashion making sure that we hit our deadlines for delivering our final releases.
Wrapping Things Up
You may have noticed that there is very little here that is specific to building video games. You absolutely could use this strategy to provide cloud builds for any type of project. However there are quite a few solutions out there that can provide cloud builds for simpler or at least more predictable project environments. However if you never know what kind of project you're going to need to build next and the projects often have extraordinary prerequisites required to build then using an architecture similar to the one outlined here may be the solution for you.
If you have a game project and need some help bringing it to other platforms PlayEveryWare can assist you in realizing these goals. Please reach out to us if you'd like to learn more about how we might support your game coming to current and next generation consoles at firstname.lastname@example.org.
And don't forget to check out our latest release Among Us where you join your crewmates in a multiplayer game of teamwork and betrayal available now on the Nintendo eShop for Nintendo Switch.