Microsoft Flight Simulator: The Future of Game Development

As gaming continues to take leaps forward, the truly transformational, boundary-pushing next-gen game development will be powered by the cloud. Microsoft Flight Simulator is but one example of a title stepping boldly into the potential unlocked when compute, networking, and storage are no longer locked to the device gamers are playing on. In this deep dive, we share a glimpse into the tech stack Asobo Studios used to build Microsoft Flight Simulator. Leaning heavily upon Microsoft technologies, the tools and services from Microsoft focus on empowering game developers of all sizes, building games across all platforms, from PC to Xbox, PlayStation, Nintendo Switch and mobile. If you're a developer or publisher who's interested to learn more about how Microsoft can help you to push the boundaries of what's possible for your own games, then this piece is worth the read. ​

What does it take to make a "digital twin" of planet Earth on Azure? The Microsoft Flight Simulator team found out. This article outlines the team's journey, the product that resulted, and how Microsoft Flight Simulator expands the bounds of simulation, gaming, and deep experiences yet to be created. Additionally, we'll share insight into how Asobo Studio leveraged Microsoft's portfolio of commercially available game-development tools to push the boundaries of what a next-generation game can be, harnessing the power of the cloud to deliver a fully explorable planet Earth with levels of detail not previously possible.

With Microsoft Flight Simulator, enthusiasts are embracing a wholly new version of the product, the first major release of the famous franchise since 2006. With an interval of 14 years between offerings, it's not unreasonable to expect that the new version would represent a leap forward. Even so, it's fair to say that Microsoft Flight Simulator provides a simulation experience that is radically different and unprecedented in the scale of the realistic digital twin world that brings something very close to the real world to you.

For developers across the ecosystem (and not just gaming), Microsoft Flight Simulator represents nothing less than a new paradigm for creating cloud-enabled experiences that utilize multiple big data sources, massive cloud-based processing and storage in the Azure architecture, and AI-enabled high-fidelity user realism. Just as important, it offers all of this with a development model that greatly reduces the effort and cost of delivering highly detailed simulation. The call to action for game and simulation developers is clear: by enabling an offering like Microsoft Flight Simulator 2020, Azure raises the quality and capabilities bar sky-high but simultaneously brings the ability to create these sorts of high-quality experiences within reach of many first- and third-party developers today.

For users, the experience is also unparalleled. Created in partnership with French game developer Asobo Studio, Microsoft Flight Simulator 2020 offers unprecedented realism, both in the aircraft flying experience and in the virtual world. Frequent updates continually improve the world representation and include additional environmental realism, customization, and fun AI features.

The result in the last year? Enthusiastic user embrace. Within the first six months, Microsoft Flight Simulator 2020 had more than 2 million users, making it the fastest launch of any version in the nearly 40-year history of the title. An active community shares experience and advice and livestreams flights, and users contribute game content, such as more detailed and realistic airports that are usable within the simulator.

The Microsoft Flight Simulator experience continues to quickly evolve and grow, in part because of the extensibility of the Azure technologies on which it is built. These allow for frequent updates and a robust third-party market for aircraft, scenery, and airport add-ons. "What's really cool about it is how open it is," says Jorg Neumann, Head of Microsoft Flight Simulator at Microsoft. He points out that with Azure, "One can support and extend bespoke game engines like the one used for Microsoft Flight Simulator, and developers can quickly add features to an existing product." The ability to plug in tools is virtually boundless. "You can bring in all kinds of stuff and it just works because the computing is done elsewhere."

Much has been written and recorded about the origins of Microsoft Flight Simulator 2020. Those stories have been well-shared in the community: 

• Noclip video: How Microsoft Flight Simulator Recreated Our Entire Planet
• Gizmodo: A Pilot's Review of Microsoft Flight Simulator 2020
• Engadget: How Microsoft Flight Simulator became a 'living game' with Azure AI
  

We won't retread this familiar ground. Instead, we'll look at Microsoft Flight Simulator as a groundbreaking offering that has wide-ranging implications and opportunities for developers across many applications. ​

Origins

Microsoft Flight Simulator has always been a definitive offering for Microsoft. The company's very first packaged product dates from 1982, and each release added more resolution, features, and detail. In particular, each pushed the capabilities of graphics processing to emulate the real world as closely as possible. Consequently, the power of desktop computers at each release has always been an important consideration for the title.

Microsoft Flight Simulator 2020 takes a huge step away from this dependency by moving much of the processing functionality to the cloud. The product still requires a beefy desktop device (though this will change—more on that later), but the experience delivered today is far beyond the limitations of any desktop because it fully harnesses the massive storage, processing power, integration, and streaming bandwidth of Azure. The result: a high-fidelity, 3D world with real-time conditions, realistic and intelligent interaction, and an extensible, scalable integration platform for any range of experiences imaginable.

Takeoff: The technology

"If you don't have something new to say, don't make a sequel."
—Jorg Neumann, Head of Microsoft Flight Simulator, Microsoft

From the outset, for the team, Microsoft Flight Simulator 2020 had to be worth making. And, to make it worth making, it had to be what customers wanted.

Customers wanted Microsoft Flight Simulator to be capable of supporting flying under Visual Flight Rules (VFR). VFR is the set of conditions under which real-world pilots may operate an airplane by sight, as opposed to relying on instruments (Instrument Flight Rules, or IFR). Flying under VFR means that weather conditions are favorable and that the pilot can orient themselves by observing what they see on the ground, along with basic reference to instruments. IFR, by contrast, requires the pilot to primarily use instruments to operate the aircraft, such as in adverse weather when visibility conditions are below minimum.

Although IFR simulation requires more technical flying skills, VFR can be a more viscerally engaging experience—if the simulation of the outside world is sufficiently accurate and highly detailed.

Providing this believable and exciting experience means not just displaying obvious topographical features, like major bodies of water and mountain ranges. It also means showing every bit of detail between any two points on Earth—down to a level of specificity that enables a simulator pilot to find their way from one spot to the next, based on real-world representation.

This is what customers wanted. The implications of this initial vision caused the product team to rethink the traditional approach and to assemble resources and services in novel ways:

• Navigation. Navigating the Earth means maps—lots of maps—with high levels of detail, available in real time.
• Maps. Maps are generally 2D, however, and a rich user experience requires creating objects on the ground in 3D—not just airports, which are a special case, but also cityscapes down to the level of your house on your block. It also includes natural features in topographic relief down to the level of individual trees.
• Weather. Simulating weather requires not just a menu of presets but actual weather conditions—anywhere in the world—rendered accurately in real time.
• Aircraft details. The aircraft you pilot should behave like its real-world counterpart. Enabling this degree of realism requires a host of aircraft data points that allow aircraft to behave in more subtly detailed and realistic ways than ever before.
• Realistic interaction with other aircraft. Air traffic control (ATC) interactions should reflect real-time aircraft tracking worldwide and ATC communications—not a finite set of canned interactions.
• Airports. The ground experience should reflect what you would see and experience if you were actually piloting an aircraft. This means including highly accurate and detailed models of every airport in the world.
  
  These are the starting requirements that made Microsoft Flight Simulator worth making. Add to them the need to make these resources work together seamlessly and to make them routinely updatable and easily extensible. With requirements like these, the new Microsoft Flight Simulator could never live primarily on your desktop. It depends on a very different technology landscape than existed in 2006. 
  

These user requirements drive the technology choices that Microsoft and its development partner, Asobo Studio, make, including:

• A Bing Maps data set comprising more than 2.5 petabytes of data with over 37,000 airports, 2 million cities, 1.5 billion buildings, 2 trillion trees, and 117 million lakes.
• Real-time atmospheric simulation and live weather.
• AI-generated air traffic control voice.
• Azure PlayFab to create a robust marketplace to deliver additional features from day one, like community-contributed content, such as airport models with increased detail.
• Cloud-powered multiplayer features. ​

Figure 1: The Microsoft Flight Simulator high-level architecture creates a digital twin of the real world.

Azure brings exponentially more computing power and storage to Microsoft Flight Simulator. Petabytes of Bing data and images are preprocessed and stored—along with photogrammetry data, vector-based maps, and trillions of accurate tree and other environmental masks—entirely on Azure.

Given the earth-sized scale of the need to accurately render the digital twin, or high-resolution digital version of the real world, suffice it to say that the processing work is immense.

It's impossible to place trillions of trees across the earth accurately as a manual process, for example. Sebastian Wloch, Co-founder and CEO of Asobo Studio, describes the process that Asobo used to render trees as an intelligent algorithm—machine learning. "Instead of finding an algorithm that is going to find trees [in satellite imagery],…you just train it." Asobo developed tools that use examples of tree imagery to render trees in 3D within the Microsoft Flight Simulator environment. You might suspect that identifying trees across the entire world would require a lot of computing power. It does—but Azure Compute (including Azure Cloud Services) can scale as needed to process vast amounts of data. "It's basically able to paint by itself and, well, it's just another computing job." Albeit a big one. "Azure is taking hundreds or tens of thousands of jobs and dispatching them on many computers," Wloch says, "in order to not wait two years until you get your trees." Detecting trees on a worldwide scale may take days of computing time—but once detected, the tree metadata is stored in Azure Storage, ready to stream to any number of players as needed.

In addition to the vast amount of precomputed and stored data, Azure enables the integration of real-time data sources. Most prominent among these are real-time weather effects, which allow users to experience the actual weather in their flight path at the moment they're flying. With weather data sourced by partner meteoblue, weather effects are processed in Azure Compute and layered into the user experience that streams down to the user's device. Likewise, partner services, such as FlightAware, deliver real-time air traffic. And cloud-based AI services provide air traffic control voices to Microsoft Flight Simulator pilots.

All of this rich data streams to the desktop device as needed—a much smaller data stream than would be required if, for example, terrain and weather effects were stored on the desktop.

Overall, the level of integration between cloud and desktop device is extraordinary—but it's also logical and generalizable. The approach offers the rich experience of the entire earth—or, potentially, other earths.

In flight: Realizing the experience

The significance of Microsoft Flight Simulator extends beyond gaming and beyond simulation. It's really a new way to do game development, Neumann explains. In particular, the ability to model rich environments via machine learning can revolutionize world construction cost. The ability to store this vast amount of data in the cloud can also help drastically reduce cost.

The Microsoft Flight Simulator paradigm makes it easy for developers to create significant new features and functionality. Indeed, the potential goodies that could be tackled are exciting. Imagine, in the not-too-distant future, Neumann suggests, realistic AI-driven ground-life features—a million caribou marching through the countryside, for example, could all be precomputed in Azure and streamed to the desktop.

If you're a game or simulation developer, these sorts of data- and processing-intensive features become doable on a routine basis because the platform does the heavy storage and processing for you. The savings implications for development are huge. Developers typically spend a large proportion of budget creating the environment in either bounded (and static) or procedurally generated (and not highly realistic) worlds, Neumann points out, and are subject to the limitations of either approach. The capabilities of Azure—for storage, computing at scale, integration of real-time data sources, and ability to stream with low latency—all expand opportunities for game (and other) developers to make richer, more realistic, and unbounded experiences, at lower cost. "My cost dropped 50 percent since I started," Neumann reports. "All my estimates were too high. Because frankly both storage and compute got cheaper by a lot and it continues to be so."

There are solid benefits. "If I were a developer, I'd be very happy. There are new solutions coming your way that make your game production more feasible. You certainly can get something really cool off the ground at a much lower price point," Neumann says. "I believe there is a chance building large games ​will become cheaper again. We were throwing money at those things, but I think we can go be clever now. This is another way of being clever."

Landing: The future

With the Microsoft Flight Simulator architecture, the future comes quickly in the form of ongoing extensions of the product and platform. Chief among these will be accessibility by a broader range of devices. Although the initial product release requires powerful PCs, the Azure architecture will enable the Microsoft Flight Simulator experience on other devices, too—notably Xbox consoles and, ultimately, mobile devices. This device democratization means that the product does not require more power on the device to scale, rather it just needs streaming bandwidth. Put another way, Azure allows "every device to punch above its weight," notes Neumann. "What you see right now that people are impressed by on a $3,000 PC—guess what? It comes pretty much unchanged to a $500 console, and we are putting things in place to bring it to even lower spec devices, like phones."

At the same time, game and simulation developers extend similar experiences and enrich a range of applications, including real-world pilot training that exceeds the verisimilitude of current pilot training technology. Or adds a 1 million AI-generated caribou herd to a tundra landscape. The opportunity is for developers to provide more meaningful experiences for users via Azure, limited only by imagination. Everything everywhere.

"The cloud is not just the infrastructure it enables you to bring in. It's all kinds of the tools of humanity, quite frankly, and I know that maybe sounds cheesy, but it's totally true."

              —Jorg Neumann, Head of Microsoft Flight Simulator, Microsoft​

To learn more about how Microsoft can help you push the boundaries of what's possible for your own games, visit us at Microsoft Game Dev.