Hadean helps CCP Games realize its vision with Azure

Since 2003, the massively multiplayer online (MMO) game EVE Online has continued to attract new players into real-time battles between several thousand concurrent users. But its growth meant developer CCP Games was facing challenges inherent to its aging architecture. London-based engineering company Hadean saw the sci-fi title as an opportunity to demonstrate its powerful Aether Engine spatial simulation library. Using Aether Engine, Microsoft Azure, and Azure PlayFab, Hadean created EVE: Aether Wars, a low-latency, high-performance playable tech demo that can quickly scale to support more than 10,000 concurrent players and ensure an enjoyable player experience.

Using Azure and PlayFab, we were able to improve performance, reduce latency, and dynamically scale the multiplayer experience. -- Michael Gunadi: Partner Manager Hadean​

A legendary MMO pushes new boundaries​​

Since its launch in 2003, the massively multiplayer online (MMO) game EVE Online has gained hundreds of thousands of loyal fans across the world. So loyal that in January 2018, it successfully broke the Guinness World Record for “most concurrent players involved in a single multiplayer PvP [player versus player] videogame battle.” A total of 6,142 players took part, but the moment-to-moment gameplay experience wasn’t optimal. 

EVE Online’s back-end architecture struggled to accommodate the enormous player count, leading to persistent performance issues, while authentication difficulties meant many players were locked out of the space-themed game. It was an ambitious spectacle and one that inspired London-based engineering startup Hadean to meet with the development studio, CCP Games, to discuss the title’s future ambitions.

Hadean, creator of Aether Engine—a distributed spatial simulation library built on its cloud-first platform—believed it could help CCP Games overcome its technical difficulties and realize exciting new gameplay possibilities in EVE Online.

“We asked CCP what their ambitions for the title were, and we found they had all these great ideas that they couldn’t pursue because they were hitting a technological ceiling,” says Michael Gunadi, Partner Manager at Hadean.

The meeting sparked a partnership between Hadean and CCP Games, leading to the birth of EVE: Aether Wars, a groundbreaking real-time tech demo. Powered by Aether Engine, Microsoft Azure, and Azure PlayFab (a part of Microsoft Game Stack), it first premiered at the 2019 Game Developers Conference (GDC).​

EVE: Aether Wars goes live at GDC

Founded in 2015, and a graduate of the Microsoft Accelerator program (now Microsoft ScaleUp), Hadean was built to solve a shared challenge that all its founders had contended with for years: dynamically scaling performance as an application grows. To achieve this, Hadean built a development platform using Azure—with deployment assistance from the Microsoft Global Black Belt team—to help developers create powerful, dynamically scalable applications. 

“We’ve been working with Azure since day one at Hadean,” says Gunadi. “CCP Games saw the resiliency of Azure and how we were able to replicate the technical stresses that they were bracing for during a simulated gaming event.”

Gaming wasn’t Hadean’s initial focus. But it quickly became clear that common challenges in the massively multiplayer gaming sector offered an ideal opportunity to demonstrate its technology’s capabilities.

“Gaming doesn’t have the regulatory requirements of fintech and the pharmaceutical industry, and it’s home to some of the most interesting computational problems,” says Gunadi. “Scaling gameplay for thousands of concurrent players worldwide is one of the most unique and challenging of them all.”

To coincide with GDC 2019, Hadean and CCP Games went live with EVE: Aether Wars to demonstrate what Aether Engine could do with Azure as its backbone. The live event was joined by 3,852 players from across the globe, and with Hadean introducing AI-controlled bots, the number of concurrent users peaked at 10,412.

“We were trying to strain the system and network as much as possible,” says Gunadi. “We connected bots from another cloud into Azure, on which we ran the simulation. This helped us create a technically authentic replication of players joining the game from random parts of the world.”

Although Hadean provided its own login server for the event, huge demand meant its servers struggled to authenticate players in real time, delaying the start of the test by approximately 30 minutes. Also, the simulation couldn’t deliver a consistent 30-hertz tick rate—the accepted industry standard—leading to an unpredictable gameplay experience.​

EVE: Aether Wars Phases II and III

As a pure technology test, EVE: Aether War’s debut at GDC was a success. But for the second phase, which went live at Gamescom, a global interactive trade show, in August 2019, Hadean passed its technology directly to CCP Games to refine the experience.

“In eight weeks, CCP Games was able to massively improve the demo,” says Jeremiah Vocaturo, Backend Game Developer at Hadean. “We saw 140,000 missiles in flight, a stable 30-hertz tick rate, and a more user-friendly login experience.”

And with Aether Engine at its disposal, CCP Games no longer had to use Time Dilation, a gameplay mechanic employed by EVE Online to slow down the in-game clock and allow the simulation room to catch up. Instead, Aether Engine can dynamically allocate Azure resources within approximately 50 milliseconds to achieve real-time in-game simulation.

“In the second phase, we had 4,369 concurrent players join the game, which is fantastic,” says Gunadi. “From there, we needed to continue to find a better way to authenticate players quickly.” 

For its third and final test phase at the EVE Fanfest in November 2019, Hadean added PlayFab as a player authentication layer, with a stress test proving it could now handle more than 30,000 people signing on simultaneously. 

​​PlayFab passed all our sign-up stress tests. We also gained improved security against people attacking the game with malicious sign-ups—that was a load off our minds.        -- Jeremiah Vocaturo: Vocaturo, Backend Game Developer Hadean​​

For CCP Games, the combination of Aether Engine, Azure, and PlayFab also meant it could build and integrate the game locally, then seamlessly shift and deploy it on any cloud at scale. “With Aether Engine dynamically scaling on Azure, CCP Games’ engineers spent less time pulling levers to keep the game running or having to guess how much compute they need to allocate in anticipation of large player counts,” says Gunadi.

Over the three phases, Hadean has refined and proven Aether Engine’s capacity and power on a global stage. It was also able to reduce egress costs by more than 100 times from the first GDC testing phase to its final demonstration at CCP’s EVE London player convention.  

EVE Online enters a new era of growth​​​

With Aether Engine running on Azure and PlayFab, CCP Games has everything it needs to improve the game experience and free its developers to spend more time doing what they love: building enormous, creative worlds where thousands of players can connect and play together.

“Using Azure and PlayFab, we were able to improve performance, reduce latency, and dynamically scale the multiplayer experience,” says Gunadi. “That’s paved the way for CCP Games to grow its vision for EVE Online and grow its business by servicing more players simultaneously around the world.” 

Find out more about Hadean on Twitter, Facebook, and LinkedIn.