Research Spotlight: Project Acoustics

The more you listen as you move about the world, the more detail and nuance you can hear. Consider the sound of a voice from across the room, compared to a voice down the hall, or from around a corner. Listen to footsteps around the house – how they sound on the carpet in the room you're in, versus how they sound on the hardwood floors from room upstairs, or on the concrete sidewalk outside.

Games, VR, AR and mixed reality can transport people to other worlds through a believable visual experience, with environments and objects that appear and behave much as they would in the real world – or not, to magical effect. They can become even more immersive when the sound that reaches the ears is just as realistic as the light that reaches the eyes.

A decade ago, Microsoft researchers began exploring the physical modeling of the propagation of sound to create more realistic and immersive audio. Starting with proof-of-concept research and continuing to iterate with Microsoft game developers, Project Triton models how sound propagates within a scene given its shape and materials. In doing so, it automatically models immersive sound propagation effects like sound occlusion and reverberation. Project Triton is unique in accurately modeling the true wave physics of sound, including diffraction, while still being fast enough to scale from desktop to mobile devices.

Over the years, researchers worked closely with game developers to improve the performance of Triton, eventually incorporating it into major titles like Gears of War, Sea of Thieves, and Borderlands 3.

The technology behind Project Triton is now available to game developers as Project Acoustics. Project Acoustics' key innovation is to couple a real soundwave-based acoustic simulation with traditional sound design concepts, translating simulation results into traditional audio DSP parameters for occlusion, portaling and reverb.

On August 13, Microsoft Principal Researcher Nikunj Raguvanshi will lead a webinar on the ins and outs of creating practical, high-quality sound simulations. It will include an overview of the three components of sound simulation: synthesis, propagation, and spatialization. For each, Nikunj will review the underlying physics, research techniques, practical considerations, and open research questions. The webinar will also explore:

• Challenges and requirements of practical interactive simulation 
• Fundamentals of the physics of sound generation, propagation, and perception 
• Overview of research in sound simulation and future research directions 
• Detailed discussion of Project Triton for fast sound wave propagation  
• Project Acoustics for plugin integration of Triton and HRTF spatialization in your projects 
• Various audio-visual demos of immersive sound simulation 

To learn more about Project Triton and Project Acoustics check out this Microsoft Research podcast to take a deep dive into Project Triton over on their website visit the Project Triton page for a deep dive into the research.