Development overview

Mixed reality apps are built with the Universal Windows Platform. All mixed reality apps are Universal Windows apps, and all Universal Windows apps can be made to run on Windows Mixed Reality devices. With Windows 10 and familiarity with middleware tools like Unity, you can start building mixed reality experiences today.

Basics of mixed reality development

Mixed reality experiences are enabled by new Windows features for environmental understanding. These enable developers to places a hologram in the real world, and allow users to move through digital worlds by literally walking about. These are the fundamental building blocks for mixed reality development:

Feature HoloLens Immersive headsets
Input
Gaze✔️✔️
Gestures✔️
Voice✔️✔️
Gamepad✔️✔️
Motion controllers✔️
Perception and spatial features
World coordinates✔️✔️
Spatial sound✔️✔️
Spatial mapping✔️

The basic interaction model for HoloLens is gaze, gesture and voice, sometimes referred to as GGV. All mixed reality devices benefit from the input ecosystem available to Windows, including mouse, keyboard, gamepads, and more. With HoloLens, hardware accessories are connected via Bluetooth. With immersive devices, accessories connect to PCs via Bluetooth, USB, and other supported protocols.

The environmental understanding features like coordinates, spatial sound and spatial mapping provide the necessary capabilities for mixing reality. Spatial mapping enables holograms to interact with both the user and the world around them. Coordinate systems allow the user's movement to affect movement in the digital world.

Holograms are made of light and sound, which rely on rendering. Understanding the experience of placement and persistence, as demonstrated in the mixed reality shell is a great way ground yourself in the user experience.

Tools for developing for mixed reality

The tools you use will depend on the type of app you want to build.

  • 2D apps can use any tools for building Universal Windows Apps suited for environments like Windows Phone, PC and tablets. These apps are experienced as 2D projections and can work across multiple device types.
  • Immersive and holographic apps need tools designed to take advantage of the Windows Mixed Reality APIs. In particular, if building an app we recommend using Unity. Developers interested in building their own engine can use DirectX and other Windows APIs.

Regardless of the type of app you're building, these tools will facilitate your app development experience:

See also