This tutorial will walk you through creating a basic mixed reality app built with Unity.
Chapter 1 - Create a New Project
To build an app with Unity, you first need to create a project. This project is organized into a few folders, the most important of which is your Assets folder. This is the folder that holds all assets you import from digital content creation tools such as Maya, Max Cinema 4D or Photoshop, all code you create with Visual Studio or your favorite code editor, and any number of content files that Unity creates as you compose scenes, animations and other Unity asset types in the editor.
To build and deploy UWP apps, Unity can export the project as a Visual Studio solution that will contain all necessary asset and code files.
- Start Unity
- Select New
- Enter a project name (e.g. "MixedRealityIntroduction")
- Enter a location to save your project
- Ensure the 3D toggle is selected
- Select Create project
Congrats, you are all setup to get started with your mixed reality customizations now.
Chapter 2 - Setup the Camera
The Unity Main Camera handles head tracking and stereoscopic rendering. There are a few changes to make to the Main Camera to use it with mixed reality.
- Select File > New Scene
First, it will be easier to lay out your app if you imagine the starting position of the user as (X: 0, Y: 0, Z: 0). Since the Main Camera is tracking movement of the user's head, the starting position of the user can be set by setting the starting position of the Main Camera.
- Select Main Camera in the Hierarchy panel
- In the Inspector panel, find the Transform component and change the Position from (X: 0, Y: 1, Z: -10) to (X: 0, Y: 0, Z: 0)
Second, the default Camera background needs some thought.
For HoloLens applications, the real world should appear behind everything the camera renders, not a Skybox texture.
- With the Main Camera still selected in the Hierarchy panel, find the Camera component in the Inspector panel and change the Clear Flags dropdown from Skybox to Solid Color.
- Select the Background color picker and change the RGBA values to (0, 0, 0, 0)
For mixed reality applications targeted to immersive headsets, we can use the default Skybox texture that Unity provides.
- With the Main Camera still selected in the Hierarchy panel, find the Camera component in the Inspector panel and keep the Clear Flags dropdown to Skybox.
Third, let us consider the near clip plane in Unity and prevent objects from being rendered too close to the users eyes as a user approaches an object or an object approaches a user.
For HoloLens applications, the near clip plane can be set to the HoloLens recommended 0.85 meters.
- With the Main Camera still selected in the Hierarchy panel, find the Camera component in the Inspector panel and change the Near Clip Plane field from the default 0.3 to the HoloLens recommended 0.85.
For mixed reality applications targeted to immersive headsets, we can use the default setting that Unity provides.
- With the Main Camera still selected in the Hierarchy panel, find the Camera component in the Inspector panel and keep the Near Clip Plane field to the default 0.3.
Finally, let us save our progress so far. To save the scene changes, select File > Save Scene As, name the scene Main, and select Save.
Chapter 3 - Setup the Project Settings
In this chapter, we will set some Unity project settings that help us target the Windows Holographic SDK for development. We will also set some quality settings for our application. Finally, we will ensure our build targets are set to Windows Store.
Unity performance and quality settings
Unity quality settings for HoloLens
Since maintaining high framerate on HoloLens is so important, we want the quality settings tuned for fastest performance. For more detailed performance information, Performance recommendations for Unity
- Select Edit > Project Settings > Quality
- Select the dropdown under the Windows Store logo and select Fastest. You'll know the setting is applied correctly when the box in the Windows Store column and Fastest row is green.
For mixed reality applications targeted to occluded displays, you can leave the quality settings to its default values.
Target Windows 10 SDK
Target Windows Holographic SDK
We need to let Unity know that the app we are trying to export should create a holographic view instead of a 2D view. We do this by enabling Virtual Reality support on Unity targeting the Windows 10 SDK.
- Select Edit > Project Settings > Player
- In the Inspector panel click on the Windows Store tab
- Expand the Other Settings group.
- In the Rendering section, check the Virtual Reality Supported checkbox to add a new Virtual Reality SDKs list and confirm "Windows Mixed Reality" is listed as a supported SDK.
Awesome job on getting all the project settings applied. Next, let us add a hologram!
Chapter 4 - Create a cube
Creating a cube in your Unity project is just like creating any other object in Unity. Placing a cube in front of the user is easy because Unity's coordinate system is mapped to the real world - where one meter in Unity is approximately one meter in the real world.
- In the top left corner of the Hierarchy panel, select the Create dropdown and choose 3D Object > Cube.
- Select the newly created Cube in the Hierarchy panel
- In the Inspector find the Transform component and change Position to (X: 0, Y: 0, Z: 2). This positions the cube 2 meters in front of the user's starting position.
- In the Transform component, change Rotation to (X: 45, Y: 45, Z: 45) and change Scale to (X: 0.25, Y: 0.25, Z: 0.25). This scales the cube to 0.25 meters.
- To save the scene changes, select File > Save Scene.
Chapter 5 - Verify on device from Unity editor
Now that we have created our cube, it is time to do a quick check in device. You can do this directly from within the Unity editor.
For HoloLens use Unity Remoting
- On your HoloLens, install and run the Holographic Remoting Player, available from the Windows Store. Launch the application on the device, and it will enter a waiting state and show the IP address of the device. Note down the IP.
- On your development PC, in Unity, open Window > Holographic Emulation.
- Change Emulation Mode from None to Remote to Device.
- In Remote Machine, enter the IP address of your HoloLens noted earlier.
- Click Connect.
- Ensure the Connection Status changes to green Connected.
- Now you can now click Play in the Unity editor.
You will now be able to see the cube in device and in the editor. You can pause, inspect objects, and debug just like you are running an app in the editor, because that’s essentially what’s happening, but with video, audio, and device input transmitted back and forth across the network between the host machine and the device.
For other mixed reality supported headsets
- Connect the headset to your development PC using the USB cable and the HDMI or display port cable.
- Launch the Mixed Reality Portal and ensure you have completed the first run experience.
- From Unity, you can now press the Play button.
You will now be able to see the cube rendering in your mixed reality headset and in the editor.
Chapter 6 - Build and deploy to device from Visual Studio
We are now ready to compile our project to Visual Studio and deploy to our target device.
Export to the Visual Studio solution
- Open File > Build Settings window.
- Click Add Open Scenes to add the scene.
- Change Platform to Windows Store and click Switch Platform.
- In Windows Store settings ensure, SDK is Universal 10.
- For Target device, leave to Any Device for occluded displays or switch to HoloLens.
- UWP Build Type should be D3D.
- UWP SDK could be left at Latest installed.
- Check Unity C# Projects under Debugging.
- Click Build.
- In the file explorer, click New Folder and name the folder "App".
- With the App folder selected, click the Select Folder button.
- When Unity is done building, a Windows File Explorer window will appear.
- Open the App folder in file explorer.
- Open the generated Visual Studio solution (MixedRealityIntroduction.sln in this example)
Compile the Visual Studio solution
Finally, we will compile the exported Visual Studio solution, deploy it, and try it out on the device.
- Using the top toolbar in Visual Studio, change the target from Debug to Release and from ARM to X86.
The instructions differ for deploying to a device versus the emulator. Follow the instructions that match your setup.
Deploy to mixed reality device over Wi-Fi
- Click on the arrow next to the Local Machine button, and change the deployment target to Remote Machine.
- Enter the IP address of your mixed reality device and change Authentication Mode to Universal (Unencrypted Protocol) for HoloLens and Windows for other devices.
- Click Debug > Start without debugging.
For HoloLens, If this is the first time deploying to your device, you will need to pair Using Visual Studio.
Deploy to mixed reality device over USB
Ensure you device is plugged in via the USB cable.
- For HoloLens, click on the arrow next to the Local Machine button, and change the deployment target to Device.
- For targeting occluded devices attached to your PC, keep the setting to Local Machine. Ensure you have the Mixed Reality Portal running.
- Click Debug > Start without debugging.
Deploy to Emulator
- Click on the arrow next to the Device button, and from drop down select HoloLens Emulator.
- Click Debug > Start without debugging.
Try out your app
Now that your app is deployed, try moving all around the cube and observe that it stays in the world in front of you.