Holograms 101

This tutorial will walk you through a complete project, built in Unity, that demonstrates core Windows Holographic Platform and HoloLens features including gaze, gestures, voice input, spatial sound and spatial mapping.

The tutorial will take approximately 1 hour to complete.

Prerequisites

Project files

  • Download the files required by the project.
  • Unarchive the files to your desktop or other easy to reach location. Keep the folder name as Origami.

Chapter 1 - "Holo" world

In this chapter, we'll setup our first Unity project and step through the build and deploy process.

Objectives

  • Set up Unity for holographic development.
  • Make a hologram.
  • See a hologram that you made.

Instructions

  • Start Unity.
  • Select Open.
  • Enter location as the Origami folder you previously unarchived.
  • Select Origami and click Select Folder.
  • Since the Origami project does not contain a scene, save the empty default scene to a new file using: File / Save Scene As.
  • Name the new scene Origami and press the Save button.

Setup the main virtual camera

  • In the Hierarchy Panel, select Main Camera.
  • In the Inspector set its transform position to 0,0,0.
  • Find the Clear Flags property, and change the dropdown from Skybox to Solid color.
  • Click on the Background field to open a color picker.
  • Set R, G, B, and A to 0.

Setup the scene

  • In the Hierarchy Panel, click on Create and Create Empty.
  • Right-click the new GameObject and select Rename. Rename the GameObject to OrigamiCollection.
  • From the Holograms folder in the Project Panel (expand Assets and select Holograms or double click the Holograms folder in the Project Panel):
    • Drag Stage into the Hierarchy to be a child of OrigamiCollection.
    • Drag Sphere1 into the Hierarchy to be a child of OrigamiCollection.
    • Drag Sphere2 into the Hierarchy to be a child of OrigamiCollection.
  • Right-click the Directional Light object in the Hierarchy Panel and select Delete.
  • From the Holograms folder, drag Lights into the root of the Hierarchy Panel.
  • In the Hierarchy, select the OrigamiCollection.
  • In the Inspector, set the transform position to 0, -0.5, 2.0.
  • Press the Play button in Unity to preview your holograms.
  • You should see the Origami objects in the preview window.
  • Press Play a second time to stop preview mode.

Export the project from Unity to Visual Studio

  • In Unity select File > Build Settings.
  • Select Windows Store in the Platform list and click Switch Platform.
  • Set SDK to Universal 10 and Build Type to D3D.
  • Check Unity C# Projects.
  • Click Add Open Scenes to add the scene.
  • Click Build.
  • In the file explorer window that appears, create a New Folder named "App".
  • Single click the App Folder.
  • Press Select Folder.
  • When Unity is done, a File Explorer window will appear.
  • Open the App folder.
  • Open (double click) Origami.sln.
  • Using the top toolbar in Visual Studio, change the target from Debug to Release and from ARM to X86.
  • Click on the arrow next to the Device button, and select Remote Device.
    • Set the Address to the name or IP address of your HoloLens. If you do not know your device IP address, look in Settings > Network & Internet > Advanced Options or ask Cortana "Hey Cortana, What's my IP address?"
    • Leave the Authentication Mode set to Universal.
    • Click Select
  • Click Debug > Start Without debugging or press Ctrl + F5. If this is the first time deploying to your device, you will need to pair it with Visual Studio.
  • The Origami project will now build, deploy to your HoloLens, and then run.
  • Put on your HoloLens and look around to see your new holograms.

Chapter 2 - Gaze

In this chapter, we are going to introduce the first of three ways of interacting with your holograms -- gaze.

Objectives

  • Visualize your gaze using a world-locked cursor.

Instructions

  • Go back to your Unity project, and close the Build Settings window if it's still open.
  • Select the Holograms folder in the Project panel.
  • Drag the Cursor object into the Hierarchy panel at the root level.
  • Double-click on the Cursor object to take a closer look at it.
  • Right-click on the Scripts folder in the Project panel.
  • Click the Create sub-menu.
  • Select C# Script.
  • Name the script WorldCursor. Note: The name is case-sensitive. You do not need to add the .cs extension.
  • Select the Cursor object in the Hierarchy panel.
  • Drag and drop the WorldCursor script into the Inspector panel.
  • Double-click the WorldCursor script to open it in Visual Studio.
  • Copy and paste this code into WorldCursor.cs and Save All.

WorldCursor.cs[show]

  • Rebuild the app from File > Build Settings.
  • Return to the Visual Studio solution previously used to deploy to your HoloLens.
  • Select 'Reload All' when prompted.
  • Click Debug -> Start Without debugging or press Ctrl + F5.
  • Now look around the scene and notice how the cursor interacts with the shape of objects.

Chapter 3 - Gestures

In this chapter, we'll add support for gestures. When the user selects a paper sphere, we'll make the sphere fall by turning on gravity using Unity's physics engine.

Objectives

  • Control your holograms with the Select gesture.

Instructions

We'll start by creating a script then can detect the Select gesture.

  • In the Scripts folder, create a script named GazeGestureManager.
  • Drag the GazeGestureManager script onto the OrigamiCollection object in the Hierarchy.
  • Open the GazeGestureManager script in Visual Studio and add the following code:

GazeGestureManager.cs[show]

  • Create another script in the Scripts folder, this time named SphereCommands.
  • Expand the OrigamiCollection object in the Hierarchy view.
  • Drag the SphereCommands script onto the Sphere1 object in the Hierarchy panel.
  • Drag the SphereCommands script onto the Sphere2 object in the Hierarchy panel.
  • Open the script in Visual Studio for editing, and replace the default code with this:

SphereCommands.cs[show]

  • Export, build and deploy the app to your HoloLens.
  • Look at one of the spheres.
  • Perform the select gesture and watch the sphere drop onto the surface below.

Chapter 4 - Voice

In this chapter, we'll add support for two voice commands: "Reset world" to return the dropped spheres to their original location, and "Drop sphere" to make the sphere fall.

Objectives

  • Add voice commands that always listen in the background.
  • Create a hologram that reacts to a voice command.

Instructions

  • In the Scripts folder, create a script named SpeechManager.
  • Drag the SpeechManager script onto the OrigamiCollection object in the Hierarchy
  • Open the SpeechManager script in Visual Studio.
  • Copy and paste this code into SpeechManager.cs and Save All:

SpeechManager.cs[show]

  • Open the SphereCommands script in Visual Studio.
  • Update the script to read as follows:

SphereCommands.cs[show]

  • Export, build and deploy the app to your HoloLens.
  • Look at one of the spheres, and say "Drop Sphere".
  • Say "Reset World" to bring them back to their initial positions.

Chapter 5 - Spatial sound

In this chapter, we'll add music to the app, and then trigger sound effects on certain actions. We'll be using spatial sound to give sounds a specific location in 3D space.

Objectives

  • Hear holograms in your world.

Instructions

  • In Unity select from the top menu Edit > Project Settings > Audio
  • In the Inspector Panel on the right side, find the Spatializer Plugin setting and select MS HRTF Spatializer.
  • From the Holograms folder in the Project panel, drag the Ambience object onto the OrigamiCollection object in the Hierarchy Panel.
  • Select OrigamiCollection and find the Audio Source component in the Inspector panel. Change these properties:
    • Check the Spatialize property.
    • Check the Play On Awake.
    • Change Spatial Blend to 3D by dragging the slider all the way to the right. The value should change from 0 to 1 when you move the slider.
    • Check the Loop property.
    • Expand 3D Sound Settings, and enter 0.1 for Doppler Level.
    • Set Volume Rolloff to Custom Rolloff.
  • In the Scripts folder, create a script named SphereSounds.
  • Drag and drop SphereSounds to the Sphere1 and Sphere2 objects in the Hierarchy.
  • Open SphereSounds in Visual Studio, update the following code and Save All.

SphereSounds.cs[show]

  • Save the script, and return to Unity.
  • Export, build and deploy the app to your HoloLens.
  • Move closer and further from the Stage and turn side-to-side to hear the sounds change.

Chapter 6 - Spatial mapping

Now we are going to use spatial mapping to place the game board on a real object in the real world.

Objectives

  • Bring your real world into the virtual world.
  • Place your holograms where they matter most to you.

Instructions

  • In Unity, click on the Holograms folder in the Project panel.
  • Drag the Spatial Mapping asset into the root of the Hierarchy.
  • Click on the Spatial Mapping object in the Hierarchy.
  • In the Inspector panel, change the following properties:
    • Check the Draw Visual Meshes box.
    • Locate Draw Material and click the circle on the right. Type "wireframe" into the search field at the top. Click on the result and then close the window. When you do this, the value for Draw Material should get set to Wireframe.
  • Export, build and deploy the app to your HoloLens.
  • When the app runs, a wireframe mesh will overlay your real world.
  • Watch how a rolling sphere will fall off the stage, and onto the floor!

Now we'll show you how to move the OrigamiCollection to a new location:

  • In the Scripts folder, create a script named TapToPlaceParent.
  • In the Hierarchy, expand the OrigamiCollection and select the Stage object.
  • Drag the TapToPlaceParent script onto the Stage object.
  • Open the TapToPlaceParent script in Visual Studio, and update it to be the following:

TapToPlaceParent.cs[show]

  • Export, build and deploy the app.
  • Now you should now be able to place the game in a specific location by gazing at it, using the Select gesture and then moving to a new location, and using the Select gesture again.

Chapter 7 - Holographic fun

Objectives

  • Reveal the entrance to a holographic underworld.

Instructions

Now we'll show you how to uncover the holographic underworld:

  • From the Holograms folder in the Project Panel:
    • Drag Underworld into the Hierarchy to be a child of OrigamiCollection.
  • In the Scripts folder, create a script named HitTarget.
  • In the Hierarchy, expand the OrigamiCollection.
  • Expand the Stage object and select the Target object (blue fan).
  • Drag the HitTarget script onto the Target object.
  • Open the HitTarget script in Visual Studio, and update it to be the following:

HitTarget.cs[show]

  • In Unity, select the Target object.
  • Two public properties are now visible on the Hit Target component and need to reference objects in our scene:
    • Drag Underworld from the Hierarchy panel to the Underworld property on the Hit Target component.
    • Drag Stage from the Hierarchy panel to the Object to Hide property on the Hit Target component.
  • Export, build and deploy the app.
  • Place the Origami Collection on the floor, and then use the Select gesture to make a sphere drop.
  • When the sphere hits the target (blue fan), an explosion will occur. The collection will be hidden and a hole to the underworld will appear.

The end

And that's the end of this tutorial!

You learned:

  • How to create a holographic app in Unity.
  • How to make use of gaze, gesture, voice, sound, and spatial mapping.
  • How to build and deploy an app using Visual Studio.

You are now ready to start creating your own holographic experience!