Holograms 240

Holograms are given presence in our world by remaining in place as we move about in space. HoloLens keeps holograms in place by using various coordinate systems to keep track of the location and orientation of objects. When we share these coordinate systems between devices, we can create a shared experience that allows us to take part in a shared holographic world.

In this tutorial, we will:

  • Setup a network for a shared experience.
  • Share holograms across HoloLens devices.
  • Discover other people in our shared holographic world.
  • Create a shared interactive experience where you can target other players - and launch projectiles at them!

Prerequisites

Project files

  • Download the files required by the project. Requires Unity 5.6.2 or later.
    • If you still need Unity 5.5 support, please use this release.
    • If you still need Unity 5.4 support, please use this release.
  • Unarchive the files to your desktop or other easy to reach location. Keep the folder name as SharedHolograms.

Chapter 1 - Holo World

In this chapter, we'll setup our first Unity project and step through the build and deploy process.

Objectives

  • Setup Unity to develop holographic apps.
  • See your hologram!

Instructions

  • Start Unity.
  • Select Open.
  • Enter location as the SharedHolograms folder you previously unarchived.
  • Select Project Name and click Select Folder.
  • In the Hierarchy, right-click the Main Camera and select Delete.
  • In the HoloToolkit-Sharing-240/Prefabs/Camera folder, find the Main Camera prefab.
  • Drag and drop the Main Camera into the Hierarchy.
  • In the Hierarchy, click on Create and Create Empty.
  • Right-click the new GameObject and select Rename.
  • Rename the GameObject to HologramCollection.
  • Select the HologramCollection object in the Hierarchy.
  • In the Inspector set the transform position to: X: 0, Y: -0.25, Z: 2.
  • In the Holograms folder in the Project panel, find the EnergyHub asset.
  • Drag and drop the EnergyHub object from the Project panel to the Hierarchy as a child of HologramCollection.
  • Select File > Save Scene As...
  • Name the scene SharedHolograms and click Save.
  • Press the Play button in Unity to preview your holograms.
  • Press Play a second time to stop preview mode.

Export the project from Unity to Visual Studio

  • In Unity select File > Build Settings.
  • Click Add Open Scenes to add the scene.
  • Select Windows Store in the Platform list and click Switch Platform.
  • Set SDK to Universal 10.
  • Set Target device to HoloLens and UWP Build Type to D3D.
  • Check Unity C# Projects.
  • Click Build.
  • In the file explorer window that appears, create a New Folder named "App".
  • Single click the App folder.
  • Press Select Folder.
  • When Unity is done, a File Explorer window will appear.
  • Open the App folder.
  • Open SharedHolograms.sln to launch Visual Studio.
  • Using the top toolbar in Visual Studio, change the target from Debug to Release and from ARM to X86.
  • Click on the drop-down arrow next to Local Machine, and select Remote Device.
    • Set the Address to the name or IP address of your HoloLens. If you do not know your device IP address, look in Settings > Network & Internet > Advanced Options or ask Cortana "Hey Cortana, What's my IP address?"
    • Leave the Authentication Mode set to Universal.
    • Click Select
  • Click Debug > Start Without debugging or press Ctrl + F5. If this is the first time deploying to your device, you will need to pair it with Visual Studio.
  • Put on your HoloLens and find the EnergyHub hologram.

Chapter 2 - Interaction

In this chapter, we'll interact with our holograms. First, we'll add a cursor to visualize our Gaze. Then, we'll add Gestures and use our hand to place our holograms in space.

Objectives

  • Use gaze input to control a cursor.
  • Use gesture input to interact with holograms.

Instructions

Gaze

  • In the Hierarchy panel select the HologramCollection object.
  • In the Inspector panel click the Add Component button.
  • In the menu, type in the search box Gaze Manager. Select the search result.
  • In the HoloToolkit-Sharing-240\Prefabs\Input folder, find the Cursor asset.
  • Drag and drop the Cursor asset onto the Hierarchy.

Gesture

  • In the Hierarchy panel select the HologramCollection object.
  • Click Add Component and type Gesture Manager in the search field. Select the search result.
  • In the Hierarchy panel, expand HologramCollection.
  • Select the child EnergyHub object.
  • In the Inspector panel click the Add Component button.
  • In the menu, type in the search box Hologram Placement. Select the search result.
  • Save the scene by selecting File > Save Scene.

Deploy and enjoy

  • Build and deploy to your HoloLens, using the instructions from the previous chapter.
  • Once the app launches on your HoloLens, move your head around and notice how the EnergyHub follows your gaze.
  • Notice how the cursor appears when you gaze upon the hologram, and changes to a point light when not gazing at a hologram.
  • Perform an air-tap to place the hologram. At this time in our project, you can only place the hologram once (redeploy to try again).

Chapter 3 - Shared Coordinates

It's fun to see and interact with holograms, but let's go further. We'll set up our first shared experience - a hologram everyone can see together.

Objectives

  • Setup a network for a shared experience.
  • Establish a common reference point.
  • Share coordinate systems across devices.
  • Everyone sees the same hologram!

Note

The InternetClientServer and PrivateNetworkClientServer capabilities must be declared for an app to connect to the sharing server. This is done for you already in Holograms 240, but keep this in mind for your own projects.

  1. In the Unity Editor, go to the player settings by navigating to "Edit > Project Settings > Player"
  2. Click on the "Windows Store" tab
  3. In the "Publishing Settings > Capabilities" section, check the InternetClientServer capability and the PrivateNetworkClientServer capability

Instructions

  • In the Project panel navigate to the HoloToolkit-Sharing-240\Prefabs\Sharing folder.
  • Drag and drop the Sharing prefab into the Hierarchy panel.

Next we need to launch the sharing service. Only one PC in the shared experience needs to do this step.

  • In Unity - in the top-hand menu - select the HoloToolkit-Sharing-240 menu.
  • Select the Launch Sharing Service item in the drop-down.
  • Check the Private Network option and click Allow Access when the firewall prompt appears.
  • Note down the IPv4 address displayed in the Sharing Service console window. This is the same IP as the machine the service is being run on.

Follow the rest of the instructions on all PCs that will join the shared experience.

  • In the Hierarchy, select the Sharing object.
  • In the Inspector, on the Sharing Stage component, change the Server Address from 'localhost' to the IPv4 address of the machine running SharingService.exe.
  • In the Hierarchy select the HologramCollection object.
  • In the Inspector click the Add Component button.
  • In the search box, type Import Export Anchor Manager. Select the search result.
  • In the Project panel navigate to the Scripts folder.
  • Double-click the HologramPlacement script to open it in Visual Studio.
  • Replace the contents with the code below.

HologramPlacement.cs[show]

  • Back in Unity, select the HologramCollection in the Hierarchy panel.
  • In the Inspector panel click the Add Component button.
  • In the menu, type in the search box App State Manager. Select the search result.

Deploy and enjoy

  • Build the project for your HoloLens devices.
  • Designate one HoloLens to deploy to first. You will need to wait for the Anchor to be uploaded to the service before you can place the EnergyHub (this can take ~30-60 seconds). Until the upload is done, your tap gestures will be ignored.
  • After the EnergyHub has been placed, its location will be uploaded to the service and you can then deploy to all other HoloLens devices.
  • When a new HoloLens first joins the session, the location of the EnergyHub may not be correct on that device. However, as soon as the anchor and EnergyHub locations have been downloaded from the service, the EnergyHub should jump to the new, shared location. If this does not happen within ~30-60 seconds, walk to where the original HoloLens was when setting the anchor to gather more environment clues. If the location still does not lock on, redeploy to the device.
  • When the devices are all ready and running the app, look for the EnergyHub. Can you all agree on the hologram's location and which direction the text is facing?

Chapter 4 - Discovery

Everyone can now see the same hologram! Now let's see everyone else connected to our shared holographic world. In this chapter, we'll grab the head location and rotation of all other HoloLens devices in the same sharing session.

Objectives

  • Discover each other in our shared experience.
  • Choose and share a player avatar.
  • Attach the player avatar next to everyone's heads.

Instructions

  • In the Project panel navigate to the Holograms folder.
  • Drag and drop the PlayerAvatarStore into the Hierarchy.
  • In the Project panel navigate to the Scripts folder.
  • Double-click the AvatarSelector script to open it in Visual Studio.
  • Replace the contents with the code below.

AvatarSelector.cs[show]

  • In the Hierarchy select the HologramCollection object.
  • In the Inspector click Add Component.
  • In the search box, type Local Player Manager. Select the search result.
  • In the Hierarchy select the HologramCollection object.
  • In the Inspector click Add Component.
  • In the search box, type Remote Player Manager. Select the search result.
  • Open the HologramPlacement script in Visual Studio.
  • Replace the contents with the code below.

HologramPlacement.cs[show]

  • Open the AppStateManager script in Visual Studio.
  • Replace the contents with the code below.

AppStateManager.cs[show]

Deploy and Enjoy

  • Build and deploy the project to your HoloLens devices.
  • When you hear a pinging sound, find the avatar selection menu and select an avatar with the air-tap gesture.
  • If you're not looking at any holograms, the point light around your cursor will turn a different color when your HoloLens is communicating with the service: initializing (dark purple), downloading the anchor (green), importing/exporting location data (yellow), uploading the anchor (blue). If your point light around your cursor is the default color (light purple), then you are ready to interact with other players in your session!
  • Look at other people connected to your space - there will be a holographic robot floating above their shoulder and mimicking their head motions!

Chapter 5 - Placement

In this chapter, we'll make the anchor able to be placed on real-world surfaces. We'll use shared coordinates to place that anchor in the middle point between everyone connected to the shared experience.

Objectives

  • Place holograms on the spatial map based on players’ head position.

Instructions

  • In the Project panel navigate to the Holograms folder.
  • Drag and drop the CustomSpatialMapping prefab onto the Hierarchy.
  • In the Project panel navigate to the Scripts folder.
  • Double-click the AppStateManager script to open it in Visual Studio.
  • Replace the contents with the code below.

AppStateManager.cs[show]

  • In the Project panel navigate to the Scripts folder.
  • Double-click the HologramPlacement script to open it in Visual Studio.
  • Replace the contents with the code below.

HologramPlacement.cs[show]

Deploy and enjoy

  • Build and deploy the project to your HoloLens devices.
  • When the app is ready, stand in a circle and notice how the EnergyHub appears in the center of everyone.
  • Tap to place the EnergyHub.
  • Try the voice command 'Reset Target' to pick the EnergyHub back up and work together as a group to move the hologram to a new location.

Chapter 6 - Real-World Physics

In this chapter we'll add holograms that bounce off real-world surfaces. Watch your space fill up with projects launched by both you and your friends!

Objectives

  • Launch projectiles that bounce off real-world surfaces.
  • Share the projectiles so other players can see them.

Instructions

  • In the Hierarchy select the HologramCollection object.
  • In the Inspector click Add Component.
  • In the search box, type Projectile Launcher. Select the search result.

Deploy and enjoy

  • Build and deploy to your HoloLens devices.
  • When the app is running on all devices, perform an air-tap to launch projectile at real world surfaces.
  • See what happens when your projectile collides with another player's avatar!

Chapter 7 - Grand Finale

In this chapter, we'll uncover a portal that can only be discovered with collaboration.

Objectives

  • Work together to launch enough projectiles at the anchor to uncover a secret portal!

Instructions

  • In the Project panel navigate to the Holograms folder.
  • Drag and drop the Underworld asset as a child of HologramCollection.
  • With HologramCollection selected, click the Add Component button in the Inspector.
  • In the menu, type in the search box ExplodeTarget. Select the search result.
  • With HologramCollection selected, from the Hierarchy drag the EnergyHub object to the Target field in the Inspector.
  • With HologramCollection selected, from the Hierarchy drag the Underworld object to the Underworld field in the Inspector.

Deploy and enjoy

  • Build and deploy to your HoloLens devices.
  • When the app has launched, collaborate together to launch projectiles at the EnergyHub.
  • When the underworld appears, launch projectiles at underworld robots (hit a robot three times for extra fun).