Building 2D apps
Microsoft HoloLens enables a user to see holograms as if they are right there in your world. At its core, the HoloLens is a Windows 10 device; this means that the HoloLens is able to run almost all of the Universal Windows Platform (UWP) apps in the Store as 2D apps.
Microsoft has been rapidly evolving the Windows platform over the past few years. That means many developers have different starting points even if they deliver an app to the Windows 10 Store today on Desktop, Mobile, or Xbox. This guide will focus on helping you get started when you have an existing app that you are trying to bring to HoloLens no matter where you're starting from.
To build an app for HoloLens, you must target the Universal Windows Platform - the developer platform introduced in Windows 10. That means to bring your app to HoloLens, you must ensure it targets Windows.Universal. We'll talk about ways that you can restrict your app to the HoloLens device using Windows.Holographic below. Here are all the potential starting points you may have with your app today:
Change your Target to Windows.Universal
Congratulations! Your app is now using the Windows 10 Universal Windows Platform (UWP). Your app is now capable of running on todays Windows devices like Desktop, Mobile, Xbox, and HoloLens as well as future Windows devices.
Now let's jump into your AppX manifest to ensure your Windows 10 UWP app can run on HoloLens.
- Open your app's solution file with Visual Studio and navigate to the app package manifest
- Right click the Package.appxmanifest file in your Solution and go to View Code
- Ensure your Target Platform is Windows.Universal in the dependencies section
<TargetDeviceFamily Name="Windows.Universal" MinVersion="10.0.10240.0" MaxVersionTested="10.0.10586.0" />
If you do not use Visual Studio for your development environment, you can open AppXManifest.xml in the text editor of your choice to ensure you're targeting the Windows.Universal TargetDeviceFamily.
Run in the HoloLens Emulator
Now that your UWP app targets Windows Universal, lets build your app and run it in the HoloLens Emulator.
- Select HoloLens Emulator in the deployment target drop-down menu
- Select Debug > Start Debugging to deploy your app and start debugging.
- The emulator will start and run your app.
- With a keyboard, mouse, and/or an Xbox controller, place your app in the world to launch it.
At this point, one of two things can happen:
- Your app will show its splash and start running after it is placed in the Emulator! Awesome!
- Or after you see a loading animation for a 2D hologram, loading will stop and you will just see your app at its splash screen. This means that something went wrong and it will take more investigation to understand how to bring your app to life on HoloLens.
How to debug
HoloLens Development Edition is a new device target of the Windows 10 operating system, so there are Universal Windows Platform APIs that are still undergoing testing and development. We've experienced our own challenges bringing Microsoft UWP apps to HoloLens.
Here are some high level areas that we've found to be a problem:
- Querying for File System paths not supported on startup.
- Using legacy authentication methods outside of Web Authentication Broker or Web Account Manager.
- Making deep device hardware queries on startup (i.e. games).
- Using Calendar, People, Contact APIs on startup.
- Using large 3rd party libraries or services that may not be fully vetted on HoloLens.
To get to the bottom of what's causing your UWP app not to start on HoloLens, you'll have to debug.
Running your UWP app in the debugger
These steps will walk you through debugging your UWP app using the Visual Studio debugger.
- If you haven't already done so, open your solution in Visual Studio. Change the target to the HoloLens Emulator and the build configuration to x86.
- Select Debug > Start Debugging to deploy your app and start debugging.
- Place the app in the world with your mouse, keyboard, or Xbox controller.
- Visual Studio should now break somewhere in your app code.
- If your app doesn't immediately crash or break into the debugger because of an un-handled error, then go through a test pass of the core features of your app to make sure everything is running and functional. You may see errors like pictured below (internal exceptions that are being handled). To ensure you don't miss internal errors that impact the experience of your app, run through your automated tests and unit tests to make sure everything behaves as expected.
Understanding the failure
As mentioned above, there are known issues with APIs under testing and development for the HoloLens Development Edition. If you find that your app uses one of the APIs in the name spaces listed as having potential problems, use the Windows Feedback tool to send feedback to Microsoft.
How to open the Windows Feedback tool
- Bloom to see the Start menu
- Launch and place the Windows Feedback app.
- Select Developer Platform and send us the details of your problem.
We are continually fixing platform bugs in the APIs of UWP. For APIs that are failing by design - because they are not supported on HoloLens - here are the patterns that you can expect in your app and design around:
- Should not return a special failure HRESULT just because the API is partially-functional. Instead, APIs should signal failure via the design of the API itself by using empty collections, boolean return values, explicit status codes, etc. Note that if the API already returns HRESULTs due to programming errors (eg, passing invalid arguments) then it will continue to return those failures as appropriate.
- Should not return a null IVector[View], IMap[View], or Array from a property getter or method return. Instead, they will return a valid object that has zero items in it. If the type of the map or vector is mutable (ie, not a View) then allow the app to make changes even if they are not used anywhere.
- In some rare cases, APIs expose a read / write collection property that allows the app to provide a value of their own. In these cases, if the API already returns null in the fully-functional case, it should continue to return null in the partially-functional case. Note however that this is an API Design anti-pattern and should not generally be followed by most UWP APIs.
- Should not return a null IAsyncAction or IAsyncOperation from an Async method. Instead, they will return a valid object that is already in the completed state and has the appropriate result (empty collection, status code, etc.).
- Will not fail or ignore event registrations. Instead, API should accept the event registration / un-registration but simply never raise the event. The API must hold on to any registered event handlers (rather than silently ignoring them) since an app might inadvertently rely on the registration for lifetime management.
Update your UI
Now that your UWP app is running on HoloLens as a 2D hologram, next we'll make sure it looks beautiful. Here are some things to consider:
- HoloLens will run all 2D apps at a fixed resolution and DPI that equates to 853x480 effective pixels. Consider if your design needs refinement at this scale and think through our 2D UI guidance to improve your experience on HoloLens.
- HoloLens does not support 2D live tiles. If your core functionality is showing information on a live tile, consider moving that information back into your app.
- HoloLens does not support the Share contract, 2D Printing, full screen mode, casting, and other features that may be in your app.
New input possibilities
HoloLens uses advanced depth sensors to see the world and see users. This enables advanced gestures like bloom and air-tap. Powerful microphones also enable voice experiences. HoloLens takes care of all of this complexity for UWP apps, translating your gaze and gestures to pointer events that abstract away the input mechanism. For example, the HoloLens Clicker emulates the air tap gesture, but 2D applications don't need to know where the input came from.
Here are the high level concepts/scenarios you should understand for input when bringing your UWP app to HoloLens:
- Gaze turns into hover events, which can unexpectedly trigger menus, flyouts or other user interface elements to pop up just by gazing around your app.
- Gaze is not as precise as mouse input. Use appropriately sized hit targets for HoloLens, similar to touch-friendly mobile applications. Small elements near the edges of the app are especially hard to interact with.
- Users must switch input modes to go from scrolling to dragging to two finger panning. If your app was designed for touch input, consider ensuring that no major functionality is locked behind two finger panning. If so, consider having alternative input mechanisms like buttons that can initiate two finger panning. For example, the Maps app can zoom with two finger panning but has a plus, minus, and rotate button to simulate the same zoom interactions with single clicks.
Voice input is a critical part of the HoloLens experience. We've enabled all of the speech APIs that are in Windows 10 powering Cortana on HoloLens.
Publish and Maintain your Universal app on HoloLens
Once your app is up and running, package your app to submit it to the Universal Windows Store.