Keyboard input in DirectX

Keyboard input can be a useful alternate form of input for Windows Holographic. Bluetooth keyboards and mice are both supported as alternate forms of input, and for debugging your app. Your Store app must continue to work with full functionality without these devices attached.

Subscribe for CoreWindow input events

Keyboard input

In the Windows Holographic app template, we include an event handler just like the one you would use for UWP apps. If you hook up a Bluetooth keyboard to your HoloLens, your app can consume the data in the same way.

From AppView.cpp:

// Register for keypress notifications.
   window->KeyDown +=
       ref new TypedEventHandler<CoreWindow^, KeyEventArgs^>(this, &AppView::OnKeyPressed);

// Input event handlers

   void AppView::OnKeyPressed(CoreWindow^ sender, KeyEventArgs^ args)
       // TODO: Bluetooth keyboards are supported by HoloLens. You can use this method for
       //       keyboard input if you want to support it as an optional input method for
       //       your holographic app.

Mouse Input

You can also attach a mouse and use it in the same way, using standard UWP CoreWindow APIs. Copy this code into your template-based app to get up and running with Bluetooth mouse input more quickly. (Don’t forget to update the header, too!)

For AppView.cpp:

// Register for (non-spatial) pointer notifications.
   window->PointerMoved +=
       ref new TypedEventHandler<CoreWindow^, PointerEventArgs^>(this, &AppView::OnPointerMoved);
   window->PointerPressed +=
       ref new TypedEventHandler<CoreWindow^, PointerEventArgs^>(this, &AppView::OnPointerPressed);

Void AppView::OnPointerMoved(Windows::UI::Core::CoreWindow ^ sender, Windows::UI::Core::PointerEventArgs ^ args)
       // TODO: Process bluetooth mouse input here, as a secondary input source.

   void HolographicDesktopApp::AppView::OnPointerPressed(Windows::UI::Core::CoreWindow ^ sender, Windows::UI::Core::PointerEventArgs ^ args)
       // TODO: Process bluetooth mouse input here, as a secondary input source.

Important guidelines for keyboard and mouse input

There are some key differences in how this code can be used on the Microsoft HoloLens – which is a device that primarily relies upon natural user input – versus what we are used to implementing for desktop PCs, laptops, and tablets.

  • You can’t rely on keyboard or mouse input to be present. All of your app's functionality must work with gaze, gesture, and speech input.
  • When a Bluetooth keyboard is attached, it can be helpful to enable keyboard input for any text that your app might ask for. This can be a great supplement for dictation, for example.
  • When it comes to designing your app, don’t rely on (for example) WASD and mouse look controls for your game. HoloLens is designed for the user to walk around the room. In this case, the user controls the camera directly. An interface for driving the camera around the room with move/look controls won't provide the same experience.
  • Keyboard input can be an excellent way to control the debugging aspects of your app or game engine, especially since the user will not be required to use the keyboard. Wiring it up is the same as you're used to, with CoreWindow event APIs. In this scenario, you might choose to implement a way to configure your app to route keyboard events to a "debug input only" mode during your debug sessions.
  • Bluetooth controllers work as well.

See also