Motion controllers are hardware accessories that allow users to take action in mixed reality. An advantage of motion controllers over gestures is that the controllers have a precise position in space, allowing for fine grained interaction with digital objects. For Windows Mixed Reality immersive headsets, motion controllers are the primary way that users will take action in their world.
Windows Mixed Reality motion controllers offer precise and responsive tracking of movement in your field of view using the sensors in the immersive headset, meaning there is no need to install hardware on the walls in your space. These motion controllers will offer the same ease of setup and portability as Windows Mixed Reality immersive headsets. Our device partners plan to market and sell these controllers on retail shelves this holiday.
Before you begin
You will need:
A set of two motion controllers.
Four AA batteries.
A PC capable of Bluetooth 4.0.
Update Windows and Unity for motion controller support:
Insert 2 AA batteries into the back of the controller. Leave the battery cover off for now.
Press and hold the controller's Windows button to turn on the controller, release once it buzzes.
Press and hold the pairing button (tab in the battery compartment) until the LEDs begin pulsing. Reattach battery cover.
Open Windows Settings -> Bluetooth & other devices -> Add Bluetooth or other device -> Bluetooth and wait for "Motion controller - Left" or "Motion controller - Right" to appear. Select to pair. Controller will vibrate once when connected.
Repeat steps 1-4 for the second controller.
Updating controller firmware
If an immersive headset is connected to your PC, and new controller firmware is available, the firmware will be pushed to your motion controllers automatically the next time they're turned on. Controller firmware updates are indicated by a pattern of illuminating LED quadrants in a circular motion, and take 1-2 minutes.
After the firmware update completes, the controllers will reboot and reconnect.
Important! For current driver versions, after the firmware update completes, unpair and re-pair your motion controllers via Bluetooth for the best experience:
Go to Windows Settings -> Devices -> Bluetooth -> Bluetooth & other devices tab
Remove earlier paired controllers
Unpair your controllers
Clear the pairing cache by holding the pairing button (under the battery cover) until the LEDs start to pulse (pairing mode indicator). Once in pairing mode, press and hold the Windows button on each controller to power it off (or let pairing time out after ~60 seconds). This deletes the pairing information on your controllers and will help with a clean update.
Motion Controllers use the same Bluetooth technology found in many consumer devices. Motion Controllers are designed to work with Bluetooth capability included in any recent PC. If your PC doesn’t have Bluetooth, we recommend you purchase the Plugable dongle from Amazon and plug it into a USB 2.0 port.(Amazon link)
Motion Controllers should work with other Bluetooth keyboards, mice and game controllers, but the experience will vary depending on the model of keyboard, mouse or game controller you use.
Motion Controllers do not work well with Bluetooth headphones at the same time. It is best to turn off your Bluetooth headphones or speakers when using Motion Controllers. You can use the headphone jack or built-in speakers on your Mixed Reality headset for the best audio experience.
If you run into problems with Motion Controllers, we recommend you remove / un-pair some of your other Bluetooth devices:
Game controllers, including the Xbox game controller
Keyboards, mice, pens and other input devices
If you are still having problems with the Motion Controllers, we recommend you:
Make sure your external Bluetooth radio is plugged into a USB 2.0 port
Move your external Bluetooth radio away from other wireless transmitters, such as those for keyboards, a mouse or headphones.
Move your external Bluetooth radio away from other USB 3.0 devices such as USB flash drives or your Mixed Reality headset. Your Bluetooth radio should be plugged into a USB port as far away from other USB 3.0 devices as possible.
Close the Windows Settings app if currently displaying Bluetooth & other devices page. Windows Settings will continue to search for new Bluetooth devices when the Bluetooth & other devices page is open.
Ensure your PC is connected to a 5Ghz WiFi network.
If your computer has Bluetooth already, but you are still having problems with the Motion Controllers, consider replacing your Bluetooth radio with an external Bluetooth radio plugged into USB. Note: you’ll need to disable your existing Bluetooth radio in Device Manager and un-pair / re-pair all of your devices.
Switching between Bluetooth and USB
We recommend developers use motion controllers via Bluetooth. However, if you must use USB in order to unblock a critical bug, please be sure to follow these exact steps below on any switch from USB to Bluetooth or Bluetooth to USB.
To avoid controllers enumerating both as Bluetooth and USB devices, Bluetooth pairing information has to be removed before using controllers over USB:
Go to Windows Settings > Devices > Bluetooth and remove both motion controllers.
Put batteries in each controller, and power them on by holding the Windows button for 2 seconds (or until controller powers on).
Hold the pairing button until the LEDs start to pulse (pairing mode indicator).
Once in pairing mode, press and hold the power button to power off each controller. This deletes the pairing information on your controllers.
Take out the batteries and replace with the USB battery back.
If the controller isn’t working correctly and you’re unable to update the device, you can always restore the device to factory conditions. To recover your device:
Unplug and power off the controllers.
Open the battery cover.
Insert batteries (ensure you have good batteries when performing device recovery).
Press and hold pairing button (tab at the bottom under the batteries).
While holding pairing button, power on the controller by pressing and holding the Windows button for 5 seconds (keep both buttons held).
Release buttons and wait for controller to power on. This takes up to 15 seconds and there are no indicators when device recovery is happening. If device powers on immediately on button release, recovery button sequence did not get registered and you need to try again.
Remove old controller associations from Bluetooth settings (Settings->Bluetooth & other devices, select "Motion controller - Left" or "Motion controller - Right" and Remove device. After that pair controller with PC again.
After connecting with host and HMD, device will update firmware to latest available.
After firmware has been updated you need to unpair and pair controller again to get it working properly.
Lights and indicators
Windows button press
LED goes on/off
Device motionless 30 seconds and battery powered
Slow blink (every 3 seconds)
Pairing button press
Bluetooth connected or disconnected
No LED change
During power on when battery level is "Low"
During power on when battery level is "Critical" and shuts off
There are two key models for interaction, gaze and commit and point and commit:
With gaze and commit, users target an object with their gaze and then use select interactions.
With point and commit, a user can aim a pointing-capable motion controller at the target object and use select interactions.
Apps that support motion controllers should enable both gaze-driven and pointing-driven interactions to give users choice in what input device they use.
Grip pose vs. pointing pose
Windows Mixed Reality supports motion controllers in a variety of form factors, with each controller's design differing in its relationship between the user's hand position and the natural "forward" direction that apps should use for pointing when rendering the controller.
To better represent these controllers, there are two kinds of poses you can investigate for each interaction source:
The grip pose, representing the location of either the palm of a hand detected by a HoloLens, or the palm holding a motion controller.
On immersive headsets, this pose is best used to render the user's hand or an object held in the user's hand, such as a sword or gun.
The grip position: The palm centroid when holding the controller naturally, adjusted left or right to center the position within the grip.
The grip orientation's Right axis: When you completely open your hand to form a flat 5-finger pose, the ray that is normal to your palm (forward from left palm, backward from right palm)
The grip orientation's Forward axis: When you close your hand partially (as if holding the controller), the ray that points "forward" through the tube formed by your non-thumb fingers.
The grip orientation's Up axis: The Up axis implied by the Right and Forward definitions.
The pointer pose, representing the tip of the controller pointing forward.
This pose is best used to raycast when pointing at UI when you are rendering the controller model itself.
Interactions: Low-level spatial input
The core interactions across hands and motion controllers are Select, Menu, Grasp, Touchpad, Thumbstick, and Home.
Select is the primary interaction to activate a hologram, consisting of a press followed by a release. For motion controllers, you perform a Select press using the controller's trigger. Other ways to perform a Select are by speaking the voice command "Select". The same select interaction can be used within any app. Think of Select as the equivalent of a mouse click, a universal action that you learn once and then apply across all your apps.
Menu is the secondary interaction for acting on an object, used to pull up a context menu or take some other secondary action. With motion controllers, you can take a menu action using the controller's menu button. (i.e. the button with the hamburger "menu" icon on it)
Grasp is how users can directly take action on objects at their hand to manipulate them. With motion controllers, you can do a grasp action by squeezing your fist tightly. A motion controller may detect a Grasp with a grab button, palm trigger or other sensor.
Touchpad allows the user to adjust an action in two dimensions along the surface of a motion controller's touchpad, committing the action by clicking down on the touchpad. Touchpads provide a pressed state, touched state and normalized XY coordinates. X and Y range from -1 to 1 across the range of the circular touchpad, with a center at (0, 0). For X, -1 is on the left and 1 is on the right. For Y, -1 is on the bottom and 1 is on the top.
Thumbstick allows the user to adjust an action in two dimensions by moving a motion controller's thumbstick within its circular range, committing the action by clicking down on the thumbstick. Thumbsticks also provide a pressed state and normalized XY coordinates. X and Y range from -1 to 1 across the range of the circular touchpad, with a center at (0, 0). For X, -1 is on the left and 1 is on the right. For Y, -1 is on the bottom and 1 is on the top.
Home is a special system action that is used to go back to the Start Menu. It is similar to pressing the Windows key on a keyboard or the Xbox button on an Xbox controller. You can go home by pressing the Windows button on a motion controller. Note, you can also always return to Start by saying "Hey Cortana, Go Home". Apps cannot react specifically to home actions, as these are handled by the system.
Composite gestures: High-level spatial input
Both hand gestures and motion controllers can be tracked over time to detect a common set of high-level composite gestures. This enables your app to detect high-level tap, hold, manipulation and navigation gestures, whether users end up using hands or controllers.