Room scan UX

Applications that require spatial mapping data rely on the device to automatically collect this data over time and across sessions as the user explores their environment with the device active. The completeness and quality of this data depends on a number of factors including the amount of exploration the user has done, how much time has passed since the exploration and whether objects such as furniture and doors have moved since the device scanned the area. To ensure useful spatial mapping data, applications developers have several options:

  • Rely on what may have already been collected. This data may be incomplete initially.
  • Ask the user to bloom to the Shell and explore the area the users wishes to use for the experience. In this way they can use the familiar Shell feature of air tapping to confirm that all the necessary area is known to the device.
  • Build a custom exploration experience in their own application.

Note that in all these cases the actual data gathered during the exploration is stored by the system and the application does not need to do this.

Building a custom scanning experience

Applications may decide to analyze the spatial mapping data at the start of the experience to judge whether they want the user to perform additional steps to improve its completeness and quality. If analysis indicates quality should be improved, developers should provide a visualization to overlay on the world to indicate:

  • How much of the total volume in the users vicinity needs to be part of the experience
  • Where the user should go to improve data

Users do not know what makes a scan "good". They need to be shown or told what to look for if they’re asked to evaluate a scan – flatness, distance from actual walls, etc. The developer should implement a feedback loop that includes refreshing the spatial mapping data during the scanning or exploration phase.

In many cases, it may be best to tell the user what they need to do (e.g. look at the ceiling, look behind furniture), in order to get the necessary scan quality.

Cached versus Continuous Spatial Mapping

The spatial mapping data is the most heavy weight data source applications can consumed and to avoid performance issues such as dropped frames or stuttering consumption of this data should be done carefully.

Active scanning during an experience can be both beneficial or detrimental and the developer will need to decide which method to use based on the experience.

Cached Spatial Mapping

In the case of cached spatial mapping, the application typically takes a snapshot of the spatial mapping data and uses this snapshot for the duration of the experience. Benefits include:

  • Reduced overhead on the system while the experience is running leading to dramatic power, thermal and cpu performance gains.
  • A simpler implementation of the main experience since it is not interrupted by changes in the spatial data.
  • A single one time cost on any post processing of the spatial data for physics, graphics and other purposes.

Drawbacks include:

  • The movement of real world objects or people is not reflected by the cached data. E.g. the application might consider a door open when it is actually closed now.
  • Potentially more application memory to maintain the cached version of the data.

A good case for this method is a controlled environment or a table top game.

Continuous Spatial Mapping

Certain applications may rely on continues scanning to refresh spatial mapping data.


  • You may decide to forgo a building separate scanning or exploration phase.
  • The movement of real world object can be reflected by the game, although with some delay.


  • Higher complexity in the implementation of the main experience.
  • Potential overhead of the additional processing for graphic or physics as changes need to be incrementally ingested by these systems.
  • Higher power, thermal and cpu impact.

A good case for this method is one where holograms are expected to interact with moving objects, e.g. a holographic car that drives on the floor may want to correctly bump into a door depending on whether it is open or closed.