• Diyan
• On August 22, 2017

A new way to interact with future mobile devices and tablets

Microsoft has been granted a patent for a 3D camera setup for possible use in new Surface devices (mobile phones and tablets), allowing users to control and interact with the device with in-air hand gestures or drawings on a virtual surface. This patent is in line with the recent leak about Microsoft’s Surface Phone having a feature called ‘onto table‘ to project the Windows UI onto any surface including table or draw-bord.

According to the description, the 3D camera would bring several entirely new ways to interact with the mobile device. The user will be able to draw or write on any surface, while the 3D camera will capture the input. There are no limitations of how the user writes on the surface, so he/she could theoretically use a finger, coffee stirrer or a professional stylus.

Microsoft’s 3D camera will enable interaction with the device without touching the display, while the device itself may have a built-in projector to show the Windows UI, virtual keyboard and other elements. The processor will translate the gestures into commands that will affect execution of code.
From the patent description:

The processor may then map or translate the gesture into a command that may affect execution of code, which may change how elements (e.g., text, audio, drawings, and so on) in display are being displayed, for example. In some examples, such gestures may be used to control pan and zoom of elements in the display. In other examples, such gestures may be used to rotate or translate elements in display. In still other examples, such gestures may be used for any of a number of editing processes for elements in display.

Microsoft has filed the application for this patent in early 2016, but it’s uncertain if it will be used in a real product in the near future.

Source