Facebook patented hand tracking system

Facebook patented hand tracking with finger emitters.

Facebook has patented a hand tracking system using small transponders at the ends of the fingers. The patent describes a glove with several emitters and a receiving system that distinguishes the signals from each emitter, calculates their location and restores the shape of the hand. The Patently Apple website drew attention to the patent.

Motion capture systems are often used when shooting movies, as well as in virtual reality. There are several standard tracking methods. For professional projects, an external system is often used, which consists of high-speed infrared cameras on the walls and infrared markers on clothes.

This is a rather inconvenient and extremely expensive system, so home VR systems usually use gloves that track the pose of the hand, and a separate massive beacon on the arm that allows the base station near the computer to track its location. Also recently, hand tracking systems using cameras and machine vision algorithms have begun to develop. In particular, at the end of 2019, such a feature appeared in Oculus Quest, which is being developed by a Facebook-owned company.

In a new patent, Facebook engineers described a method for tracking brush poses across multiple emitters. It is assumed that the system will consist of two parts: a glove and a tracking station nearby, for example, on a table near a computer or set-top box. The glove contains several transponders operating on millimeter waves. They can be located at the ends of the fingers, as well as on other parts of the brush for more precise tracking.

The tracking station has several antennas. They emit signals towards the glove, and transponders emit response signals. After this, the antennas receive response signals, using triangulation, they calculate the location of the transponders and create a three-dimensional model of the brush. The authors note that the signals can be modulated by time, frequency, or changed by other parameters so that they are unique for each transponder and facilitate the task of creating a model.

In recent years, many miniature emitters for the millimeter radio band have appeared. Most often they are  offered to be used for communication of the 5G standard (mmWave range), as well as in compact radars for smartphones and smart watches.