Apple continues to work on 3D interfaces and has been granted a patent (number 9,235,753) for “extraction of skeletons from 3D maps.” While this sounds like something from a horror movie, it involves a method for processing data that includes receiving a timed sequence of depth maps of a scene containing a humanoid form having a head.
A "depth map" has the form of an image in which the pixel values indicate topographical information, rather than brightness and/or color of the objects in the scene. They can be used to segment and identify objects in a scene. Identification of humanoid forms (in other words, 3D shapes whose structure resembles that of a human being) in a depth map, and changes in these forms from scene to scene, may be used as a means for controlling computer applications.
For example, a depth map may be segmented to identify a contour of a humanoid body. The contour is processed in order to identify a torso and one or more limbs of the body. An input is generated to control an application program running on a computer by analyzing a disposition of at least one of the identified limbs in the depth map. This could be used for gaming, moving onscreen objects by gesture, and more.
According to the patent, the depth maps include a matrix of pixels having respective pixel depth values. A digital processor processes at least one of the depth maps so as to find a location of the head and estimates dimensions of the humanoid form based on the location. The processor tracks movements of the humanoid form over the sequence using the estimated dimensions.
This isn’t Apple’s first patent regarding 3D interfaces. Patent number 9,218,063 is for a “sessionless pointing user interface” that involves a sequence of 3D maps on a Mac and other devices that could detect your gestures and respond accordingly.
U.S. Patent No. 7,348,963, describes an interactive video display system, in which a display screen displays a visual image, and a camera captures 3D information regarding an object in an interactive area located in front of the display screen. A computer system directs the display screen to change the visual image in response to changes in the object.
What’s more, U.S. Patent No. 2010/0034457 describes a method for modeling humanoid forms from depth maps. The depth map is segmented so as to find a contour of the body. The contour is processed in order to identify a torso and one or more limbs of the subject. An input is generated to control an application program running on a computer by analyzing a disposition of at least one of the identified limbs in the depth map.
A 3D interface could be used for gaming, manipulating onscreen objects, and turning on off gadgets, such as lights in a room. The gestures could be used to scrolling, selecting, zooming, and more.