Patent report: gesture and gaze controlled user interfaces, more

Apple files for — and is granted — lots of patents by the U.S. Patent & Trademark Office. Many are for inventions that never see the light of day. However, you never can tell which ones will materialize in a real product, so here are this week’s patent highlights: 

The world of Minority Report draws closer. Apple has been granted a patent (number 9,285,874) for “gaze detection in a 3D mapping environment” that would allow you to control a Mac or a TV with gestures. 

In the patent filing, Apple notes that a variety of user interface devices and methods are currently available. Common tactile interface devices include a computer keyboard, a mouse and a joystick. Touch screens detect the presence and location of a touch by a finger or other object within the display area. Infrared remote controls are widely used, and "wearable" hardware devices have been developed, as well, for purposes of remote control. 

Computer interfaces based on three-dimensional (3D) sensing of parts of a user's body have also been proposed, and, apparently, Apple thinks it’s an idea worth investigating. According to the invention, a 3D sensor, typically positioned in a room in proximity to the user, provides position information, which is used to identify gestures created by a body part of interest. The gestures are recognized based on the shape of the body part and its position and orientation over an interval. The gesture is classified for determining an input into a related electronic device such as a Mac or a TV.  The patent also mentions a user interface system that track the detection of a user’s eyes.

For the technically minded among you, here’s the summary of the patent: “A method, including receiving a three-dimensional (3D) map of at least a part of a body of a user of a computerized system, and receiving a two dimensional (2D) image of the user, the image including an eye (34) of the user. 3D coordinates of a head of the user are extracted from the 3D map and the 2D image, and a direction of a gaze performed by the user is identified based on the 3D coordinates of the head and the image of the eye.”

Apple has also been granted a patent (number 9,287,657) for a “headset connector” that will only fuel the rumors (which I think are on target) that the next generation iPhone will ditch the 3.5mm headset jack and only accept headsets/earbuds that use a Lightning connector or Bluetooth connectivity.

Here’s Apple’s summary of the invention: “Headset assemblies and headset connectors are provided. Headset connectors can include a magnetic mating face and a plurality of electrical contacts disposed within the mating face. Engaging assemblies and engaging connectors are also provided. The engaging connectors can include a housing having a mating side, a magnetic array structure, and a plurality of spring biased contact members. The magnetic array structure can be fixed within the housing and house a plurality of spring biased contact members. The spring biased contact members can include tips that extend out of the mating side. The tips can electrically couple with electrical contacts in a headset connector.”

Future Macs and iOS devices may pack even better speakers. Apple has applied for a patent (number 20160080870) for “audio speaker surround geometry for improved histrionic motion.” It’s for a speaker surround having a film that undulates in a peripheral direction around a speaker diaphragm.


Audio speakers typically include a moving assembly that is connected to a stationary assembly by a suspension system that includes elements that keep the moving assembly centered relative to the stationary assembly. However, the non-pistonic motion in some modes may cause the voicecoil to stretch and/or rub against the magnet, and over time, this can lead to issues with the emitted sound quality or cause failure of the audio speaker. Apple’s audio speaker patent is designed to overcome this problem.

In addition to better speakers, upcoming Apple products may also have flexible screens. The company has applied for another bendable screen patent, this one (number 20160077371) dubbed “display having a flexured element.” However, this patent is designed to make better quality displays rather than ones that truly fold.

According to the patent, the display device may incorporate a cover glass with one or more flexure bearing sections. The flexure bearing may permit the cover glass to flex and/or bend in response to stresses or strains while still maintaining contact with an attachment point.

In the patent filing, Apple notes that many modern electronic devices include integrated displays, or interact with standalone displays. One common form of display is the liquid crystal display, which uses liquid crystals to selectively transmit light and thus form visible images.  In many such displays, a backlight is used to provide light that may be gated or transmitted by the liquid crystals. 

In some cases, however, light from the backlight may leak to the front of the display, even when such light is not desired. This may cause visible illumination at the front of the display at undesirable times, while light leakage may distort text, images and the like, or render them visually unappealing. Apple wants to fix this.