Abstract: An electronic device can utilize one or more sensors and/or imaging elements to determine the relative position of at least one light source relative to the device. In various embodiments, occlusions can be used to cause shadows to be cast on certain sensors. By determining the relative position of each occlusion relative to the sensor, the device can determine an approximate direction of the light source. Utilizing the relative position of a light source, the electronic device can properly light or shade a graphical object to be rendered by the device or otherwise process image information captured by the device.

Abstract: An electronic device performs functionality adjustments based on a determined attention level of a user of the device. In some embodiments, the device automatically enters a sleep mode upon determining that the user has fallen asleep. The device may automatically modify functionality of the device such that disturbances to the user is minimized or power consumption is reduced. For example, the device may lower the volume of a ring tone or deactivate a communication channel. In some instances, the device may determine that the user is half-awake or paying an intermediate level of attention and activate an intermediate mode where at least some functionality is modified or adjusted. A device subsequently determining that the user is fully awake or actively engaged with the device causes the device to return to a standard mode of operation, where functionality of the device is adjusted to one appropriate for the user's attention level.

Abstract: A user of a computing device can be authenticated using image information captured by at least one camera of the computing device. In addition to analyzing the image information using a facial recognition algorithm, for example, variations in color of a portion of the captured image information corresponding to a user's face can be monitored over a period of time. The variations can be analyzed to determine whether the captured image information likely corresponds to an actual human user instead of a representation (e.g., photo) of a human user, such as where the chroma variations in at least a red channel occur with an oscillation frequency and amplitude consistent with changes due to a pulse or heartbeat.

Abstract: A low distraction interface can be provided for an electronic device by monitoring information such as a gaze direction of a user. A device can be configured to only display information to a user when the user is looking substantially toward the device, and can be configured to present that information in a way that is of minimal distraction to others nearby. The user can control aspects of the display by looking away from the device, continuing to look at the device for a period of time, or otherwise providing input without physically interacting with the device. Such an approach can be beneficial for settings such as business meetings where a user might want to obtain information from the device but does not want to appear inconsiderate by picking up and checking information on the device.

Abstract: A computing device and/or application executing on the device can utilize fingertip tracking using a camera. However, when the fingertip is in a dead zone (an area that is not viewable by the camera), the fingertip tracking cannot function properly. Nevertheless, the position of the fingertip, when in the dead zone, can still be estimated. An image of a user's hand can be captured by at least one camera. A portion of a pointing finger can be detected in the captured image. An orientation of the portion of the pointing finger can be determined. One or more joint lines of the pointing finger can be identified. Based on data about a slant and/or a bend of the pointing finger obtained using information relating to the identified joint line(s), and/or on data obtained via calibration and/or historic/current usage, the position of the fingertip, when in the dead zone, can be approximated.

Abstract: Image information displayed on an electronic device can be modified based at least in part upon a relative position of a user with respect to a device. Mapping, topological or other types of positional data can be used to render image content from a perspective that is consistent with a viewing angle for the current relative position of the user. As that viewing angle changes, as a result of movement of the user and/or the device, the content can be re-rendered or otherwise updated to display the image content from a perspective that reflects the change in viewing angle. Simulations of effects such as parallax and occlusions can be used with the change in perspective to provide a consistent user experience that provides a sense of three-dimensional content even when that content is rendered on a two-dimensional display. Lighting, shading and/or other effects can be used to enhance the experience.

Abstract: An electronic device including two or more display elements can provide enhanced functionality with improved rates of power consumption. A user can cause information that does not change rapidly to be provided or moved to a relatively static display element, such as an electronic ink display, which enables that information to be displayed for a period of time with little additional power consumption. Similarly, content (e.g., video) that changes rapidly can be displayed on a relatively dynamic display element, such as and LCD or OLED display. Each display can be touch sensitive, such that a user can move content between the displays by pressing on, or making a motion in contact with, at least one of the displays. Various modes can be activated which cause certain types of content to be displayed on the dynamic and/or static display element.

Abstract: A user attempting to obtain information about an object can capture image information including a view of that object, and the image information can be used with a matching or identification process to provide information about that type of object to the user. Information about the orientation of the camera and/or device used to capture the image can be provided in order to limit an initial search space for the matching or identification process. In some embodiments, images can be selected for matching based at least in part upon having a view matching the orientation of the camera or device. In other embodiments, images of objects corresponding to the orientation can be selected. Such a process can increase the average speed and efficiency in locating matching images. If a match cannot be found in the initial space, images of other views and categories can be analyzed as well.

Abstract: A user of a computing device can be authenticated using image information captured by at least one camera of the computing device. In addition to analyzing the image information using a facial recognition algorithm, for example, variations in color of a portion of the captured image information corresponding to a user's face can be monitored over a period of time. The variations can be analyzed to determine whether the captured image information likely corresponds to an actual human user instead of a representation (e.g., photo) of a human user, such as where the chroma variations in at least a red channel occur with an oscillation frequency and amplitude consistent with changes due to a pulse or heartbeat.

Abstract: A user can make a symbol with their hand, or other such gesture, at a distance from a computing device that can be captured by at least one imaging element of the device. The captured information can be analyzed to attempt to determine the location of distinguishing features of the symbol in the image information. The image information is then compared to hand gesture information stored in, for example, a library of hand gestures for the user. Upon identifying a match, an input to an application executing on the computing device is provided when the image information contains information matching at least one hand gesture with at least a minimum level of certainty. The hand gesture could include a single “static” gesture, such as a specific letter in sign language, for example, or include two or more “static” gestures. The gesture could also include motion, such as hand movement.

Abstract: A low distraction interface can be provided for an electronic device by monitoring information such as a gaze direction of a user. A device can be configured to only display information to a user when the user is looking substantially toward the device, and can be configured to present that information in a way that is of minimal distraction to others nearby. The user can control aspects of the display by looking away from the device, continuing to look at the device for a period of time, or otherwise providing input without physically interacting with the device. Such an approach can be beneficial for settings such as business meetings where a user might want to obtain information from the device but does not want to appear inconsiderate by picking up and checking information on the device.

Abstract: A user can provide input to a computing device by moving a feature or object, such as a user's finger, within a field of view of at least one imaging element of the computing device. In order to ensure an accuracy of the determined input, the computing device can also attempt to determine a point of view of the user, such as by determining a relative position of the user's face or eyes. By determining a three-dimensional position of a feature and the user's point of view, a three-dimensional vector or other directional information can be determined whereby the intersection of that vector with the computing device indicates an intended location of input corresponding to the feature from the user's point of view.

Abstract: The pupil locations of a user with respect to a computing device can be determined by capturing one or more images of the user and analyzing those images using a set of pupil detection algorithms. Each algorithm can produce at least one estimated position with an associated confidence value, and this information from each algorithm can be used to determine a probable location of each pupil. In some embodiments, one or more environmental factors can be used to adjust the confidence values or select algorithms based on how the corresponding algorithms perform under those conditions. Similarly, an independence of the various algorithms can be utilized in some embodiments to adjust the confidence levels or weight results based on a level of dependence between those algorithms.

Abstract: A user can make a symbol with their hand, or other such gesture, at a distance from a computing device that can be captured by at least one imaging element of the device. The captured information can be analyzed to attempt to determine the location of distinguishing features of the symbol in the image information. The image information is then compared to hand gesture information stored in, for example, a library of hand gestures for the user. Upon identifying a match, an input to an application executing on the computing device is provided when the image information contains information matching at least one hand gesture with at least a minimum level of certainty. The hand gesture could include a single “static” gesture, such as a specific letter in sign language, for example, or include two or more “static” gestures. The gesture could also include motion, such as hand movement.

Abstract: Various approaches enable automatic communication generation based on patterned behavior in a particular context. For example, a computing device can monitor behavior of a user to determine patterns of communication behavior in certain situations. In response to detecting multiple occurrences of the certain situation, a computing device can prompt a user to perform an action corresponding to the pattern of behavior. In some embodiments, a set of speech models corresponding to a type of contact is generated. The speech models include language consistent with patterns of speech between a user and the type of contact. Based on context and on the contact, a message using language consistent with past communications between the user and contact is generated from a speech model associated with the type of contact.

Abstract: Computing devices can collaborate in order to take advantage of various components distributed across those devices. In various embodiments, image information captured by multiple devices can be used to identify and determine the relative locations of various persons and objects near those devices, even when not every device can view those persons or objects. In some embodiments, one or more audio or video capture elements can be selected based on their proximity and orientation to an object to be captured. In some embodiments, the information captured from the various audio and/or video elements can be combined to provide three-dimensional imaging, surround sound, and other such capture data.

Abstract: Image information displayed on an electronic device can be adjusted based at least in part upon a relative position of a viewer with respect to a device. In some embodiments, image stabilization can be provided such that an image remains substantially consistent from the point of view of the viewer, not the display element of the device. The image can be stretched, rotated, compressed, or otherwise manipulated based at least in part upon the relative viewing position. Similarly, the viewer can move relative to the device to obtain different views, but views that are consistent with the viewer looking at an object, for example, through a piece of glass. The device can overlay information on the image that will adjust with the adjusted image. Three-dimensional modeling and display can be used to offset parallax and focus point effects.

Abstract: Electronic devices can identify other nearby devices, and determine the relative positions of those devices, using a combination of techniques. Various devices are able to project one or more instances of a unique identifier, such as a barcode, which can be imaged by other devices. The devices also can communicate position and/or orientation information over a wireless sideband channel. By combining the information relating to the projected identifier with information collected over the sideband channel, devices can automatically determine the location of various devices and associate a user or device identity with those devices. A user of a device then can view relative locations of those devices on a display element, including information about the user of the device. Further, the relative position determinations can enable a user to perform certain functions with respect to another device based at least in part upon the position and/or identity of that device.

Abstract: A user can use a finger, or other such object, to provide input to a computing device. The finger does not have to contact the device, but can be positioned and/or oriented in such a way that the device can determine an input that the user is attempting to provide, such as an element or icon that the user is intended to select. One or more cameras can capture image information, which can be analyzed to attempt to determine the location and/or orientation of the finger. If the finger is at least partially outside a field of view of the camera(s), the device can use a sensor (e.g., EMF) to attempt to determine a location of at least a portion of the finger, which can be used with the image information to determine the location and/or orientation of the finger. Other estimation processes can be used as well.

Abstract: Image information displayed on an electronic device can be modified based at least in part upon a relative position of a user with respect to a device. Mapping, topological or other types of positional data can be used to render image content from a perspective that is consistent with a viewing angle for the current relative position of the user. As that viewing angle changes, as a result of movement of the user and/or the device, the content can be re-rendered or otherwise updated to display the image content from a perspective that reflects the change in viewing angle. Simulations of effects such as parallax and occlusions can be used with the change in perspective to provide a consistent user experience that provides a sense of three-dimensional content even when that content is rendered on a two-dimensional display. Lighting, shading and/or other effects can be used to enhance the experience.