Abstract: A user can provide input to an electronic device by performing a specific motion or gesture that can be detected by the device. At least one imaging or detection element captures information including the motion or gesture, such that one or more dwell points can be determined in two or three dimensions of space. The dwell points can correspond to any point where the motion pauses for at least a minimum amount of time, such as at an endpoint or a point where the motion significantly changes or reverses direction. The set of dwell points, and the order in which those dwell points occur, can be compared against a set of gestures to attempt to match a gesture associated with a particular input. Such an approach is useful for devices with image capture elements or other components that are not able to accurately capture motion or determine movements, etc.

Abstract: Image capture can be improved by capturing a sequence of images and analyzing the images to select the image with the least blur and/or an acceptable amount of blur. Gradients can be calculated for at least a portion of the images, and gradient histograms generated. Two or more component curves can be fit to each histogram, such as by using a Gaussian mixture model, and the curves can be compared to determine an amount of variation between the curves. The image with the smallest differences between component curves, or with differences less than a specified blur threshold, can be selected as a sufficiently sharp image and provided for viewing, processing, or another intended purpose of the image to be captured.

Abstract: A user attempting to obtain information about an object can capture image information including a view of that object, and the image information can be used with a matching or identification process to provide information about that type of object to the user. Information about the orientation of the camera and/or device used to capture the image can be provided in order to limit an initial search space for the matching or identification process. In some embodiments, images can be selected for matching based at least in part upon having a view matching the orientation of the camera or device. In other embodiments, images of objects corresponding to the orientation can be selected. Such a process can increase the average speed and efficiency in locating matching images. If a match cannot be found in the initial space, images of other views and categories can be analyzed as well.

Abstract: Additional components or functionality can be provided for a computing device by coupling the device with one or more device skins. Each device skin can include one or more components that can be utilized by the computing device, such that a user of the computing device can obtain desired functionality by selecting an appropriate skin. The components of the skin can be powered by the computing device, such as through wired or wireless power coupling. The computing device can determine an appropriate control scheme for the skin when the skin is coupled with the device, enabling a user to utilize the various components of the skin as if those components were part of the computing device.

Abstract: The registration of images captured at multiple locations, for purposes such as location mapping, can be improved by utilizing multiple image capture elements pointing in at least two different directions or having different viewable ranges. At least one primary image is captured at each location. If the primary image is not able to be correlated with at least one other image based on one or more matching features, image information captured by at least one other camera at substantially the same times as those images can be analyzed to attempt to determine a change in position and/or orientation of the device between those times, which can assist in correlating the primary images. In some embodiments, motion or orientation determining elements can assist in the determination of device movement, and in at least some embodiments can reduce the amount of image information to be processed for a match.

Abstract: The quality of stereoscopic imaging using cameras with wide angle lenses can be improved using various calibration approaches to determine distortions or misalignments between the cameras. A calibration object such as a checkered grid with a determined inclination between portions can be used to provide lateral calibration as well as depth information. Intersections of the checkered regions can be used to determine intersection points for the calibration object in the image. Regions about at least a portion of these points can be analyzed, where the regions are sized such that features of the calibration object are substantially linear. Nearest neighbor points can be identified, and a local orientation determined in order to determine a relationship of each of the nearest points. Once identified, the points in each image can be correlated to corresponding points of the calibration object.

Abstract: The quality of stereoscopic imaging using cameras with wide angle lenses can be improved using various calibration approaches to determine distortions or misalignments between the cameras. A calibration object such as a checkered grid with a determined inclination between portions can be used to provide lateral calibration as well as depth information. Intersections of the checkered regions can be used to determine intersection points for the calibration object in the image. Lines formed by the checkered regions can be located, and the points correlated with these lines, in order to determine correspondence between points in the image. These points are mapped to corresponding points of the calibration object in the real world, to determine system parameters and/or image adjustments to be made when subsequent images are captured, in order to remove the distortions and misalignment when providing a stereo image based on images from the cameras.

Abstract: The amount of power and processing needed to process gesture input for a computing device can be reduced by utilizing a separate gesture sensor. The gesture sensor can have a form factor similar to that of conventional camera elements, in order to reduce costs by being able to utilize readily available low cost parts, but can have a lower resolution and adjustable virtual shutter such that fast motions can be captured and/or recognized by the device. In some devices, a subset of the pixels of the gesture sensor can be used as a motion detector, enabling the gesture sensor to run in a low power state unless there is likely gesture input to process. Further, at least some of the processing and circuitry can be included with the gesture sensor such that various functionality can be performed without accessing a central processor or system bus, thus further reducing power consumption.

Abstract: A user can be identified and/or authenticated to an electronic device by analyzing aspects of a motion or gesture made by that user. At least one imaging element of the device can capture image information including the motion or gesture, and can determine time-dependent information about that motion or gesture in two or three dimensions of space. The time-dependent information can be used to identify varying speeds, motions, and other such aspects that are indicative of a particular user. The way in which a gesture or motion is made, in addition to the motion or gesture itself, can be used to authenticate an individual user. While other persons can learn the basic gesture or motion, the way in which each person makes that gesture or motion will generally be at least slightly different, which can be used to prevent unauthorized access to sensitive information, protected functionality, or other such content.

Abstract: Additional components or functionality can be provided for a computing device by coupling the device with one or more device skins. Each device skin can include one or more components that can be utilized by the computing device, such that a user of the computing device can obtain desired functionality by selecting an appropriate skin. The components of the skin can be powered by the computing device, such as through wired or wireless power coupling. The computing device can determine an appropriate control scheme for the skin when the skin is coupled with the device, enabling a user to utilize the various components of the skin as if those components were part of the computing device.

Abstract: An electronic device can utilize one or more sensors and/or imaging elements to determine the relative position of at least one light source relative to the device. In various embodiments, the intensity of light can be determined in various directions around the device. By determining the relative intensity around the device, an approximate direction of one or more light sources can be determined. Utilizing the relative position of each light source, the electronic device can properly light or shade a graphical object to be rendered by the device, or otherwise process image information captured by the device.

Abstract: The computational resources needed to perform processes such as image recognition can be reduced by determining appropriate frames of image information to use for the processing. In some embodiments, infrared imaging can be used to determine when a person is looking substantially towards a device, such that an image frame captured at that time will likely be adequate for facial recognition. In other embodiments, sound triangulation or motion sensing can be used to assist in determining which captured image frames to discard and which to select for processing based on any of a number of factors indicative of a proper frame for processing.

Abstract: Approaches to enable a computing device, such as a phone or tablet computer, to determine depth information about an object captured by a single camera of the device without requiring multiple cameras to capture that object in their field of view. For example, the computing device may track the object throughout a sequence of images using a single rear-facing camera and then determine the depth information of that object by combining data about the object's changes in size (throughout the sequence of images) and information about the distance between the computing device and a user's face (or other user's feature) that can be determined using two or more front-facing cameras configured for stereo imaging. The depth information of the object may include the distance between the computing device and the object, the distance between the object and the user's face, or the physical dimensions of the object, among other such information.

Abstract: Content displayed on a display element of a computing device can be rendered from a viewpoint that updates with changes in the relative position of a user's head, or other such feature, with respect to the device. At least one camera of the device can attempt to track the relative position in order to determine the proper viewpoint. When the user's head can no longer be seen, or when confidence in the determined relative position drops below an acceptable value, for example, the device can utilize information from another source, such as a motion or rotation sensor, to attempt to estimate the current position of the user's head. Physical limitations of the user with respect to the device also can be used to improve the accuracy of the determination.

Abstract: An object represented in an image can be segmented from the image background by capturing a pair of images, one with flash and one without, and generating a differential image. This differential image can be analyzed using an algorithm, such as a connected components or computer vision algorithm, to determine one or more portions of the image that correspond to an object. An appropriate one of these objects can be selected as corresponding to the object of interest, and an outline of the selected object can be used to determine a portion of one of the original images that corresponds to the object. This portion then can be provided to an object recognition or other such process for analysis, which can increase the efficiency and accuracy of the analysis.

Abstract: The registration of images captured at multiple locations, for purposes such as location mapping, can be improved by utilizing multiple image capture elements pointing in at least two different directions or having different viewable ranges. At least one primary image is captured at each location. If the primary image is not able to be correlated with at least one other image based on one or more matching features, image information captured by at least one other camera at substantially the same times as those images can be analyzed to attempt to determine a change in position and/or orientation of the device between those times, which can assist in correlating the primary images. In some embodiments, motion or orientation determining elements can assist in the determination of device movement, and in at least some embodiments can reduce the amount of image information to be processed for a match.

Abstract: The use of resources on a computing device can be optimized for current conditions to reduce or minimize the amount of resources needed to provide a sufficient level of performance for various types of tasks. In some embodiments, one or more optimization algorithms can analyze information such as a type of task to be performed and the state of one or more environmental conditions to attempt to select a number and configuration of components, such as cameras and illumination elements, to use in performing the tasks. The performance of the tasks can be monitored, and the selection or states updated in order to maintain a sufficient level of performance without using more resources than are necessary.

Abstract: Locating features in image information captured by multiple cameras of a computing device can be facilitated by removing background objects from the image information. A distance past which objects should be considered to be background objects is determined, and an amount of disparity for that distance calculated. Any objects having less than that amount of disparity in the image information can be removed, leaving objects that should be treated as foreground objects. An advantage of such an approach is that objects can be determined to be foreground objects even when those objects are only viewable by one of the cameras and disparity information cannot be calculated for those objects from the image information. Such an approach can be combined with other approaches as well to improve accuracy.

Abstract: A camera of a computing device can capture two or more images of a region including an object of interest, in order to allow for separation of the object from a background of the images through a process such as image subtraction. In order to compensate for rotations of the device between image captures, an element such as an electronic gyroscope can be used to monitor changes in orientation and predict an amount of shift of objects between images. The predicted shift can be used to attempt to align images captured around the time of the rotation, in order to enable subtraction or similar processes by effectively removing the shifting effect of the rotation.

Abstract: Motions and gestures can be detected using a video capture element of a computing device even when the video capture element is not able to accurately capture the motion. Information about the background in the image information can be determined, and the way in which that background information is occluded can be used to determine the motion. In at least some embodiments, edges are detected in the video information. Images of foreground objects can then be isolated from edges of background images by comparing histograms of multiple frames of video. The remaining data is indicative of a direction and speed of motion, which can be used to infer a determined gesture even though that gesture was not visible in the captured video information.