Abstract: A mobile device for projecting images on a surface and for maintaining a position of the image on the surface. The device including an image rendering module to scan the surface and to modify the image based on data obtained about a three dimensional nature of the surface. The device further including an optical image stabilization to adjust an alignment of the projector based on orientation data of the device, an electronic image stabilization module to shift the image within a projection area based on the orientation data and an image-tracking module to shift the image within the projection area based on changes in a relative positions of the image and a target.

Abstract: Various approaches for providing textual information to an application, system, or service are disclosed. In particular, various embodiments enable a user to capture an image with a camera of a portable computing device. The computing device is capable of taking the image and processing it to recognize, identify, and/or isolate the text in order to forward the text to an application or function. The application or function can then utilize the text to perform an action in substantially real-time. The text may include an email, phone number, URL, an address, and the like and the application or function may be dialing the phone number, navigating to the URL, opening an address book to save contact information, displaying a map to show the address, and so on.

Abstract: A user can capture an image of an object, using a computing device, to obtain information about that object. If a specular highlight (or other saturated region) is detected, the device can attempt to determine a location of a light source associated with the highlight. The device can then provide instructions as to a direction to move in order to reduce the presence of the specular highlight in subsequent images. Multiple images of the object can be captured and analyzed to generate a three-dimensional reconstruction of the environment, whereby a position of the light source can be determined. In other embodiments, movement of the specular reflections in response to movement of the device is used to determine a direction of the light source. In other embodiments, an image of the user is captured to determine the position of the light source based on shadows or reflections on the user.

Abstract: A computing device can detect first and second device motions within a first time period. The first and second device motions may be separated by a second time period. The computing device can determine that the second time period is within a specified time range. The computing device can determine that first and second motion metrics for the first and second device motions, respectively, are likely indicative of first and second taps, respectively, on a surface of the computing device. The computing device can determine that a first coordinate frame of the computing device associated with the first tap is within a range of angular degrees from a second coordinate frame of the computing device associated with the second tap. The computing device can further determine the first tap and the second tap as part of a double tap gesture.

Abstract: A mobile device for projecting images on a surface and for maintaining a position of the image on the surface. The device including an image rendering module to scan the surface and to modify the image based on data obtained about a three dimensional nature of the surface. The device further including an optical image stabilization to adjust an alignment of the projector based on orientation data of the device, an electronic image stabilization module to shift the image within a projection area based on the orientation data and an image-tracking module to shift the image within the projection area based on changes in a relative positions of the image and a target.

Abstract: A tracking architecture is provided that enables data for gestures and head positions to be provided to both native and non-native clients on a computing device. A pipeline component can obtain the raw image data and sensor data and synchronize that data to be processed to determine, for example, location and/or motion data that may correspond to device input. The data can be processed by separate components, such as an event publisher and an event provider, each capable of filtering the location, motion, and/or raw sensor data to generate a set of event data. The event data then can be published to registered listeners or provided in response to polling requests. Head coordinates, gesture data, and other such information can be passed through one or more interface layers enabling the data to be processed by a non-native client on the device.

Abstract: A computing device can detect first and second device motions within a first time period. The first and second device motions may be separated by a second time period. The computing device can determine that the second time period is within a specified time range. The computing device can determine that first and second motion metrics for the first and second device motions, respectively, are likely indicative of first and second taps, respectively, on a surface of the computing device. The computing device can determine that a first coordinate frame of the computing device associated with the first tap is within a range of angular degrees from a second coordinate frame of the computing device associated with the second tap. The computing device can further determine the first tap and the second tap as part of a double tap gesture.

Abstract: A device configured with a user interface (UI) that changes based on a position of a user determines the position of the user through multiple data sources including camera based head tracking and output from motion sensors such as a gyroscope. Each data source may output its own estimated head position. The device may apply a reliability weight to the head position determined by each data source. A composite head position is then determined from the weighted position. The composite position is then used to render the UI.

Abstract: A device may identify a user among multiple individuals detected by comparing a physiological condition, such as a heart rate, of the multiple detected individuals as detected by a camera of the device with a physiological condition of a user of the device using different sensors. A heart rate may be detected by a camera by monitoring blood flow to an individual's face. A heart rate may be detected by a motion sensor by monitoring vibrations of the device that are in an expected frequency range. If a heart rate of a user matches a heart rate of an individual seen by a camera, that individual may be determined to be the user of the device. The position of the individual may be used to then render a user interface.

Abstract: Various embodiments utilize two-dimensional (“2D”) and three-dimensional (“3D”) object features for purposes such as object recognition and/or image matching. For example, a user can capture an image (e.g., still images or video) of an object and can receive information about items that are determined to match the object. For example, the image can be analyzed to detect visual features (e.g., corners, edges, etc.) of the object and the detected visual features can be combined to generate a combined visual feature vector which can be used for object recognition, image matching, or other such purposes. Other approaches utilize the image to generate a 3D model of the object represented in the image, which can be used to determine at least one object or types of objects that match the object represented in the image.

Abstract: A device may recognize a tilt gesture when a device rotates about an axis and then back again. The gesture may be recognized using a state machine. Recognition of the gesture may be performed based on a context of a device, where the specific movement of the device during a tilt gesture may change based on the context. The tilt gesture may be confirmed using a classifier trained on features describing the gesture and the context.

Abstract: A user attempting to obtain information about an object can capture image information including a view of that object, and the image information can be used with a matching or identification process to provide information about that type of object to the user. In order to narrow the search space to a specific category, and thus improve the accuracy of the results and the speed at which results can be obtained, the user can be guided to capture image information with an appropriate orientation. An outline or other graphical guide can be displayed over image information captured by a computing device, in order to guide the user in capturing the object from an appropriate direction and with an appropriate scale for the type of matching and/or information used for the matching. Such an approach enables three-dimensional objects to be analyzed using conventional two-dimensional identification algorithms, among other such processes.

Abstract: The accuracy of a text recognition process can be improved using a set of semantic boosting rules, as may be contained in a sequence or other such arrangement. When text is output from a text recognition process, that text can have alternatives and confidence values for different characters or portions of the string. In order to improve the accuracy, this data can be processed using the organized rules, where rules are applied as long as any preconditions for that rule are satisfied, and each rule has the ability to modify the confidence values or modify one or more of the alternatives. When a result it produced with a minimum confidence level, or all applicable rules have been applied, the result can be provided as a refined text output of the recognition process.

Abstract: Various embodiments provide an image pre-processing method to accelerate image computing tasks. For example, an image is obtained from a camera of a portable computing device. Salient features of the image are identified by a number of techniques, such as identifying feature points, identifying gradients within the image, and other techniques and methods discussed herein. Based at least in part on the identified salient features, a saliency map of the image is computed. The saliency map includes a saliency value for each of at least a portion of the plurality of pixels. In this example pixels above a threshold saliency value are processed with an optical character recognition engine.

Abstract: A device may identify a user among multiple individuals detected by comparing a physiological condition, such as a heart rate, of the multiple detected individuals as detected by a camera of the device with a physiological condition of a user of the device using different sensors. A heart rate may be detected by a camera by monitoring blood flow to an individual's face. A heart rate may be detected by a motion sensor by monitoring vibrations of the device that are in an expected frequency range. If a heart rate of a user matches a heart rate of an individual seen by a camera, that individual may be determined to be the user of the device. The position of the individual may be used to then render a user interface.

Abstract: A tracking architecture is provided that enables data for gestures and head positions to be provided to both native and non-native clients on a computing device. A pipeline component can obtain the raw image data and sensor data and synchronize that data to be processed to determine, for example, location and/or motion data that may correspond to device input. The data can be processed by separate components, such as an event publisher and an event provider, each capable of filtering the location, motion, and/or raw sensor data to generate a set of event data. The event data then can be published to registered listeners or provided in response to polling requests. Head coordinates, gesture data, and other such information can be passed through one or more interface layers enabling the data to be processed by a non-native client on the device.

Abstract: Various embodiments utilize geometric hashing to automatically recognize and track and an object. For example, a user can capture an image of a product, or other object, and a point detection algorithm can identify particular features of the product and designate feature points of the product within the captured image. Hash values are then determined for each feature point by determining a basis for the image and determining the location of each feature point relative to that basis. A hash table including the identified hash values is then created and compared to hash values for either a stored product when performing object recognition or from a reference image when performing object tracking.

Abstract: Approaches enable the projection of one or more visual elements, such as one or more dynamically changing graphical elements, that can substantially bound, or otherwise at least partially surround or identify, an object recognized by a computing device. The computing device can project the graphical elements to collectively appear as a bounding element for the recognized/actionable object or object portion. As such, the graphical elements can appear as a bounding element that adorns, decorates, highlights, and/or emphasizes, etc., the recognized/actionable object or object portion. The graphical elements to be dynamic. For example, the graphical elements can be projected to move around individually over time, while still appearing to at least partially surround the recognized/actionable object or object portion. Further, the graphical elements can be used to improve various object recognition approaches.

Abstract: A user attempting to obtain information about an object can capture image information including a view of that object, and the image information can be used with a matching or identification process to provide information about that type of object to the user. In order to narrow the search space to a specific category, and thus improve the accuracy of the results and the speed at which results can be obtained, the user can be guided to capture image information with an appropriate orientation. An outline or other graphical guide can be displayed over image information captured by a computing device, in order to guide the user in capturing the object from an appropriate direction and with an appropriate scale for the type of matching and/or information used for the matching. Such an approach enables three-dimensional objects to be analyzed using conventional two-dimensional identification algorithms, among other such processes.

Abstract: Various embodiments enable a computing device to capture multiple images (or video) of text and provide at least a portion of the same to a recognizer to separately recognize text from each image. Each of the recognized outputs will typically include one or more text strings for each image. Substrings common to each of the one or more text strings are computed and compared to each text string within each image to determine an alignment consensus for each substring within the text. A template string is generated that includes each common substring in a position corresponding to a determined alignment for a respective substring. A character frequency vote is then applied to unresolved portions and the final text string is determined by filling the unresolved spaces with the character having the highest occurrence rate for a respective space.