Abstract: This disclosure describes systems and techniques for detecting events, determining a result of each respective event using a first hypothesis source, and calculating a likelihood that a second (and/or additional) hypothesis source would determine the same result of the respective event. The calculated likelihood may then be used to be determine whether to request that the second hypothesis source determine the result of the event, determine an amount of resources of the second hypothesis source to use to make this determination, and/or like.

Abstract: Systems and techniques for detecting events involving a user that occurred during a session in a facility, and performing a holistic, session-based analysis of the events to determine items taken by the user from the facility. An inventory management system may initially analyze the events to determine a hypothesis for a result of each respective event and a probability that the hypotheses are accurate. An initial listing of the items determined to be taken by the user may be generated along with probabilities that the items were taken. Upon determining that a probability an item was taken is lower than a threshold, the management system may select an algorithm to process an event in the session that is likely to resolve uncertainty in the initial listing. The management system may iteratively process events until a final listing represents the items taken by the user with an allowable probability.

Abstract: This disclosure describes systems and techniques for detecting events, determining a result of each respective event using a first hypothesis source, and calculating a likelihood that a second (and/or additional) hypothesis source would determine the same result of the respective event. The calculated likelihood may then be used to be determine whether to request that the second hypothesis source determine the result of the event, determine an amount of resources of the second hypothesis source to use to make this determination, and/or like.

Abstract: Systems and techniques for detecting events involving users that occurred at an inventory location, and then using the events to determines states of the inventory location at various times. For instance, a system may generate event data representing first events that occur within an inventory location. The first events may include an associate performing a task, a customer removing an item, a customer returning an item, and/or so forth. The system may then use the event data to determine a state of the inventory location during a second event. In some examples, the state may include a tidy state or an untidy state. If the state includes a tidy state, then the system may generate a listing of items that includes at least one item associated with the second event. However, if the state includes an untidy state, then the system may further process the second event.

Abstract: Devices, systems and methods are disclosed for improving facial recognition and/or speaker recognition models by using results obtained from one model to assist in generating results from the other model. For example, a device may perform facial recognition for image data to identify users and may use the results of the facial recognition to assist in speaker recognition for corresponding audio data. Alternatively or additionally, the device may perform speaker recognition for audio data to identify users and may use the results of the speaker recognition to assist in facial recognition for corresponding image data. As a result, the device may identify users in video data that are not included in the facial recognition model and may identify users in audio data that are not included in the speaker recognition model. The facial recognition and/or speaker recognition models may be updated during run-time and/or offline using post-processed data.

Abstract: A video capture device may include multiple cameras that simultaneously capture video data. The video capture device and/or one or more remote computing resources may stitch the video data captured by the multiple cameras to generate stitched video data that corresponds to 360° video. The remote computing resources may apply one or more algorithms to the stitched video data to identify one or more frames that depict content that is likely to be of interest to a user. The video capture device and/or the remote computing resources may generate one or more images from the one or more frames, and may send the one or more images to the user.

Abstract: Devices, systems and methods are disclosed for improving facial recognition and/or speaker recognition models by using results obtained from one model to assist in generating results from the other model. For example, a device may perform facial recognition for image data to identify users and may use the results of the facial recognition to assist in speaker recognition for corresponding audio data. Alternatively or additionally, the device may perform speaker recognition for audio data to identify users and may use the results of the speaker recognition to assist in facial recognition for corresponding image data. As a result, the device may identify users in video data that are not included in the facial recognition model and may identify users in audio data that are not included in the speaker recognition model. The facial recognition and/or speaker recognition models may be updated during run-time and/or offline using post-processed data.

Abstract: A viewing device, such as a virtual reality headset, allows a user to view a panoramic scene captured by one or more video capture devices that may include multiple cameras that simultaneously capture 360° video data. The viewing device may display the panoramic scene in real time and change the display in response to moving the viewing device and/or changing perspectives by switching to video data being captured by a different video capture device within the environment. Moreover, multiple video capture devices located within an environment can be used to create a three-dimensional representation of the environment that allows a user to explore the three-dimensional space while viewing the environment in real time.

Abstract: Devices, systems and methods are disclosed for improving facial recognition and/or speaker recognition models by using results obtained from one model to assist in generating results from the other model. For example, a device may perform facial recognition for image data to identify users and may use the results of the facial recognition to assist in speaker recognition for corresponding audio data. Alternatively or additionally, the device may perform speaker recognition for audio data to identify users and may use the results of the speaker recognition to assist in facial recognition for corresponding image data. As a result, the device may identify users in video data that are not included in the facial recognition model and may identify users in audio data that are not included in the speaker recognition model. The facial recognition and/or speaker recognition models may be updated during run-time and/or offline using post-processed data.

Abstract: Systems and methods may be directed to de-blurring panoramic images and/or video. An image processor may receive a frame, where the frame comprises a plurality of pixel values arranged in a grid. The image processor may divide the frame into a first section and a second section. The image processor may determine a first motion kernel for the first section and apply the first motion kernel to the first section. The image processor may also determine a second motion kernel for the second section and apply the second motion kernel to the second section.

Abstract: The hand which a user is using to hold an electronic device can be determined by analyzing data captured by one or more motion sensors on the device. The curvature to the motion can be indicative of handedness, and processing motion features using a classifier algorithm can enable the determination of handedness with a corresponding confidence. In some embodiments, motion data is collected over a monitoring window, and handedness values are accepted when the handedness value remains the same with at least a minimum confidence for at least a minimum number of window periods. A determination of handedness enables an operating system and/or applications executing on the device to adjust one or more operational or interface aspects in order to make it easier for the user to operate the device using the hand currently holding the device.

Abstract: A video capture device may include multiple cameras that simultaneously capture video data. The video capture device and/or one or more remote computing resources may stitch the video data captured by the multiple cameras to generate stitched video data that corresponds to 360° video. The remote computing resources may apply one or more algorithms to the stitched video data to adjust the color characteristics of the stitched video data, such as lighting, exposure, white balance contrast, and saturation. The remote computing resources may further smooth the transition between the video data captured by the multiple cameras to reduce artifacts such as abrupt changes in color as a result of the individual cameras of the video capture device having different video capture settings. The video capture device and/or the remote computing resources may generate a panoramic video that may include up to a 360° field of view.

Abstract: Devices, systems and methods are disclosed for identifying content in video data and creating content-based zooming and panning effects to emphasize the content. Contents may be detected and analyzed in the video data using computer vision, machine learning algorithms or specified through a user interface. Panning and zooming controls may be associated with the contents, panning or zooming based on a location and size of content within the video data. The device may determine a number of pixels associated with content and may frame the content to be a certain percentage of the edited video data, such as a close-up shot where a subject is displayed as 50% of the viewing frame. The device may identify an event of interest, may determine multiple frames associated with the event of interest and may pan and zoom between the multiple frames based on a size/location of the content within the multiple frames.

Abstract: A computing device can detect first and second device motions within a first time period. The first and second device motions may be separated by a second time period. The computing device can determine that the second time period is within a specified time range. The computing device can determine that first and second motion metrics for the first and second device motions, respectively, are likely indicative of first and second taps, respectively, on a surface of the computing device. The computing device can determine that a first coordinate frame of the computing device associated with the first tap is within a range of angular degrees from a second coordinate frame of the computing device associated with the second tap. The computing device can further determine the first tap and the second tap as part of a double tap gesture.

Abstract: A system determines what regions in a scene captured by a camera include highly reflective (specular) surfaces. The system illuminates the scene by outputting images to a display screen such as a television screen so as to illuminate the scene from different angles. Images of the scene are captured by the camera under the different illuminations. The images are analyzed to determine which regions in the scene exhibit changes in reflected luminance that correspond to the changes in illumination, indicating a specular surface. Regions containing diffuse surfaces are also identified based on such regions exhibiting a reflected luminance that is substantially independent of the changes in illumination.

Abstract: A computing device can detect first and second device motions within a first time period. The first and second device motions may be separated by a second time period. The computing device can determine that the second time period is within a specified time range. The computing device can determine that first and second motion metrics for the first and second device motions, respectively, are likely indicative of first and second taps, respectively, on a surface of the computing device. The computing device can determine that a first coordinate frame of the computing device associated with the first tap is within a range of angular degrees from a second coordinate frame of the computing device associated with the second tap. The computing device can further determine the first tap and the second tap as part of a double tap gesture.

Abstract: A device configured with a user interface (UI) that changes based on a position of a user determines the position of the user through multiple data sources including camera based head tracking and output from motion sensors such as a gyroscope. Each data source may output its own estimated head position. The device may apply a reliability weight to the head position determined by each data source. A composite head position is then determined from the weighted position. The composite position is then used to render the UI.

Abstract: Devices, systems and methods are disclosed for identifying content in video data and creating content-based zooming and panning effects to emphasize the content. Contents may be detected and analyzed in the video data using computer vision, machine learning algorithms or specified through a user interface. Panning and zooming controls may be associated with the contents, panning or zooming based on a location and size of content within the video data. The device may determine a number of pixels associated with content and may frame the content to be a certain percentage of the edited video data, such as a close-up shot where a subject is displayed as 50% of the viewing frame. The device may identify an event of interest, may determine multiple frames associated with the event of interest and may pan and zoom between the multiple frames based on a size/location of the content within the multiple frames.

Abstract: A device may identify a user among multiple individuals detected by comparing a physiological condition, such as a heart rate, of the multiple detected individuals as detected by a camera of the device with a physiological condition of a user of the device using different sensors. A heart rate may be detected by a camera by monitoring blood flow to an individual's face. A heart rate may be detected by a motion sensor by monitoring vibrations of the device that are in an expected frequency range. If a heart rate of a user matches a heart rate of an individual seen by a camera, that individual may be determined to be the user of the device. The position of the individual may be used to then render a user interface.

Abstract: A computing device can be controlled based on changes in the angle of a user's head with respect to the device, such as due to the user tilting the device and/or the user tilting his head with respect to the device. Such control based on the angle of the user's head can be achieved even when the user is operating the device “off-axis” or when the device is not orthogonal and/or not centered with respect to the user. This can be accomplished by using an elastic reference point that dynamically adjusts to a detected angle of the user's head with respect to the device. Such an approach can account for differences between when the user is changing his natural resting position and/or the resting position of the device and when the user is intending to perform a gesture based on the angle of the user's head relative to the device.