Abstract: Movement of an autonomously motile device may be controlled by a user device. The user device may display image data captured by a camera of the autonomously motile device; a user may provide input, such as a touch gesture on a display screen, indicating a command for the autonomously motile device to move to a location indicated by the input. The autonomously motile device determines a coordinate of the input and a time of the touch input; the autonomously motile device then determines a direction and distance of a corresponding movement.

Abstract: An autonomous mobile device (AMD) uses sensors to explore a physical space and determine the locations of obstacles. Simultaneous localization and mapping (SLAM) techniques are used by the AMD to designate as keyframes some images and their associated descriptors of features in the space. Each keyframe indicates a location and orientation of the AMD relative to those features. Anchors are specified relative to keyframes. A marker is specified relative to one or more anchors. Because markers are associated with features in the physical space, they maintain their association with the physical space through various processes such as SLAM loop closures. Markers may specify locations in the physical space, such as navigation waypoints, navigation destinations such as a goal pose for exploring an unexplored area, as an observation target to facilitate exploration, and so forth. Markers may also be used to specify block listed locations to be avoided during exploration.

Abstract: An autonomous mobile device moves through a physical space using simultaneous localization and mapping (SLAM) techniques. SLAM processes images from cameras to determine localization and trajectory of the device based on features that are assumed to be stationary. SLAM performance is improved by removing moving features from consideration. A first position of a feature at a first time and data from an inertial sensor are used to determine a predicted position at a second time. The predicted position is compared to a second position of the feature at the second time. This comparison takes into consideration an assumed Gaussian error distribution of how the positions are determined. If the predicted position differs from the second position by less than a threshold value, the feature may be determined to be stationary. The stationary features are then processed using SLAM to determine the localization and trajectory information.

Abstract: Movement of an autonomously motile device may be controlled by a user device. The user device may display image data captured by a camera of the autonomously motile device; a user may provide input, such as a touch gesture on a display screen, indicating a command for the autonomously motile device to move to a location indicated by the input. The autonomously motile device determines a coordinate of the input and a time of the touch input; the autonomously motile device then determines a direction and distance of a corresponding movement.

Abstract: A point cloud or map of an environment is generated by determining sets of landmark points from multiple images, such as through use of a Simultaneous Localization and Mapping (SLAM) algorithm. Images acquired using a depth camera are used to determine depth points indicative of at least one object not represented by the landmark points. A combined map is generated to include both the landmark points and the depth points. The set of depth points is mapped to a corresponding set of landmark points based on proximity of the points, similarity of the camera poses, or times that the images were acquired. The relationship between the depth and landmark points may be determined. When the landmark points are moved, such as to account for error in the SLAM algorithm, the depth points may be moved to a modified location relative to the landmark points so that the relationship remains constant.

Abstract: Various examples are directed to systems and methods for detection motion in video data. An image processor may receive video data. A first frame and a second frame may describe a plurality of pixel locations. The second frame may be positioned after the first frame in the frame sequence. The image processor may determine that a first subset of pixel locations from the plurality of pixel locations differ by less than a threshold amount between frames for at least a threshold number of frames in the frame sequence prior to the first frame. The image processor may identify from the first subset of pixel locations, a second subset of pixel locations that differ by more than the threshold amount between the first frame and the second frame and determine that a number of pixel locations in the second subset of pixel locations is greater than a motion pixel threshold.

Abstract: The techniques described herein may identify images that likely depict one or more items by comparing features of the items to features of different regions-of-interest (ROIs) of the images. For instance, some of the images may include a user, and the techniques may define multiple regions within the image corresponding to different portions of the user. The techniques may then use a trained convolutional neural network or any other type of trained classifier to determine, for each region of the image, whether the region depicts a particular item. If so, the techniques may designate the corresponding image as depicting the item and may output an indication that the image depicts the item. The techniques may perform this process for multiple images, outputting an indication of each image deemed to depict the particular item.

Abstract: The techniques described herein may identify images that likely depict one or more items by comparing features of the items to features of different regions-of-interest (ROIs) of the images. When a user requests to identify images that depict a particular item, the techniques may determine a region-of-interest (ROI) size based on the size of the requested item. The techniques may then search multiple images using the ROI size.

Abstract: Systems and approaches are provided for robustly detecting and tracking a user. Image data can be captured and processed to provide an estimated position and/or orientation of the user. Other sensor data, such as from an accelerometer and/or gyroscope, can be determined for a more robust estimation of the user's position and/or orientation. Multiple user detection processes and/or motion estimation approaches and their corresponding confidence levels can also be combined to determine a final estimated position and orientation of the user. The multiple user pose estimations and/or motion estimations can be combined via an approach such as probabilistic system modeling and maximum likelihood estimation.

Abstract: A tilt-based interface controls the position of a cursor on a graphical user interface (GUI) for mobile devices based on changes in the orientation of the device. Cursor positioning and user interface element selection may be accomplished without touching the screen or clicking a button to indicate the user's selection. The underlying coordinate system to which cursor movement is mapped is transformed to alter the speed and behavior of the cursor when the cursor reaches a user-selectable interface elements.

Abstract: Systems and approaches are provided for tracking an object using multiple tracking processes. By combining multiple lightweight tracking processes, object tracking can be robust, use a limited amount of power, and enable a computing device to respond to input corresponding to the motion of the object in real time. The multiple tracking processes can be run in parallel to determine the position of the object by selecting the results of the best performing tracker under certain heuristics or combining the results of multiple tracking processes in various ways. Further, other sensor data of a computing device can be used to improve the results provided by one or more of the tracking processes.

Abstract: A tilt-based interface controls the position of a cursor on a graphical user interface (GUI) for mobile devices based on changes in the orientation of the device. Cursor positioning and user interface element selection may be accomplished without touching the screen or clicking a button to indicate the user's selection. The underlying coordinate system to which cursor movement is mapped is transformed to alter the speed and behavior of the cursor when the cursor reaches a user-selectable interface elements.

Abstract: Systems and approaches are provided for tracking an object using multiple tracking processes. By combining multiple lightweight tracking processes, object tracking can be robust, use a limited amount of power, and enable a computing device to respond to input corresponding to the motion of the object in real time. The multiple tracking processes can be run in parallel to determine the position of the object by selecting the results of the best performing tracker under certain heuristics or combining the results of multiple tracking processes in various ways. Further, other sensor data of a computing device can be used to improve the results provided by one or more of the tracking processes.