Abstract: Methods, systems, and apparatus, including computer programs encoded on computer-storage media, for obtaining a sample Light Detection and Ranging (LIDAR) profile generated by a drone; selecting a reference position based on the sample LIDAR profile; determining a LIDAR profile-based translation and rotation relative to a reference LIDAR profile of the reference position; determining an image-based translation and rotation relative to a reference image of the reference position; determining whether the LIDAR profile-based translation and rotation and the image-based translation and rotation satisfy a similarity threshold; and verifying, using a result of the determination, a predicted position of the drone.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for feature selection for object tracking. One of the methods includes: obtaining first feature points of an object in a first image of a scene captured by a camera; obtaining a second image of the scene captured by the camera after the first image was captured; determining whether a motion prediction of the object is available that indicates an area of the second image where the object is likely located; in response to determining that the motion prediction of the object is available, identifying, in the area of the second image where the object is likely located, second feature points that satisfy a similarity threshold for the first feature points in the first image; and detecting the object in the second image using the identified second feature points.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for anatomical feature-based medical predictions. One of the methods includes capturing a set of images depicting at least a portion of a subject body. The set of images are processed to extract a subset of images that satisfy one or more feature thresholds, such as thresholds related to anatomical pose or image quality. One or more target anatomical structures, such as craniofacial or oral cavity structures, are detected within the subset of images. One or more anatomical features are generated for the target anatomical structures, for instance, by fitting a three-dimensional model to the structures and computing geometric metrics. Using the generated anatomical features and stored medical data, a prediction is generated indicating a likelihood of a medical condition, such as obstructive sleep apnea or a comorbid condition, which can be transmitted to a device.

Abstract: Methods, systems, and apparatus for ground plane filtering of video events are disclosed. A method includes obtaining a first set of images of a scene from a camera; determining a ground plane from the first set of images of the scene; obtaining a second set of images of the scene after the first set of images of the scene is obtained; determining that movement shown by a group of pixels in the second set of images of the scene satisfies motion criteria; determining that the ground plane corresponds with at least a portion of the group of pixels; and in response to determining that movement shown by the group of pixels in the second set of images of the scene satisfies motion criteria, and that the ground plane corresponds with at least a portion of the group of pixels, classifying the group of pixels as showing ground plane based motion.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for robot navigation. One of the methods includes obtaining one or more images of an area from a robot; detecting two or more lines within the one or more images; identifying at least two of the two or more lines as vanishing lines; determining, using the vanishing lines, a correction maneuver; and controlling the robot to implement the correction maneuver.

Abstract: Methods, systems, and apparatus for camera detection of human activity with co-occurrence are disclosed. A method includes detecting a person in an image captured by a camera; in response to detecting the person in the image, determining optical flow in portions of a first set of images; determining that particular portions of the first set of images satisfy optical flow criteria; in response to determining that the particular portions of the first set of images satisfy optical flow criteria, classifying the particular portions of the first set of images as indicative of human activity; receiving a second set of images captured by the camera after the first set of images; and determining that the second set of images likely shows human activity based on analyzing portions of the second set of images that correspond to the particular portions of the first set of images classified as indicative of human activity.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for medical condition identification. One of the methods includes obtaining visual data representing at least one body part of a subject; obtaining non-visual data that corresponds to one or more biological characteristics of the subject; providing data representing (i) the visual data and (ii) the non-visual data to one or more machine learning models, wherein the one or more machine learning models are trained to predict presence of a medical condition; obtaining an output of the one or more machine learning models that is generated based on the one or more machine learning models processing the visual data and the non-visual data; and determining presence of the medical condition for the subject using the output of the one or more machine learning models.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for object detection. One of the methods includes determining, using first sensor data, a detection result on whether to trigger an event alerting a presence of an object in a target area by executing one or more models; determining, using second sensor data, a ground truth for the event that indicates whether an object is present in the target area; determining a difference value by comparing the detection result and the ground truth; adjusting at least one parameter of the one or more models in response to determining that the difference value does not satisfy the one or more threshold criteria; and determining a new detection result on whether to trigger a second event by executing the one or more models with adjusted parameters using new first sensor data.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for object detection. One of the methods includes determining, using first sensor data, a detection result on whether to trigger an event alerting a presence of an object in a target area by executing one or more models; determining, using second sensor data, a ground truth for the event that indicates whether an object is present in the target area; determining a difference value by comparing the detection result and the ground truth; adjusting at least one parameter of the one or more models in response to determining that the difference value does not satisfy the one or more threshold criteria; and determining a new detection result on whether to trigger a second event by executing the one or more models with adjusted parameters using new first sensor data.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for improving visual inertial odometry (VIO). One of the methods includes identifying two or more parameters of a robot; generating, using the two or more parameters, a multi-dimensional space; generating two or more configurations for the robot by sampling the multi-dimensional space; determining, for each of the two or more configurations, a visual inertial odometry (VIO) trajectory; generating, for each of the trajectories using the corresponding trajectory and a ground truth trajectory, (i) error data representing a difference of the corresponding trajectory from the ground truth trajectory and (ii) processing data representing processing metrics from the determination of the corresponding trajectory; selecting, using (i) the error data and (ii) the processing data, a configuration of the two or more configurations; and providing, to the robot, the selected configuration for navigating an area.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for detecting packages delivering in the camera's blindspot. One of the methods includes detecting, using one or more images captured by a camera at a property, movement in an area of interest i) at the property, ii) that is included in a field of view of the camera and iii) was generated using historical data for packages delivered to the property; determining, using the detected movement in the area of interest, that a package was likely delivered; and in response to determining that the package was likely delivered, performing one or more automated actions for the package.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for detecting packages delivering in the camera's blindspot. One of the methods includes detecting, using one or more images captured by a camera at a property, movement in an area of interest i) at the property, ii) that is included in a field of view of the camera and iii) was generated using historical data for packages delivered to the property; determining, using the detected movement in the area of interest, that a package was likely delivered; and in response to determining that the package was likely delivered, performing one or more automated actions for the package.

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for retroactive event detection. The methods, systems, and apparatus include actions of obtaining an image captured by a camera at a current time, determining that the image depicts a change in a region from a previous image captured by the camera at a previous time, determining, based on determining that the image depicts the change in the region, whether the change depicted in the image is of a known object type, determining, based on the determination that the change depicted in the image is of a known object type, whether the change does not correspond to a previously detected event, and determining, based on the determination that the change does not correspond to a previously detected event, whether the images captured by the camera between the current time and the previous time depict an event.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for monitoring events using a Virtual Inductive Loop system. In some implementations, image data is obtained from cameras. A region depicted in the obtained image data is identified, the region comprising lines spaced by a distance that satisfies a distance threshold. For each line included in the region: an object depicted crossing the line is determined whether to satisfy a height criteria indicating that the line is activated. In response to determining that an object depicted crossing the line satisfies the height criteria, an event is determined to have likely occurred using data indicating (i) which lines of the lines were activated and (ii) an order in which each of the lines were activated. In response to determining that an event likely occurred, actions are performed using at least some of the data.

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for retroactive event detection. The methods, systems, and apparatus include actions of obtaining an image captured by a camera at a current time, determining that the image depicts a change in a region from a previous image captured by the camera at a previous time, determining, based on determining that the image depicts the change in the region, whether the change depicted in the image is of a known object type, determining, based on the determination that the change depicted in the image is of a known object type, whether the change does not correspond to a previously detected event, and determining, based on the determination that the change does not correspond to a previously detected event, whether the images captured by the camera between the current time and the previous time depict an event.

Abstract: Methods and systems, including computer programs encoded on a storage medium, are described for implementing item monitoring using a doorbell camera. A system generates an input video stream that has image frames corresponding to detection of activity at a property. Timing information is generated for the video stream and includes a timestamp for each image frame of the stream. Using the timing information, the system processes a pre-event image frame that precedes detection of the activity and a post-event image frame that coincides with detection of the activity. An image score is computed with respect to placement of a candidate item at the property in response to processing the pre-event and post-event image frames. The image score is used to determine that a first item was delivered to the property or that a second item was removed after being delivered to the property.

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for training an event detector. The methods, systems, and apparatus include actions of identifying a portion of a first interframe difference image that represents motion of an OI, determining that a second interframe difference image represents motion by a non-OI, combining the portion of the first interframe difference image and the second interframe difference image as a third interframe difference image labeled as motion of both an OI and a non-OI, and training an event detector with the third interframe difference image.

Abstract: Disclosed are methods, systems, and apparatus for adjusting areas of interest for motion detection in camera scenes. A method includes obtaining a map of false motion event detections using a first area of interest; identifying an overlap area between the map of false detections and the first area of interest; determining a second area of interest that includes portions of the first area of interest and excludes at least a part of the overlap area; obtaining a map of true motion event detections using the first area of interest; determining whether true detections using the second area of interest compared to true detections using the first area of interest satisfies performance criteria; and in response to determining that true detections using the second area of interest compared to true detections using the first area of interest satisfies performance criteria, providing the second area of interest for use in detecting events.

Abstract: Disclosed are methods, systems, and apparatus for adjusting areas of interest for motion detection in camera scenes. A method includes obtaining a map of false motion event detections using a first area of interest; identifying an overlap area between the map of false detections and the first area of interest; determining a second area of interest that includes portions of the first area of interest and excludes at least a part of the overlap area; obtaining a map of true motion event detections using the first area of interest; determining whether true detections using the second area of interest compared to true detections using the first area of interest satisfies performance criteria; and in response to determining that true detections using the second area of interest compared to true detections using the first area of interest satisfies performance criteria, providing the second area of interest for use in detecting events.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for using feature descriptors to track objects depicted in images. One of the methods includes receiving hue, saturation, value data for an image and data that indicates an object detected in the image, generating a feature descriptor that includes hue data and saturation data, determining, for each of two or more tracked objects that each have a historical feature descriptor that includes historical hue data and historical saturation data, a distance between (i) the respective historical feature descriptor and (ii) the feature descriptor, associating the feature descriptor for the object with a tracked object from the two or more tracked objects, and tracking the tracked object in one or more images from a video sequence using the feature descriptor and the historical feature descriptor.