Abstract: The present disclosure provides a method for calibrating a humanoid robot, comprising obtaining a humanoid robot with original kinematic biasing values, controlling the humanoid robot through predetermined poses, capturing image data of body parts using vision sensors mounted on the humanoid robot while moving through the poses, determining revised kinematic biasing values by processing the image data using a bipedal spatial perception model trained using synthetic image data containing keypoints, and replacing the original kinematic biasing values with the revised kinematic biasing values. The bipedal spatial perception model processes captured image data to generate observed keypoint locations on robot components, which are compared with kinematic-based locations from joint encoder measurements to minimize discrepancies through optimization algorithms.

Abstract: A humanoid robot system comprises vision sensors for capturing image data, a computing architecture with processing hardware and memory, and a bipedal spatial perception model. The model includes a feature extractor that extracts hierarchical feature maps from input images, a robot data module that detects robot parts, and a robot vector data module that calculates three-dimensional spatial position and orientation data for each detected robot part. The feature extractor uses a feature pyramid network generating multi-scale feature maps through bottom-up and top-down pathways with lateral connections. The robot vector data module predicts 2D-to-3D point correspondences and solves perspective-n-point problems to obtain final position and orientation vectors, enabling real-time robot self-awareness and closed-loop visual servoing for precise object interaction.

Abstract: Systems and methods are described for detecting changes at a location based on image data by a mobile robot. A system can instruct navigation of the mobile robot to a location. For example, the system can instruct navigation to the location as part of an inspection mission. The system can obtain input identifying a change detection. Based on the change detection and obtained image data associated with the location, the system can perform the change detection and detect a change associated with the location. For example, the system can perform the change detection based on one or more regions of interest of the obtained image data. Based on the detected change and a reference model, the system can determine presence of an anomaly condition in the obtained image data.

Abstract: A computer-implemented method executed by data processing hardware of a robot causes the data processing hardware to perform operations. The operations include receiving a sensor pointing command that commands the robot to use a sensor to capture sensor data of a location in an environment of the robot. The sensor is disposed on the robot. The operations include determining, based on an orientation of the sensor relative to the location, a direction for pointing the sensor toward the location, and an alignment pose of the robot to cause the sensor to point in the direction toward the location. The operations include commanding the robot to move from a current pose to the alignment pose. After the robot moves to the alignment pose and the sensor is pointing in the direction toward the location, the operations include commanding the sensor to capture the sensor data of the location in the environment.

Abstract: A computer-implemented method executed by data processing hardware of a robot causes the data processing hardware to perform operations. The operations include receiving a sensor pointing command that commands the robot to use a sensor to capture sensor data of a location in an environment of the robot. The sensor is disposed on the robot. The operations include determining, based on an orientation of the sensor relative to the location, a direction for pointing the sensor toward the location, and an alignment pose of the robot to cause the sensor to point in the direction toward the location. The operations include commanding the robot to move from a current pose to the alignment pose. After the robot moves to the alignment pose and the sensor is pointing in the direction toward the location, the operations include commanding the sensor to capture the sensor data of the location in the environment.

Abstract: A method of constrained mobility mapping includes receiving from at least one sensor of a robot at least one original set of sensor data and a current set of sensor data. Here, each of the at least one original set of sensor data and the current set of sensor data corresponds to an environment about the robot. The method further includes generating a voxel map including a plurality of voxels based on the at least one original set of sensor data. The method also includes generating a spherical depth map based on the current set of sensor data and determining that a change has occurred to an obstacle represented by the voxel map based on a comparison between the voxel map and the spherical depth map. The method additional includes updating the voxel map to reflect the change to the obstacle.

Abstract: A method of constrained mobility mapping includes receiving from at least one sensor of a robot at least one original set of sensor data and a current set of sensor data. Here, each of the at least one original set of sensor data and the current set of sensor data corresponds to an environment about the robot. The method further includes generating a voxel map including a plurality of voxels based on the at least one original set of sensor data. The plurality of voxels includes at least one ground voxel and at least one obstacle voxel. The method also includes generating a spherical depth map based on the current set of sensor data and determining that a change has occurred to an obstacle represented by the voxel map based on a comparison between the voxel map and the spherical depth map. The method additional includes updating the voxel map to reflect the change to the obstacle.

Abstract: Systems and methods are described for instructing performance of a localization and an action by a mobile robot based on composite data. A system may obtain satellite-based position data and one or more of odometry data or point cloud data. The system may generate composite data by merging the satellite-based position data and the one or more of the odometry data or the point cloud data. The system may instruct performance of a localization by the mobile robot based on the composite data. Based on the localization by the mobile robot, the system may identify an action and instruct performance of the action by a mobile robot.

Abstract: A computer-implemented method when executed by data processing hardware of a legged robot causes the data processing hardware to perform operations including receiving sensor data corresponding to an area including at least a portion of a docking station. The operations include determining an estimated pose for the docking station based on an initial pose of the legged robot relative to the docking station. The operations include identifying one or more docking station features from the received sensor data. The operations include matching the one or more identified docking station features to one or more known docking station features. The operations include adjusting the estimated pose for the docking station to a corrected pose for the docking station based on an orientation of the one or more identified docking station features that match the one or more known docking station features.

Abstract: A computer-implemented method when executed by data processing hardware of a legged robot causes the data processing hardware to perform operations including receiving sensor data corresponding to an area including at least a portion of a docking station. The operations include determining an estimated pose for the docking station based on an initial pose of the legged robot relative to the docking station. The operations include identifying one or more docking station features from the received sensor data. The operations include matching the one or more identified docking station features to one or more known docking station features. The operations include adjusting the estimated pose for the docking station to a corrected pose for the docking station based on an orientation of the one or more identified docking station features that match the one or more known docking station features.

Abstract: Systems and methods are described for detecting changes at a location based on image data by a mobile robot. A system can instruct navigation of the mobile robot to a location. For example, the system can instruct navigation to the location as part of an inspection mission. The system can obtain input identifying a change detection. Based on the change detection and obtained image data associated with the location, the system can perform the change detection and detect a change associated with the location. For example, the system can perform the change detection based on one or more regions of interest of the obtained image data. Based on the detected change and a reference model, the system can determine presence of an anomaly condition in the obtained image data.

Abstract: Systems and methods are described for reacting to a feature in an environment of a robot based on a classification of the feature. A system can detect the feature in the environment using a first sensor on the robot. For example, the system can detect the feature using a feature detection system based on sensor data from a camera. The system can detect a mover in the environment using a second sensor on the robot. For example, the system can detect the mover using a mover detection system based on sensor data from a lidar sensor. The system can fuse the data from detecting the feature and detecting the mover to produce fused data. The system can classify the feature based on the fused data and react to the feature based on classifying the feature.

Abstract: A computer-implemented method executed by data processing hardware of a robot causes the data processing hardware to perform operations. The operations include receiving a sensor pointing command that commands the robot to use a sensor to capture sensor data of a location in an environment of the robot. The sensor is disposed on the robot. The operations include determining, based on an orientation of the sensor relative to the location, a direction for pointing the sensor toward the location, and an alignment pose of the robot to cause the sensor to point in the direction toward the location. The operations include commanding the robot to move from a current pose to the alignment pose. After the robot moves to the alignment pose and the sensor is pointing in the direction toward the location, the operations include commanding the sensor to capture the sensor data of the location in the environment.

Abstract: A method of constrained mobility mapping includes receiving from at least one sensor of a robot at least one original set of sensor data and a current set of sensor data. Here, each of the at least one original set of sensor data and the current set of sensor data corresponds to an environment about the robot. The method further includes generating a voxel map including a plurality of voxels based on the at least one original set of sensor data. The plurality of voxels includes at least one ground voxel and at least one obstacle voxel. The method also includes generating a spherical depth map based on the current set of sensor data and determining that a change has occurred to an obstacle represented by the voxel map based on a comparison between the voxel map and the spherical depth map. The method additional includes updating the voxel map to reflect the change to the obstacle.

Abstract: A method of constrained mobility mapping includes receiving from at least one sensor of a robot at least one original set of sensor data and a current set of sensor data. Here, each of the at least one original set of sensor data and the current set of sensor data corresponds to an environment about the robot. The method further includes generating a voxel map including a plurality of voxels based on the at least one original set of sensor data. The plurality of voxels includes at least one ground voxel and at least one obstacle voxel. The method also includes generating a spherical depth map based on the current set of sensor data and determining that a change has occurred to an obstacle represented by the voxel map based on a comparison between the voxel map and the spherical depth map. The method additional includes updating the voxel map to reflect the change to the obstacle.

Abstract: A computer-implemented method when executed by data processing hardware of a legged robot causes the data processing hardware to perform operations including receiving sensor data corresponding to an area including at least a portion of a docking station. The operations include determining an estimated pose for the docking station based on an initial pose of the legged robot relative to the docking station. The operations include identifying one or more docking station features from the received sensor data. The operations include matching the one or more identified docking station features to one or more known docking station features. The operations include adjusting the estimated pose for the docking station to a corrected pose for the docking station based on an orientation of the one or more identified docking station features that match the one or more known docking station features.

Abstract: A method of constrained mobility mapping includes receiving from at least one sensor of a robot at least one original set of sensor data and a current set of sensor data. Here, each of the at least one original set of sensor data and the current set of sensor data corresponds to an environment about the robot. The method further includes generating a voxel map including a plurality of voxels based on the at least one original set of sensor data. The plurality of voxels includes at least one ground voxel and at least one obstacle voxel. The method also includes generating a spherical depth map based on the current set of sensor data and determining that a change has occurred to an obstacle represented by the voxel map based on a comparison between the voxel map and the spherical depth map. The method additional includes updating the voxel map to reflect the change to the obstacle.

Abstract: A method for imaging a moving object includes scanning a predetermined area with at least one distance sensor to form an image of a structure of a moving object using a safe sensing time window to periodically refresh the image. The images of the structure are compared to a known model of the structure to estimate rates of motion. A refined time window is determined based on the estimated rates of motion to monitor the moving object with increased accuracy and/or range compared to the safe time window.

Abstract: According to an aspect of the invention, a method of optimal safe landing area determination for an aircraft includes accessing a probabilistic safe landing area map that includes a plurality of probabilistic indicators of safe landing areas for the aircraft. A processing subsystem that includes one or more processing resources generates a list of candidate safe landing areas based on the probabilistic safe landing area map and one or more constraints. At least two of the candidate safe landing areas are provided to a path planner. The list of candidate safe landing areas is ranked based on results from the path planner indicating an estimated cost to reach each of the candidate safe landing areas. Based on the ranking, an indicator of an optimal safe landing area is output as a desired landing location for the aircraft.

Abstract: A system for registering a target includes a first sensor, a second sensor, and a processor. The first sensor measures a plurality of ranges from a source to a target, and the second sensor obtains a plurality of location measurements of the source. The system further includes a processor configured for determining one or more weighting criteria associated with each one of the plurality of location measurements based on an estimated reliability of each one of the plurality of location measurements. The processor calculates a plurality of target location values based on the plurality of ranges measured by the first sensor and the plurality of locations measured by the second sensor and calculates an estimated target location value based on the plurality of target location values weighted according to the weighting criteria.