Abstract: A method for estimating a pose of a deformable object includes: receiving, by a processor, a plurality of images depicting the deformable object from multiple viewpoints; computing, by the processor, one or more object-level correspondences and a class of the deformable object depicted in the images; loading, by the processor, a 3-D model corresponding to the class of the deformable object; aligning, by the processor, the 3-D model to the deformable object depicted in the plurality of images to compute a six-degree of freedom (6-DoF) pose of the object; and outputting, by the processor, the 3-D model and the 6-DoF pose of the object.

Abstract: An optical system includes: a beam splitter system configured to split an input beam into a plurality of output beams including a first output beam, a second output beam, and a third output beam; a first polarizing filter having a first polarization angle and configured to filter the first output beam to produce a first filtered output beam; a second polarizing filter having a second angle of polarization and configured to filter the second output beam to produce a second filtered output beam; and a third polarizing filter having a third angle of polarization and configured to filter the third output beam to produce a third filtered output beam, the first, second, and third angles of polarization being different from one another.

Abstract: The method for increasing the accuracy of grasping an object can include: labelling an image based on an attempted object grasp by a robot and generating a trained graspability network using the labelled images. The method can additionally or alternatively include determining a grasp point using the trained graspability network; executing an object grasp at the grasp point S400; and/or any other suitable elements.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for measuring and reporting calibration accuracy of robots and sensors assigned to perform a task in an operating environment. One of the methods includes receiving a request to perform a calibration process for one or more robots in an operating environment; in response, performing the calibration process including executing a calibration program that generates movement data representing movements by the one or more robots within the operating environment; computing a measure of calibration accuracy from the movement data; receiving an input program to be executed in the operating environment; determining that the measure of calibration accuracy does not satisfy an accuracy tolerance of the input program; and in response, generating a notification representing that the measure of calibration accuracy generated from performing the calibration process does not satisfy the accuracy tolerance of the input program.

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium that automatically calibrates robots and sensors assigned to perform a task in an operating environment. One of the methods includes obtaining a representation of a robotic operating environment. A user selection of a plurality of components to be configured to operate in the robotic operating environment is received. A mapping is obtained between pairs of components to be calibrated and one or more respective calibration processes to perform to calibrate each pair of components. From the mapping, one or more calibration processes to be performed on pairs of components based on the user selection of the plurality of components is computed. Calibration instruction data describing how to perform the one or more calibration processes to be performed on the pairs of components of the user selection is determined and presented.

Abstract: Techniques and systems are disclosed for using swept volume profile data cached in association with a PRM to improve various aspects of motion planning for a robot. In some implementations, a first probabilistic road map representing possible paths to be travelled by a robot within a physical area is generated. An initial path for the robot within the first probabilistic road map is determined. Data indicating a second probabilistic road map representing a path to be travelled by a movable object within the physical area is obtained. A potential obstruction associated with one or more edges included in the subset of edges is detected. An adjusted path for the robot within the first probabilistic road map is then determined based on the potential obstruction.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for performing transformation mode switching in a robotics control system.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for performing robot planning using a process definition graph. The techniques can include receiving a process definition graph having a plurality of task nodes that represent respective tasks to be performed by a respective robot of a plurality of robots, wherein each task node is associated with a location at which the task will be performed; generating, from the process definition graph, an initial modified process definition graph that adds constraints for respective swept volumes occupied by each task represented by the plurality of task nodes; and generating, from the initial modified process definition graph, a refined process definition graph, wherein the refined process definition graph includes respective motion plans for robots moving between tasks, wherein the motion plans define transitions that avoid the swept volumes occupied by each task represented by the plurality of task nodes.

Abstract: A computer-implemented method for computing a prediction on images of a scene includes: receiving one or more polarization raw frames of a scene, the polarization raw frames being captured with a polarizing filter at a different linear polarization angle; extracting one or more first tensors in one or more polarization representation spaces from the polarization raw frames; and computing a prediction regarding one or more optically challenging objects in the scene based on the one or more first tensors in the one or more polarization representation spaces.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for planning a path of motion for a robot. In some implementations, a candidate path of movement is determined for each of multiple robots. A swept region, for each of the multiple robots, is determined that the robot would traverse through along its candidate path. At least some of the swept regions for the multiple robots is aggregated to determine amounts of overlap among the swept regions at different locations. Force vectors directed outward from the swept regions are assigned, wherein the force vectors have different magnitudes assigned according to the respective amounts of overlap of the swept regions at the different locations. A path for a particular robot to travel is determined based on the swept regions and the assigned magnitudes of the forces.

Abstract: The system can include: a container no, a set of sensors 120, and a controller 130. The system can optionally include a robot 140. However, the system 100 can additionally or alternatively include any other suitable set of components. The system functions to monitor and/or maintain a fullness level of a container. The system can additionally or alternatively function to enable robotic picking out of the container (e.g., in a pick-and-place setting). The system can additionally function to maintain candidate objects within reach of the robot's end effector to increase robot uptime while minimizing the extent of the robot's required motion (e.g., in the z-axis).

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for using simulated local demonstration data for robotic demonstration learning. One of the methods includes receiving perceptual data of a workcell of a robot to be configured to execute a task according to a skill template, wherein the skill template specifies one or more subtasks required to perform the skill, wherein at least one of the subtasks is a demonstration subtask that relies on learning visual characteristics of the workcell. A virtual model is generated of a portion of the workcell. A training system generates simulated local demonstration data from the virtual model of the portion of the workcell and tunes a base control policy for the demonstration subtask using the simulated local demonstration data generated from the virtual model of the portion of the workcell.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for generating paths for a robot based on optimizing multiple objectives. One of the methods includes: receiving, by a motion planner, request to generate a path for a robot between a start point and an end point in a workcell of the robot, wherein the workcell is associated with one or more soft margin values that define spaces in which the robot should avoid when transitioning between points in the workcell; classifying path segments within the workcell as being inside the soft margin or outside the soft margin; generating a respective cost for each of the plurality of path segments within the workcell; generating a plurality of alternative paths; evaluating the plurality of alternative paths according to the respective costs; and selecting an alternative path based on respective total costs of the plurality of alternative paths.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for robot motion planning using unmanned aerial vehicles (UAVs). One of the methods includes determining that a current plan for performing a particular task with a robot requires modification; in response, generating one or more flight plans for an unmanned aerial vehicle (UAV) based on a robotic operating environment comprising the robot; obtaining, using the UAV in accordance with the one or more flight plans, a new measurement of the robotic operating environment comprising the robot; and generating, based at least on a difference between the new measurement of the robotic operating environment and a previous measurement of the robotic operating environment, a modified plan for performing the particular task with the robot.

Abstract: Systems and method for grasp execution using height maps.

Abstract: A method of generating synthetic images of virtual scenes includes: placing, by a synthetic data generator implemented by a processor and memory, three-dimensional (3-D) models of objects in a 3-D virtual scene; adding, by the synthetic data generator, lighting to the 3-D virtual scene, the lighting including one or more illumination sources; applying, by the synthetic data generator, imaging modality-specific materials to the 3-D models of objects in the 3-D virtual scene in accordance with a selected multimodal imaging modality, each of the imaging modality-specific materials including an empirical model; setting a scene background in accordance with the selected multimodal imaging modality; and rendering, by the synthetic data generator, a two-dimensional image of the 3-D virtual scene based on the selected multimodal imaging modality to generate a synthetic image in accordance with the selected multimodal imaging modality.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for performing robot planning using a process definition graph. One of the methods includes receiving an initial underconstrained process definition graph for one or more robots, wherein the process definition graph is a directed acyclic graph having constraint nodes and action nodes. A plurality of transformers are repeatedly applied to the initial process definition graph, wherein each application of a transformer generates a respective modified process definition graph according to the constraint nodes of the process definition graph, wherein applying the plurality of transformers generates a schedule that specifies which of the one or more robots are to perform which of one or more actions represented by actions nodes according to constraints imposed by the constraint nodes in the process definition graph.

Abstract: Techniques and systems are disclosed for using swept volume profile data cached in association with a PRM to improve various aspects of motion planning for a robot. In some implementations, a first probabilistic road map representing possible paths to be travelled by a robot within a physical area is generated. An initial path for the robot within the first probabilistic road map is determined. Data indicating a second probabilistic road map representing a path to be travelled by a movable object within the physical area is obtained. A potential obstruction associated with one or more edges included in the subset of edges is detected. An adjusted path for the robot within the first probabilistic road map is then determined based on the potential obstruction.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for controlling robotic movements. One of the methods includes receiving, for a robot, a definition of a plurality of sensor-based skills to be executed in sequence, wherein each skill is associated with an entry point and an exit point; generating a motion plan for the robot, including: generating, for a first skill of the plurality of sensor-based skills, a first path from a first entry point of the first skill to a second point at which a sensor-based interaction of the first skill begins, and generating, for the first skill of the plurality of sensor-based skills, a second path from a third point at which the sensor-based interaction of the first skill ends to a first exit point of the first skill; and executing the motion plan for the robot.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for using a demonstration device for robotic demonstration learning. One of the methods includes generating, by a demonstration device for a robot, a representation of a sequence of states input by a user of the demonstration device. The representation is provided by the demonstration device to a robot execution system. The representation of the sequence of actions is translated into a plurality of robot commands corresponding to the representation of the sequence of states input by the user on the demonstration device. The plurality of robot commands corresponding to the sequence of actions input by the user on the demonstration device are executed. Demonstration data is generated from one or more sensor streams of the robot while executing the plurality of robot commands corresponding to the sequence of actions input by the user on the demonstration device.