Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for avoiding singular configurations of a robot. A singular configuration of the robot is obtained. A location of an end effector of the robot when the robot is in the singular configuration is determined. For each of a plurality of voxels in a workcell, a distance from the voxel to the location of the end effector when the robot is in the singular configuration is computed. A negative potential gradient of the computed distance is computed. Control rules are generated, wherein the control rules, when followed by the robot, offset the trajectory of the robot according to the negative potential gradient.

Abstract: The method for increasing the accuracy of grasping an object can include: labelling an image based on an attempted object grasp by a robot and generating a trained graspability network using the labelled images. The method can additionally or alternatively include determining a grasp point using the trained graspability network; executing an object grasp at the grasp point S400; and/or any other suitable elements.

Abstract: Systems, methods, devices, and other techniques are described for planning motions of one or more robots to perform at least one specified task. In some implementations, a task to execute with a robotic system using a tool is identified. A partially constrained pose is identified for the tool that is to apply during execution of the task. A set of possible constraints for the unconstrained pose parameter are selected for each unconstrained pose parameter. The sets of possible constraints are evaluated for the unconstrained pose parameters with respect to one or more task execution criteria. A nominal pose is determined for the tool based on a result of evaluating the sets of possible constraints for the unconstrained pose parameters with respect to the one or more task execution criteria. The robotic system is then directed to execute the task, including positioning the tool according to the nominal pose.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for logging real-time data of a robot control system.

Abstract: A method for generating data explanations in a recursive cortical network includes receiving a set of evidence data at child feature nodes of a first layer of the recursive cortical network, setting a transformation configuration that directs messaging of evidence data and transformed data between layers of the network, performing a series of transformations on the evidence data according to the transformation configuration, the series including at least one forward transformation and at least one reverse transformation, and outputting the transformed evidence data.

Abstract: A method for object grasping, including: determining features of a scene; determining candidate grasp locations; determining a set of candidate grasp proposals for the candidate grasp locations; optionally modifying a candidate grasp proposal of the set; determining grasp scores associated with the candidate grasp proposals; selecting a set of final grasp proposals based on the grasp scores; and executing a grasp proposal from the set of final grasp proposals.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for integrating sensor streams for robotic demonstration learning. One of the methods includes selecting, by a learning system for a robot, a base update rate for combining multiple sensor streams into a task state representation. The learning system repeatedly generates the task state representation at the base update rate, including combining, during each time period defined by the update rate, the task state representation from most recently updated sensor data processed by the plurality of neural networks. The learning system repeatedly uses the task state representations to generate commands for the robot at the base update rate.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for performing robot plan online adjustment. A method includes receiving an initial plan for performing a particular task with a robot having a sensor. The initial plan defines an initial path having a plurality of waypoints. Each waypoint is associated with a target position and a target velocity. The method includes generating an alternative path from the initial path. Generating an alternative path includes generating a plurality of alternative paths including performing respective modifications to one or more waypoints in the initial plan, evaluating each alternative path according to a simulated total time duration required for the robot to traverse the alternative path, and selecting an alternative path having a total time duration that is less than a total time duration of the initial plan.

Abstract: A hierarchical compositional network, representable in Bayesian network form, includes first, second, third, fourth, and fifth parent feature nodes; first, second, and third pool nodes; first, second, and third weight nodes; and first, second, third, fourth, and fifth child feature nodes.

Abstract: A multi-modal sensor system includes: an underlying sensor system; a polarization camera system configured to capture polarization raw frames corresponding to a plurality of different polarization states; and a processing system including a processor and memory, the processing system being configured to control the underlying sensor system and the polarization camera system, the memory storing instructions that, when executed by the processor, cause the processor to: control the underlying sensor system to perform sensing on a scene and the polarization camera system to capture a plurality of polarization raw frames of the scene; extract first tensors in polarization representation spaces based on the plurality of polarization raw frames; and compute a characterization output based on an output of the underlying sensor system and the first tensors in polarization representation spaces.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for providing user feedback for robotic demonstration learning. One of the methods includes initiating a local demonstration learning process to collect respective local demonstration data for each of one or more demonstration subtasks defined by a skill template to be executed by a robot. Local demonstration data is repeatedly collected for each of the one or more demonstration subtasks of the skill template while a user manipulates a robot to perform each of the one or more demonstration subtasks defined by the skill template. A respective progress value for each of the one or more demonstration subtasks defined by the skill template is maintained. A user interface presentation is generated that presents a suggested demonstration to be performed by the user based on a respective progress value for each demonstration subtask.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer-storage media, for scheduling resource-constrained actions. In some implementations, data indicating a set of tasks to be performed by a group of multiple robots is received. A set of candidate plan elements is determined for each task in the set of tasks. A constraint profile for each of the candidate plan elements is generated, where each of the constraint profiles indicates a constraint to be satisfied in order to carry out the corresponding candidate plan element. Plan elements configured to perform each of the tasks in the set of tasks are selected based on the constraint profiles, and assembled into a schedule according to optimization criteria. This schedule can be used to perform tasks, and the schedule may account for variability in timing without failure.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for generating motions for components in a robotic operating environment. One of the methods includes receiving a request to generate a motion for a kinematic system having a plurality of connected entities. An entity-specific sampling module for each of multiple degree-of-freedom (DOF) groups representing respective entities of the kinematic system is identified. A plurality of joint configuration samples are generated according to an ordering of a plurality of nonfunctional DOF groups using a respective entity-specific sampling module for each nonfunctional DOF group. A final joint configuration sample is generated for one or more one or more control points using a respective entity-specific sampling module for a functional DOF group. A motion comprising a sequence of respective joint configuration samples from each of the plurality of DOF groups is generated.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for modifying a process definition to ensure accuracy, timeliness, or both of workcell measurement. One of the methods includes receiving an initial process definition for a process to be performed by a robot, wherein the process definition defines a sequence of actions to be performed in a workcell, and wherein a first action in the sequence of actions has an associated measurement tolerance; computing a predicted accumulated measurement variance for each of one or more actions that occur before the first action in the sequence; determining that the predicted accumulated measurement variance for the one or more actions that occur before the first action in the sequence exceeds a threshold; and in response, generating a modified process definition that inserts a measurement action at a location in the sequence before the first action.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for a composability framework that supports the coordination of the low-level actions of multiple subsystems.

Abstract: Embodiments are provided that include maintaining a map of a plurality of markers in an environment. The map includes a last detection time of each marker of the plurality of markers. The embodiments also include receiving a set of detected markers from a robotic device that is configured to localize in the environment using the plurality of markers. The embodiments further include updating, in the map, the last detection time of each marker which has a mapped position that corresponds to a detected position of a detected marker in the set of detected markers. The embodiments additionally include identifying, from the plurality of markers in the map, a marker having a last detection time older than a threshold amount of time. The embodiments still further include initiating an action related to the identified marker.

Abstract: In one aspect, there is provided a computer-implemented method that incudes obtaining physical sensor measurements of a physical robotic operating environment and obtaining a virtual representation of the robotic operating environment. The method further includes generating a user interface presentation including a first view of the virtual representation based virtual sensor parameters, and a second view of the physical robotic operating environment based on the physical sensor measurements. The method further includes receiving an update to values of the virtual sensor parameters, and updating, in the user interface presentation, the first view of the virtual representation based on the updated values of the virtual sensor parameters.

Abstract: Systems, methods, devices, and other techniques for a dynamic fabrication system. In some implementations, a computing system obtains a digital model of a physical structure. A fabrication plan for the physical structure is generated. The fabrication plan is provided to a fabrication system to execute the automated fabrication procedure according to the fabrication plan. A set of operations are performed for each of at least a subset of tasks from a set of tasks executed by the fabrication system during an automated fabrication procedure to identify an adjusted fabrication plan. The fabrication system is directed to continue execution of the automated fabrication procedure according to the adjusted fabrication plan.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for using a demonstration device for robotic demonstration learning. One of the methods includes generating, by a demonstration device for a robot, a representation of a sequence of states input by a user of the demonstration device. The representation is provided by the demonstration device to a robot execution system. The representation of the sequence of actions is translated into a plurality of robot commands corresponding to the representation of the sequence of states input by the user on the demonstration device. The plurality of robot commands corresponding to the sequence of actions input by the user on the demonstration device are executed. Demonstration data is generated from one or more sensor streams of the robot while executing the plurality of robot commands corresponding to the sequence of actions input by the user on the demonstration device.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for deblurring an image captured by a robot-mounted camera. One of the methods comprises capturing, using a camera that is attached to an arm of a robot, an image at a first time, wherein the image exhibits motion blur, and wherein the exhibited blur was caused by movement of the arm of the robot at the first time; receiving, from a robot control system of the robot, motion data characterizing the movement of the arm of the robot at the first time; generating a motion kernel using the received motion data; and generating a deblurred image by processing the image using the motion kernel.