Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for using skill templates for robotic demonstration learning. One of the methods includes receiving a skill template for a task to be performed by a robot, wherein the skill template defines a state machine having a plurality of subtasks and one or more respective transition conditions between one or more of the subtasks. Local demonstration data for a demonstration subtask of the skill template is received, where the local demonstration data is generated from a user demonstrating how to perform the demonstration subtask with the robot. A machine learning model is refined for the demonstration subtask and the skill template is executed on the robot, causing the robot to transition through the state machine defined by the skill template to perform the task.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for using simulated local demonstration data for robotic demonstration learning. One of the methods includes receiving perceptual data of a workcell of a robot to be configured to execute a task according to a skill template, wherein the skill template specifies one or more subtasks required to perform the skill, wherein at least one of the subtasks is a demonstration subtask that relies on learning visual characteristics of the workcell. A virtual model is generated of a portion of the workcell. A training system generates simulated local demonstration data from the virtual model of the portion of the workcell and tunes a base control policy for the demonstration subtask using the simulated local demonstration data generated from the virtual model of the portion of the workcell.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for controlling a robot using symbolic states. One of the methods includes receiving a definition of a task having multiple task states, wherein each task state is associated with a different respective control policy; executing the task using an initial control policy associated with an initial task state; during execution of the task, continually generating, from sensor data, a prediction of the task state of the task; and upon determining that a transition to a different task state has occurred, transitioning the robot to a different control policy associated with the different task state.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for training an autoencoder. One of the methods includes receiving a pair of images, wherein a first image of the pair represents a first state of a working environment at a first time step earlier than that of a second image of the pair. A first and a second latent representation is generated respectively for the first and the second image by the autoencoder; A predicted reward for an action executed in the first state is generated by a reward prediction neural network. A predicted next latent representation for the first state and the action is generated by a dynamics prediction neural network. An overall loss is determined based on the predicted reward, the predicted next latent representation, and the second latent representation. Model parameters of the autoencoder are updated to reduce the overall loss.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for using simulated local demonstration data for robotic demonstration learning. One of the methods includes receiving perceptual data of a workcell of a robot to be configured to execute a task according to a skill template, wherein the skill template specifies one or more subtasks required to perform the skill, wherein at least one of the subtasks is a demonstration subtask that relies on learning visual characteristics of the workcell. A virtual model is generated of a portion of the workcell. A training system generates simulated local demonstration data from the virtual model of the portion of the workcell and tunes a base control policy for the demonstration subtask using the simulated local demonstration data generated from the virtual model of the portion of the workcell.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for distributing skill templates for robotic demonstration learning. One of the methods includes receiving from the user device by a skill template distribution system, a selection of an available skill template. The skill template distribution system provides a skill template, wherein the skill template comprises information representing a state machine of one or more tasks, and wherein the skill template specifies which of the one or more tasks are demonstration subtasks requiring local demonstration data. The skill template distribution system trains a machine learning model for the demonstration subtask using a local demonstration data to generate learned parameter values.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for using a demonstration device for robotic demonstration learning. One of the methods includes generating, by a demonstration device for a robot, a representation of a sequence of states input by a user of the demonstration device. The representation is provided by the demonstration device to a robot execution system. The representation of the sequence of actions is translated into a plurality of robot commands corresponding to the representation of the sequence of states input by the user on the demonstration device. The plurality of robot commands corresponding to the sequence of actions input by the user on the demonstration device are executed. Demonstration data is generated from one or more sensor streams of the robot while executing the plurality of robot commands corresponding to the sequence of actions input by the user on the demonstration device.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for robotic control using demonstrations to learn category-level manipulation task. One of the methods includes obtaining a collection of object models for a plurality of different types of objects belonging to a same object category and training a category-level representation in a category-level space from the collection of object models. A category-level trajectory is generated the demonstration data of a demonstration object. For a new object in the object category, a trajectory projection is generated in the category-level space, which is used to cause a robot to perform the robotic manipulation task on the new object.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for distributing skill templates for robotic demonstration learning. One of the methods includes receiving, from the user device by a skill template distribution system, a selection of an available skill template. The skill template distribution system provides a skill template, wherein the skill template comprises information representing a state machine of one or more tasks, and wherein the skill template specifies which of the one or more tasks are demonstration subtasks requiring local demonstration data. The skill template distribution system trains a machine learning model for the demonstration subtask using a local demonstration data to generate learned parameter values.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for distributed robotic demonstration learning. One of the methods includes receiving a skill template to be trained to cause a robot to perform a particular skill having a plurality of subtasks. One or more demonstration subtasks defined by the skill template are identified, wherein each demonstration subtask is an action to be refined using local demonstration data. On online execution system uploads sets of local demonstration data to a cloud-based training system. The cloud-based training system generates respective trained model parameters for each set of local demonstration data. The skill template is executed on the robot using the trained model parameters generated by the cloud-based training system.

Abstract: A cleaning device includes: a treatment container; a workpiece carrier arranged in the treatment container and configured to hold at least one workpiece; at least one nozzle configured to discharge a cleaning jet directed onto the workpiece carrier and mounted such that the at least one nozzle is moveable on a circulation track about the workpiece carrier and is pivotable about a pivoting axis extending parallel to an axis of rotation of the workpiece carrier; a pivoting device configured to pivot the at least one nozzle; and a controller configured to control a circulating movement of the at least one nozzle on the circulation track and a pivoting movement of the at least one nozzle, such that a specified point on a surface of the workpiece is impacted repeatedly by the cleaning jet at a respectively different angle, within a specified timeframe.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for using a demonstration device for robotic demonstration learning. One of the methods includes generating, by a demonstration device for a robot, a representation of a sequence of states input by a user of the demonstration device. The representation is provided by the demonstration device to a robot execution system. The representation of the sequence of actions is translated into a plurality of robot commands corresponding to the representation of the sequence of states input by the user on the demonstration device. The plurality of robot commands corresponding to the sequence of actions input by the user on the demonstration device are executed. Demonstration data is generated from one or more sensor streams of the robot while executing the plurality of robot commands corresponding to the sequence of actions input by the user on the demonstration device.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for training a robotic control policy to perform a particular task. One of the methods includes performing a meta reinforcement learning phase including using training data collected for a plurality of different robotic control tasks and updating a robotic control policy according to the training data, wherein the robotic control policy is conditioned on an encoder network that is trained to predict which task is being performed from a context of a robotic operating environment; and performing an adaptation phase using a plurality of demonstrations for the particular task, including iteratively updating the encoder network after processing each demonstration of the plurality of demonstrations, thereby training the encoder network to learn environmental features of successful task runs.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for integrating sensor streams for robotic demonstration learning. One of the methods includes selecting, by a learning system for a robot, a base update rate for combining multiple sensor streams into a task state representation. The learning system repeatedly generates the task state representation at the base update rate, including combining, during each time period defined by the update rate, the task state representation from most recently updated sensor data processed by the plurality of neural networks. The learning system repeatedly uses the task state representations to generate commands for the robot at the base update rate.

Abstract: A computer-implemented method comprising, receiving data representing a successful trajectory for an insertion task using a robot to insert a connector into a receptacle, performing a parameter optimization process for the robot to perform the insertion task. This parameter optimization includes defining an objective function that measures a similarity of a current trajectory generated with a current set of parameters to the successful trajectory and repeatedly modifying the current set of parameters and evaluating the modified set of parameters according to the objective function until generating a final set of parameters.

Abstract: A computer-implemented method includes obtaining a collection of object models for a plurality of different types of objects belonging to a same object category, generating a canonical representation for objects belonging to the object category, performing a plurality of downstream tasks using a plurality of different robot grasps on instances of objects belonging to the category and evaluating each grasp according to success or failure of the downstream task; and generating one or more category-level grasping areas for the canonical representation for objects belonging to the object category including aggregating the evaluations of grasps according to the downstream task.

Abstract: A cleaning device includes: a treatment container; a workpiece carrier arranged in the treatment container and configured to hold at least one workpiece; at least one nozzle configured to discharge a cleaning jet directed onto the workpiece carrier and mounted such that the at least one nozzle is moveable on a circulation track about the workpiece carrier and is pivotable about a pivoting axis extending parallel to an axis of rotation of the workpiece carrier; a pivoting device configured to pivot the at least one nozzle; and a controller configured to control a circulating movement of the at least one nozzle on the circulation track and a pivoting movement of the at least one nozzle, such that a specified point on a surface of the workpiece is impacted repeatedly by the cleaning jet at a respectively different angle, within a specified timeframe.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for using a demonstration device for robotic demonstration learning. One of the methods includes generating, by a demonstration device for a robot, a representation of a sequence of states input by a user of the demonstration device. The representation is provided by the demonstration device to a robot execution system. The representation of the sequence of actions is translated into a plurality of robot commands corresponding to the representation of the sequence of states input by the user on the demonstration device. The plurality of robot commands corresponding to the sequence of actions input by the user on the demonstration device are executed. Demonstration data is generated from one or more sensor streams of the robot while executing the plurality of robot commands corresponding to the sequence of actions input by the user on the demonstration device.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for using a demonstration device for robotic demonstration learning. One of the methods includes generating, by a demonstration device for a robot, a representation of a sequence of states input by a user of the demonstration device. The representation is provided by the demonstration device to a robot execution system. The representation of the sequence of actions is translated into a plurality of robot commands corresponding to the representation of the sequence of states input by the user on the demonstration device. The plurality of robot commands corresponding to the sequence of actions input by the user on the demonstration device are executed. Demonstration data is generated from one or more sensor streams of the robot while executing the plurality of robot commands corresponding to the sequence of actions input by the user on the demonstration device.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for integrating sensor streams for robotic demonstration learning. One of the methods includes selecting, by a learning system for a robot, a base update rate for combining multiple sensor streams into a task state representation. The learning system repeatedly generates the task state representation at the base update rate, including combining, during each time period defined by the update rate, the task state representation from most recently updated sensor data processed by the plurality of neural networks. The learning system repeatedly uses the task state representations to generate commands for the robot at the base update rate.