Abstract: There are provided systems, methods, and apparatus, for optimizing a policy controller to control a robotic agent that interacts with an environment to perform a robotic task. One of the methods includes optimizing the policy controller using a neural network that generates numeric embeddings of images of the environment and a demonstration sequence of demonstration images of another agent performing a version of the robotic task.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for detecting intermediate reinforcement learning goals. One of the methods includes obtaining a plurality of demonstration sequences, each of the demonstration sequences being a sequence of images of an environment while a respective instance of a reinforcement learning task is being performed; for each demonstration sequence, processing each image in the demonstration sequence through an image processing neural network to determine feature values for a respective set of features for the image; determining, from the demonstration sequences, a partitioning of the reinforcement learning task into a plurality of subtasks, wherein each image in each demonstration sequence is assigned to a respective subtask of the plurality of subtasks; and determining, from the feature values for the images in the demonstration sequences, a respective set of discriminative features for each of the plurality of subtasks.

Abstract: Training a machine learning model (e.g., a neural network model such as a convolutional neural network (CNN) model) so that, when trained, the model can be utilized in processing vision data (e.g., from a vision component of a robot), that captures an object, to generate a rich object-centric embedding for the vision data. The generated embedding can enable differentiation of even subtle variations of attributes of the object captured by the vision data.

Abstract: This description relates to a neural network that has multiple network parameters and is configured to receive an input observation characterizing a state of an environment and to process the input observation to generate a numeric embedding of the state of the environment. The neural network can be used to control a robotic agent. The network can be trained using a method comprising: obtaining a first observation captured by a first modality; obtaining a second observation that is co-occurring with the first observation and that is captured by a second, different modality; obtaining a third observation captured by the first modality that is not co-occurring with the first observation; determining a gradient of a triplet loss that uses the first observation, the second observation, and the third observation; and updating current values of the network parameters using the gradient of the triplet loss.

Abstract: Techniques are disclosed that enable processing a video capturing a periodic activity using a repetition network to generate periodic output (e.g., a period length of the periodic activity captured in the video and/or a frame wise periodicity indication of the video capturing the periodic activity). Various implementations include a class agnostic repetition network which can be used to generate periodic output for a wide variety of periodic activities. Additional or alternative implementations include generating synthetic repetition videos which can be utilized to train the repetition network.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for detecting intermediate reinforcement learning goals. One of the methods includes obtaining a plurality of demonstration sequences, each of the demonstration sequences being a sequence of images of an environment while a respective instance of a reinforcement learning task is being performed; for each demonstration sequence, processing each image in the demonstration sequence through an image processing neural network to determine feature values for a respective set of features for the image; determining, from the demonstration sequences, a partitioning of the reinforcement learning task into a plurality of subtasks, wherein each image in each demonstration sequence is assigned to a respective subtask of the plurality of subtasks; and determining, from the feature values for the images in the demonstration sequences, a respective set of discriminative features for each of the plurality of subtasks.

Abstract: Techniques are disclosed that enable training a goal-conditioned policy based on multiple data sets, where each of the data sets describes a robot task in a different way. For example, the multiple data sets can include: a goal image data set, where the task is captured in the goal image; a natural language instruction data set, where the task is described in the natural language instruction; a task ID data set, where the task is described by the task ID, etc. In various implementations, each of the multiple data sets has a corresponding encoder, where the encoders are trained to generate a shared latent space representation of the corresponding task description. Additional or alternative techniques are disclosed that enable control of a robot using a goal-conditioned policy network. For example, the robot can be controlled, using the goal-conditioned policy network, based on free-form natural language input describing robot task(s).

Abstract: There are provided systems, methods, and apparatus, for optimizing a policy controller to control a robotic agent that interacts with an environment to perform a robotic task. One of the methods includes optimizing the policy controller using a neural network that generates numeric embeddings of images of the environment and a demonstration sequence of demonstration images of another agent performing a version of the robotic task.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for detecting intermediate reinforcement learning goals. One of the methods includes obtaining a plurality of demonstration sequences, each of the demonstration sequences being a sequence of images of an environment while a respective instance of a reinforcement learning task is being performed; for each demonstration sequence, processing each image in the demonstration sequence through an image processing neural network to determine feature values for a respective set of features for the image; determining, from the demonstration sequences, a partitioning of the reinforcement learning task into a plurality of subtasks, wherein each image in each demonstration sequence is assigned to a respective subtask of the plurality of subtasks; and determining, from the feature values for the images in the demonstration sequences, a respective set of discriminative features for each of the plurality of subtasks.

Abstract: There are provided systems, methods, and apparatus, for optimizing a policy controller to control a robotic agent that interacts with an environment to perform a robotic task. One of the methods includes optimizing the policy controller using a neural network that generates numeric embeddings of images of the environment and a demonstration sequence of demonstration images of another agent performing a version of the robotic task.

Abstract: This description relates to a neural network that has multiple network parameters and is configured to receive an input observation characterizing a state of an environment and to process the input observation to generate a numeric embedding of the state of the environment. The neural network can be used to control a robotic agent. The network can be trained using a method comprising: obtaining a first observation captured by a first modality; obtaining a second observation that is co-occurring with the first observation and that is captured by a second, different modality; obtaining a third observation captured by the first modality that is not co-occurring with the first observation; determining a gradient of a triplet loss that uses the first observation, the second observation, and the third observation; and updating current values of the network parameters using the gradient of the triplet loss.

Abstract: This description relates to a neural network that has multiple network parameters and is configured to receive an input observation characterizing a state of an environment and to process the input observation to generate a numeric embedding of the state of the environment. The neural network can be used to control a robotic agent. The network can be trained using a method comprising: obtaining a first observation captured by a first modality; obtaining a second observation that is co-occurring with the first observation and that is captured by a second, different modality; obtaining a third observation captured by the first modality that is not co-occurring with the first observation; determining a gradient of a triplet loss that uses the first observation, the second observation, and the third observation; and updating current values of the network parameters using the gradient of the triplet loss.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for controlling an agent. One of the methods includes controlling the agent using a policy neural network that processes a policy input that includes (i) a current observation, (ii) a goal observation, and (iii) a selected latent plan to generate a current action output that defines an action to be performed in response to the current observation.

Abstract: An encoder neural network is described which can encode a data item, such as a frame of a video, to form a respective encoded data item. Data items of a first data sequence are associated with respective data items of a second sequence, by determining which of the encoded data items of the second sequence is closest to the encoded data item produced from each data item of the first sequence. Thus, the two data sequences are aligned. The encoder neural network is trained automatically using a training set of data sequences, by an iterative process of successively increasing cycle consistency between pairs of the data sequences.

Abstract: Training a machine learning model (e.g., a neural network model such as a convolutional neural network (CNN) model) so that, when trained, the model can be utilized in processing vision data (e.g., from a vision component of a robot), that captures an object, to generate a rich object-centric embedding for the vision data. The generated embedding can enable differentiation of even subtle variations of attributes of the object captured by the vision data.

Abstract: There are provided systems, methods, and apparatus, for optimizing a policy controller to control a robotic agent that interacts with an environment to perform a robotic task. One of the methods includes optimizing the policy controller using a neural network that generates numeric embeddings of images of the environment and a demonstration sequence of demonstration images of another agent performing a version of the robotic task.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for detecting intermediate reinforcement learning goals. One of the methods includes obtaining a plurality of demonstration sequences, each of the demonstration sequences being a sequence of images of an environment while a respective instance of a reinforcement learning task is being performed; for each demonstration sequence, processing each image in the demonstration sequence through an image processing neural network to determine feature values for a respective set of features for the image; determining, from the demonstration sequences, a partitioning of the reinforcement learning task into a plurality of subtasks, wherein each image in each demonstration sequence is assigned to a respective subtask of the plurality of subtasks; and determining, from the feature values for the images in the demonstration sequences, a respective set of discriminative features for each of the plurality of subtasks.

Abstract: This description relates to a neural network that has multiple network parameters and is configured to receive an input observation characterizing a state of an environment and to process the input observation to generate a numeric embedding of the state of the environment. The neural network can be used to control a robotic agent. The network can be trained using a method comprising: obtaining a first observation captured by a first modality; obtaining a second observation that is co-occurring with the first observation and that is captured by a second, different modality; obtaining a third observation captured by the first modality that is not co-occurring with the first observation; determining a gradient of a triplet loss that uses the first observation, the second observation, and the third observation; and updating current values of the network parameters using the gradient of the triplet loss.