Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for using conversation histories to augment a software agent. In particular, the software agent can selectively and adaptively be provided with data from conversation history data that comprises respective segment data for each of one or more past segments of each of a plurality of past conversations between a user and a software agent, the respective segment data comprising a respective segment embedding for the past segment.

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for that can train a dual encoder model using a correction model to correct target embeddings at each training iteration without explicitly recalculating each target embedding. In one aspect, a system comprises obtaining approximated target embeddings for a plurality of target data items, processing the respective approximated target embeddings using a correction model to generate corrected target embeddings, processing a query data item using a query encoder model to generate a query embedding, electing, using the corrected target embeddings and the query embedding, a subset of the target data items as relevant target data items, and training the dual encoder model on a loss function for the retrieval task using the relevant target data items for the one or more query data items.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for resource navigation using neural networks.

Abstract: A reinforcement learning system is proposed in which a policy model neural network is trained to control an agent to perform a task in successive time steps, by training a control system including the policy model neural network to select a respective action for each time step which gives a high value for a reward function based on the action, and which indicates the contribution of the action to solving the task. The reward function includes a term based on a progress value output by a progress model. The progress model generates the progress value upon receiving a first observation of the state of the environment at a time step before the performance of the action, and a second observation of the state of the environment at a time step following the performance of the action.

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for predicting whether a given query is valid. One of the methods includes receiving a query comprising natural language text for verification; obtaining, from a set of text segments, a subset of relevant text segments that are relevant to the query; obtaining, from a current set of verified statements, a subset of relevant verified statements that are relevant to the query; generating, using a query verifier machine learning model, a prediction of whether the query is valid given the relevant text segments and the relevant verified statements; and determining whether to update the current set of verified statements to include the query based on the prediction of whether the query is valid.

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for designing a protein by jointly generating an amino acid sequence and a structure of the protein. In one aspect, a method comprises: generating data defining the amino acid sequence and the structure of the protein using a protein design neural network, comprising, for a plurality of positions in the amino acid sequence: receiving the current representation of the protein as of the current position: processing the current representation of the protein using the protein design neural network to generate design data for the current position that comprises: (i) data identifying an amino acid at the current position, and (ii) a set of structure parameters for the current position; and updating the current representation of the protein using the design data for the current position.

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for training a high-level controller neural network for controlling an agent. In particular, the high-level controller neural network generates natural language commands that can be provided as input to a low-level controller neural network, which generates control outputs that can be used to control the agent.

Abstract: A method is proposed of training a policy model to generate action data for controlling an agent to perform a task in an environment. The method comprises: obtaining, for each of a plurality of performances of the task, a corresponding demonstrator trajectory comprising a plurality of sets of state data characterizing the environment at each of a plurality of corresponding successive time steps during the performance of the task; using the demonstrator trajectories to generate a demonstrator model, the demonstrator model being operative to generate, for any said demonstrator trajectory, a value indicative of the probability of the demonstrator trajectory occurring; and jointly training an imitator model and a policy model.