Abstract: A computer-implemented method that comprises obtaining an input sequence of network inputs, processing each network input in the input sequence using a recurrent neural network to generate a sequence of recurrent outputs that includes a respective recurrent output for each network input in the input sequence, generating a sub-sampled sequence that includes a proper subset of the respective recurrent outputs, and processing the sub-sampled sequence using a self-attention neural network to generate a network output for the input sequence. The self-attention neural network comprises a self-attention subnetwork configured to apply self-attention over the sub-sampled sequence to generate a respective updated output for each recurrent output in the sub-sampled sequence and an output neural network configured to process one or more of the updated outputs to generate the network output for the input sequence.

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for training an action selection policy neural network. In one aspect, a method comprises: obtaining an expert observation; processing the expert observation using a generative neural network system to generate a given observation-given action pair, wherein the generative neural network system has been trained to be more likely to generate a particular observation-particular action pair if performing the particular action in response to the particular observation is more likely to result in the environment later reaching the state characterized by a target observation; processing the given observation using the action selection policy neural network to generate a given action score for the given action; and adjusting the current values of the action selection policy neural network parameters to increase the given action score for the given action.

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for training an action selection policy neural network. In one aspect, a method comprises: obtaining an expert observation; processing the expert observation using a generative neural network system to generate a given observation-given action pair, wherein the generative neural network system has been trained to be more likely to generate a particular observation-particular action pair if performing the particular action in response to the particular observation is more likely to result in the environment later reaching the state characterized by a target observation; processing the given observation using the action selection policy neural network to generate a given action score for the given action; and adjusting the current values of the action selection policy neural network parameters to increase the given action score for the given action.