Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for generating a system output using a remembered value of a neural network hidden state. In one aspect, a system comprises an external memory that maintains context experience tuples respectively comprising: (i) a key embedding of context data, and (ii) a value of a hidden state of a neural network at the respective previous time step. The neural network is configured to receive a system input and a remembered value of the hidden state of the neural network and to generate a system output. The system comprises a memory interface subsystem that is configured to determine a key embedding for current context data, determine a remembered value of the hidden state of the neural network based on the key embedding, and provide the remembered value of the hidden state as an input to the neural network.

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for generating a target sequence from an input sequence. In one aspect, a method comprises maintaining a set of current hypotheses, wherein each current hypothesis comprises an input prefix and an output prefix. For each possible combination of input and output prefix length, the method extends any current hypothesis that could reach the possible combination to generate respective extended hypotheses for each such current hypothesis; determines a respective direct score for each extended hypothesis using a direct model; determines a first number of highest-scoring hypotheses according to the direct scores; rescores the first number of highest-scoring hypotheses using a noisy channel model to generate a reduced number of hypotheses; and adds the reduced number of hypotheses to the set of current hypotheses.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for generating audio output samples predicted to be communicated by a user. One example system includes a first user device having a first user. The first user device initiates a communication session between the first user and a second user of a second user device. The first user device obtains a neural network model of the second user. The neural network model is trained to generate, conditioned on audio input samples received up to a current time step, an audio output sample predicted to be communicated by the second user at a next time step. The user device repeatedly provides received audio input samples as input to the neural network model and plays audio output samples generated by the neural network model in place of received audio input samples communicated by the second user.

Abstract: A computer-implemented method of training a student machine learning system comprises receiving data indicating execution of an expert, determining one or more actions performed by the expert during the execution and a corresponding state-action Jacobian, and training the student machine learning system using a linear-feedback-stabilized policy. The linear-feedback-stabilized policy may be based on the state-action Jacobian. Also a neural network system for representing a space of probabilistic motor primitives, implemented by one or more computers. The neural network system comprises an encoder configured to generate latent variables based on a plurality of inputs, each input comprising a plurality of frames, and a decoder configured to generate an action based on one or more of the latent variables and a state.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for generating an output sequence of audio data that comprises a respective audio sample at each of a plurality of time steps. One of the methods includes, for each of the time steps: providing a current sequence of audio data as input to a convolutional subnetwork, wherein the current sequence comprises the respective audio sample at each time step that precedes the time step in the output sequence, and wherein the convolutional subnetwork is configured to process the current sequence of audio data to generate an alternative representation for the time step; and providing the alternative representation for the time step as input to an output layer, wherein the output layer is configured to: process the alternative representation to generate an output that defines a score distribution over a plurality of possible audio samples for the time step.

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for performing visual speech recognition. In one aspect, a method comprises receiving a video comprising a plurality of video frames, wherein each video frame depicts a pair of lips; processing the video using a visual speech recognition neural network to generate, for each output position in an output sequence, a respective output score for each token in a vocabulary of possible tokens, wherein the visual speech recognition neural network comprises one or more volumetric convolutional neural network layers and one or more time-aggregation neural network layers; wherein the vocabulary of possible tokens comprises a plurality of phonemes; and determining a sequence of words expressed by the pair of lips depicted in the video using the output scores.

Abstract: A system implemented by one or more computers comprises a visual encoder component configured to receive as input data representing a sequence of image frames, in particular representing objects in a scene of the sequence, and to output a sequence of corresponding state codes, each state code comprising vectors, one for each of the objects. Each vector represents a respective position and velocity of its corresponding object. The system also comprises a dynamic predictor component configured to take as input a sequence of state codes, for example from the visual encoder, and predict a state code for a next unobserved frame. The system further comprises a state decoder component configured to convert the predicted state code, to a state, the state comprising a respective position and velocity vector for each object in the scene. This state may represent a predicted position and velocity vector for each of the objects.

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for selecting actions to be performed by an agent interacting with an environment. In one aspect, a system comprises a grid cell neural network and an action selection neural network. The grid cell network is configured to: receive an input comprising data characterizing a velocity of the agent; process the input to generate a grid cell representation; and process the grid cell representation to generate an estimate of a position of the agent in the environment; the action selection neural network is configured to: receive an input comprising a grid cell representation and an observation characterizing a state of the environment; and process the input to generate an action selection network output.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for processing video data. An example system receives video data and generates optical flow data. An image sequence from the video data is provided to a first 3D spatio-temporal convolutional neural network to process the image data in at least three space-time dimensions and to provide a first convolutional neural network output. A corresponding sequence of optical flow image frames is provided to a second 3D spatio-temporal convolutional neural network to process the optical flow data in at least three space-time dimensions and to provide a second convolutional neural network output. The first and second convolutional neural network outputs are combined to provide a system output.

Abstract: A method of generating an output image having an output resolution of N pixels×N pixels, each pixel in the output image having a respective color value for each of a plurality of color channels, the method comprising: obtaining a low-resolution version of the output image; and upscaling the low-resolution version of the output image to generate the output image having the output resolution by repeatedly performing the following operations: obtaining a current version of the output image having a current K×K resolution; and processing the current version of the output image using a set of convolutional neural networks that are specific to the current resolution to generate an updated version of the output image having a 2K×2K resolution.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for processing images using recurrent attention. One of the methods includes determining a location in the first image; extracting a glimpse from the first image using the location; generating a glimpse representation of the extracted glimpse; processing the glimpse representation using a recurrent neural network to update a current internal state of the recurrent neural network to generate a new internal state; processing the new internal state to select a location in a next image in the image sequence after the first image; and processing the new internal state to select an action from a predetermined set of possible actions.

Abstract: Methods and systems for determining an optimized setting for one or more process parameters of a machine learning training process are described. One of the methods includes processing a current network input using a recurrent neural network in accordance with first values of the network parameters to obtain a current network output, obtaining a measure of the performance of the machine learning training process with an updated setting defined by the current network output, and generating a new network input that includes (i) the updated setting defined by the current network output and (ii) the measure of the performance of the training process with the updated setting defined by the current network output.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for learning visual concepts using neural networks. One of the methods includes receiving a new symbol input comprising one or more symbols from a vocabulary; and generating a new output image that depicts concepts referred to by the new symbol input, comprising: processing the new symbol input using a symbol encoder neural network to generate a new symbol encoder output for the new symbol input; sampling, from the distribution parameterized by the new symbol encoder output, a respective value for each of a plurality of visual factors; and processing a new image decoder input comprising the respective values for the visual factors using an image decoder neural network to generate the new output image.

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for selecting actions to be performed by an agent interacting with an environment. In one aspect, a system includes a language encoder model that is configured to receive a text string in a particular natural language, and process the text string to generate a text embedding of the text string. The system includes an observation encoder neural network that is configured to receive an observation characterizing a state of the environment, and process the observation to generate an observation embedding of the observation. The system includes a subsystem that is configured to obtain a current text embedding of a current text string and a current observation embedding of a current observation. The subsystem is configured to select an action to be performed by the agent in response to the current observation.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for generating an adaptive audio-generation model. One of the methods includes generating an adaptive audio-generation model including learning a plurality of embedding vectors and parameter values of a neural network using training data comprising first text and audio data representing a plurality of different individual speakers speaking portions of the first text, wherein the plurality of embedding vectors represent respective voice characteristics of the plurality of different individual speakers.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for generating output images. One of the methods includes obtaining data specifying (i) a partitioning of the H by W pixel grid of the output image into K disjoint, interleaved sub-images and (ii) an ordering of the sub-images; and generating intensity values sub-image by sub-image, comprising: for each particular color channel for each particular pixel in each particular sub-image, generating, using a generative neural network, the intensity value for the particular color channel conditioned on intensity values for (i) any pixels that are in sub-images that are before the particular sub-image in the ordering, (ii) any pixels within the particular sub-image that are before the particular pixel in a raster-scan order over the output image, and (iii) the particular pixel for any color channels that are before the particular color channel in a color channel order.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for asynchronous deep reinforcement learning. One of the systems includes a plurality of workers, wherein each worker is configured to operate independently of each other worker, and wherein each worker is associated with a respective actor that interacts with a respective replica of the environment during the training of the deep neural network.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for compressing images using neural networks. One of the methods includes receiving an image; processing the image using an encoder neural network, wherein the encoder neural network is configured to receive the image and to process the image to generate an output defining values of a first number of latent variables that each represent a feature of the image; generating a compressed representation of the image using the output defining the values of the first number of latent variables; and providing the compressed representation of the image for use in generating a reconstruction of the image.

Abstract: A neural network system is proposed. The neural network can be trained by model-based reinforcement learning to select actions to be performed by an agent interacting with an environment, to perform a task in an attempt to achieve a specified result. The system may comprise at least one imagination core which receives a current observation characterizing a current state of the environment, and optionally historical observations, and which includes a model of the environment. The imagination core may be configured to output trajectory data in response to the current observation, and/or historical observations. The trajectory data comprising a sequence of future features of the environment imagined by the imagination core. The system may also include a rollout encoder to encode the features, and an output stage to receive data derived from the rollout embedding and to output action policy data for identifying an action based on the current observation.

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for language modeling. In one aspect, a system comprises: a masked convolutional decoder neural network that comprises a plurality of masked convolutional neural network layers and is configured to generate a respective probability distribution over a set of possible target embeddings at each of a plurality of time steps; and a modeling engine that is configured to use the respective probability distribution generated by the decoder neural network at each of the plurality of time steps to estimate a probability that a string represented by the target embeddings corresponding to the plurality of time steps belongs to the natural language.