Abstract: A computer-implemented video generation neural network system, configured to determine a value for each of a set of object latent variables by sampling from a respective prior object latent distribution for the object latent variable. The system comprises a trained image frame decoder neural network configured to, for each pixel of each generated image frame and for each generated image frame time step process determined values of the object latent variables to determine parameters of a pixel distribution for each of the object latent variables, combine the pixel distributions for each of the object latent variables to determine a combined pixel distribution, and sample from the combined pixel distribution to determine a value for the pixel and for the time step.

Abstract: A computer-implemented video generation neural network system, configured to determine a value for each of a set of object latent variables by sampling from a respective prior object latent distribution for the object latent variable. The system comprises a trained image frame decoder neural network configured to, for each pixel of each generated image frame and for each generated image frame time step process determined values of the object latent variables to determine parameters of a pixel distribution for each of the object latent variables, combine the pixel distributions for each of the object latent variables to determine a combined pixel distribution, and sample from the combined pixel distribution to determine a value for the pixel and for the time step.

Abstract: A computer-implemented video generation neural network system, configured to determine a value for each of a set of object latent variables by sampling from a respective prior object latent distribution for the object latent variable. The system comprises a trained image frame decoder neural network configured to, for each pixel of each generated image frame and for each generated image frame time step process determined values of the object latent variables to determine parameters of a pixel distribution for each of the object latent variables, combine the pixel distributions for each of the object latent variables to determine a combined pixel distribution, and sample from the combined pixel distribution to determine a value for the pixel and for the time step.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for controlling an agent interacting with an environment. One of the methods includes obtaining a representation of an observation; processing the representation using a convolutional long short-term memory (LSTM) neural network comprising a plurality of convolutional LSTM neural network layers; processing an action selection input comprising the final LSTM hidden state output for the time step using an action selection neural network that is configured to receive the action selection input and to process the action selection input to generate an action selection output that defines an action to be performed by the agent at the time step; selecting, from the action selection output, the action to be performed by the agent at the time step in accordance with an action selection policy; and causing the agent to perform the selected action.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for controlling an agent interacting with an environment. One of the methods includes obtaining a representation of an observation; processing the representation using a convolutional long short-term memory (LSTM) neural network comprising a plurality of convolutional LSTM neural network layers; processing an action selection input comprising the final LSTM hidden state output for the time step using an action selection neural network that is configured to receive the action selection input and to process the action selection input to generate an action selection output that defines an action to be performed by the agent at the time step; selecting, from the action selection output, the action to be performed by the agent at the time step in accordance with an action selection policy; and causing the agent to perform the selected action.