Abstract: A system implemented by one or more computers comprises a visual encoder component configured to receive as input data representing a sequence of image frames, in particular representing objects in a scene of the sequence, and to output a sequence of corresponding state codes, each state code comprising vectors, one for each of the objects. Each vector represents a respective position and velocity of its corresponding object. The system also comprises a dynamic predictor component configured to take as input a sequence of state codes, for example from the visual encoder, and predict a state code for a next unobserved frame. The system further comprises a state decoder component configured to convert the predicted state code, to a state, the state comprising a respective position and velocity vector for each object in the scene. This state may represent a predicted position and velocity vector for each of the objects.

Abstract: A system implemented by one or more computers comprises a visual encoder component configured to receive as input data representing a sequence of image frames, in particular representing objects in a scene of the sequence, and to output a sequence of corresponding state codes, each state code comprising vectors, one for each of the objects. Each vector represents a respective position and velocity of its corresponding object. The system also comprises a dynamic predictor component configured to take as input a sequence of state codes, for example from the visual encoder, and predict a state code for a next unobserved frame. The system further comprises a state decoder component configured to convert the predicted state code, to a state, the state comprising a respective position and velocity vector for each object in the scene. This state may represent a predicted position and velocity vector for each of the objects.

Abstract: A system implemented by one or more computers comprises a visual encoder component configured to receive as input data representing a sequence of image frames, in particular representing objects in a scene of the sequence, and to output a sequence of corresponding state codes, each state code comprising vectors, one for each of the objects. Each vector represents a respective position and velocity of its corresponding object. The system also comprises a dynamic predictor component configured to take as input a sequence of state codes, for example from the visual encoder, and predict a state code for a next unobserved frame. The system further comprises a state decoder component configured to convert the predicted state code, to a state, the state comprising a respective position and velocity vector for each object in the scene. This state may represent a predicted position and velocity vector for each of the objects.

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for generating data items. One of the systems is a neural network system comprising a memory storing a plurality of template data items; one or more processors configured to select a memory address based upon a received input data item, and retrieve a template data item from the memory based upon the selected memory address; an encoder neural network configured to process the received input data item and the retrieved template data item to generate a latent variable representation; and a decoder neural network configured to process the retrieved template data item and the latent variable representation to generate an output data item.

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for generating data items. One of the systems is a neural network system comprising a memory storing a plurality of template data items; one or more processors configured to select a memory address based upon a received input data item, and retrieve a template data item from the memory based upon the selected memory address; an encoder neural network configured to process the received input data item and the retrieved template data item to generate a latent variable representation; and a decoder neural network configured to process the retrieved template data item and the latent variable representation to generate an output data item.

Abstract: A system implemented by one or more computers comprises a visual encoder component configured to receive as input data representing a sequence of image frames, in particular representing objects in a scene of the sequence, and to output a sequence of corresponding state codes, each state code comprising vectors, one for each of the objects. Each vector represents a respective position and velocity of its corresponding object. The system also comprises a dynamic predictor component configured to take as input a sequence of state codes, for example from the visual encoder, and predict a state code for a next unobserved frame. The system further comprises a state decoder component configured to convert the predicted state code, to a state, the state comprising a respective position and velocity vector for each object in the scene. This state may represent a predicted position and velocity vector for each of the objects.