Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for image rendering. In one aspect, a method comprises receiving a plurality of observations characterizing a particular scene, each observation comprising an image of the particular scene and data identifying a location of a camera that captured the image. In another aspect, the method comprises receiving a plurality of observations characterizing a particular video, each observation comprising a video frame from t31he particular video and data identifying a time stamp of the video frame in the particular video. In yet another aspect, the method comprises receiving a plurality of observations characterizing a particular image, each observation comprising a crop of the particular image and data characterizing the crop of the particular image. The method processes each of the plurality of observations using an observation neural network to determine a numeric representation as output.

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for training a neural network to (i) generate accurate network outputs for a machine learning task and (ii) generate intermediate outputs that can be used to reliably classify out-of-distribution inputs.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for generating data specifying a three-dimensional mesh of an object using an auto-regressive neural network.

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for image rendering. In one aspect, a method comprises receiving a plurality of observations characterizing a particular scene, each observation comprising an image of the particular scene and data identifying a location of a camera that captured the image. In another aspect, the method comprises receiving a plurality of observations characterizing a particular video, each observation comprising a video frame from the particular video and data identifying a time stamp of the video frame in the particular video. In yet another aspect, the method comprises receiving a plurality of observations characterizing a particular image, each observation comprising a crop of the particular image and data characterizing the crop of the particular image. The method processes each of the plurality of observations using an observation neural network to determine a numeric representation as output.

Abstract: Methods, systems and apparatus, including computer programs encoded on computer storage media. One of the methods includes obtaining a plurality of images of a macromolecule having a plurality of atoms, training a decoder neural network on the plurality of images, and after the training, generating a plurality of conformations for at least a portion of the macromolecule that each include respective three-dimensional coordinates of each of the plurality of atoms, wherein generating each conformation includes sampling a conformation latent representation from a prior distribution over conformation latent representations, processing a respective input including the sampled conformation latent representation using the decoder neural network to generate a conformation output that specifies three-dimensional coordinates of each of the plurality of atoms for the conformation, and generating the conformation from the conformation output.

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for generating a plurality of possible segmentations of an image. In one aspect, a method comprises: receiving a request to generate a plurality of possible segmentations of an image; sampling a plurality of latent variables from a latent space, wherein each latent variable is sampled from the latent space in accordance with a respective probability distribution over the latent space that is determined based on the image; generating a plurality of possible segmentations of the image, comprising, for each latent variable, processing the image and the latent variable using a segmentation neural network having a plurality of segmentation neural network parameters to generate the possible segmentation of the image; and providing the plurality of possible segmentations of the image in response to the request.

Abstract: A generative adversarial neural network system to provide a sequence of actions for performing a task. The system comprises a reinforcement learning neural network subsystem coupled to a simulator and a discriminator neural network. The reinforcement learning neural network subsystem includes a policy recurrent neural network to, at each of a sequence of time steps, select one or more actions to be performed according to an action selection policy, each action comprising one or more control commands for a simulator. The simulator is configured to implement the control commands for the time steps to generate a simulator output. The discriminator neural network is configured to discriminate between the simulator output and training data, to provide a reward signal for the reinforcement learning. The simulator may be non-differentiable simulator, for example a computer program to produce an image or audio waveform or a program to control a robot or vehicle.

Abstract: According to a first aspect a network system to generate output data values from input data values according to one or In more learned data distributions comprises an input to receive a set of observations, each comprising a respective first data value for a first variable and a respective second data value for a second variable dependent upon the first variable. The system may comprise an encoder neural network system configured to encode each observation of the set of observations to provide an encoded output for each observation. The system may further comprise an aggregator configured to aggregate the encoded outputs for the set of observations and provide an aggregated output. The system may further comprise a decoder neural network system configured to receive a combination of the aggregated output and a target input value and to provide a decoder output. The target input value may comprise a value for the first variable and the decoder output may predict a corresponding value for the second variable.

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for generating a plurality of possible segmentations of an image. In one aspect, a method comprises: receiving a request to generate a plurality of possible segmentations of an image; sampling a plurality of latent variables from a latent space, wherein each latent variable is sampled from the latent space in accordance with a respective probability distribution over the latent space that is determined based on the image; generating a plurality of possible segmentations of the image, comprising, for each latent variable, processing the image and the latent variable using a segmentation neural network having a plurality of segmentation neural network parameters to generate the possible segmentation of the image; and providing the plurality of possible segmentations of the image in response to the request.

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for image rendering. In one aspect, a method comprises receiving a plurality of observations characterizing a particular scene, each observation comprising an image of the particular scene and data identifying a location of a camera that captured the image. In another aspect, the method comprises receiving a plurality of observations characterizing a particular video, each observation comprising a video frame from the particular video and data identifying a time stamp of the video frame in the particular video. In yet another aspect, the method comprises receiving a plurality of observations characterizing a particular image, each observation comprising a crop of the particular image and data characterizing the crop of the particular image. The method processes each of the plurality of observations using an observation neural network to determine a numeric representation as output.

Abstract: An inference engine is described for efficient machine learning. For example, an inference engine executes a plurality of ordered steps to carry out inference on the basis of observed data. For each step, a plurality of inputs to the step are received. A predictor predicts an output of the step and computes uncertainty of the prediction. Either the predicted output or a known output is selected on the basis of the uncertainty. If the known output is selected, the known output is computed, (for example, using a resource intensive, accurate process). The predictor is retrained using the known output and the plurality of inputs of the step as training data. For example, computing the prediction is fast and efficient as compared with computing the known output.

Abstract: Systems and methods for designing reconfigurable integrated circuits receive target data and training data; and generate a circuit design for implementing the target data which is over-provisioned with respect to the target data according to the training data.

Abstract: An inference engine is described for efficient machine learning. For example, an inference engine executes a plurality of ordered steps to carry out inference on the basis of observed data. For each step, a plurality of inputs to the step are received. A predictor predicts an output of the step and computes uncertainty of the prediction. Either the predicted output or a known output is selected on the basis of the uncertainty. If the known output is selected, the known output is computed, (for example, using a resource intensive, accurate process). The predictor is retrained using the known output and the plurality of inputs of the step as training data. For example, computing the prediction is fast and efficient as compared with computing the known output.

Abstract: Systems and methods for designing reconfigurable integrated circuits receive target data and training data; and generate a circuit design for implementing the target data which is over-provisioned with respect to the target data according to the training data.