Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for planning the future trajectory of an autonomous vehicle in an environment.

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for training a policy neural network. In one aspect, a method for training a policy neural network configured to receive a scene data input and to generate a policy output to be followed by a target agent comprises: maintaining a set of training data, the set of training data comprising (i) training scene inputs and (ii) respective target policy outputs; at each training iteration: generating additional training scene inputs; generating a respective target policy output for each additional training scene input using a trained expert policy neural network that has been trained to receive an expert scene data input comprising (i) data characterizing the current scene and (ii) data characterizing a future state of the target agent; updating the set of training data; and training the policy neural network on the updated set of training data.

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for planning the future trajectory of an autonomous vehicle in an environment.

Abstract: A method for performing one or more tasks, wherein each of the one or more tasks includes predicting behavior of one or more agents in an environment, the method comprising: obtaining a three-dimensional (3D) input tensor representing behaviors of the one or more agents in the environment across a plurality of time steps; generating an encoded representation of the 3D input tensor by processing the 3D input tensor using an encoder neural network, wherein 3D input tensor comprises a plurality of observed cells and a plurality of masked cells; and processing the encoded representation of the 3D input tensor using a decoder neural network to generate a 4D output tensor.