Abstract: Systems and methods for training a machine learning (“ML”) model for use in controlling an autonomous vehicle (“AV”) are described herein. Implementations can obtain an initial state instance from driving of a vehicle, obtain ground truth label(s) for subsequent state instance(s) that each indicate a corresponding action of the vehicle for a corresponding time instance, perform, for a given time interval, a simulated episode, of locomotion of a simulated AV, generate, for each of a plurality of time instances of the given time interval, subsequent simulated state instance(s) that differ from the subsequent state instance(s), determine, using the ML model, and for each of the time instances, a predicted simulated action of the simulated AV based on the subsequent simulated operation instance(s), generate loss(es) based on the predicted simulated actions and the ground truth labels, and update the ML model based on the loss(es).

Abstract: Techniques are disclosed for evaluating an autonomous vehicle (“AV”) control system by determining deviations between data generated using the AV control system and manual driving data. In many implementations, manual driving data captures action(s) of a vehicle controlled by a manual driver. Additionally or alternatively, multiple AV control systems can be evaluated by comparing deviations for each AV control system, where the deviations are determined using the same set of manual driving data.

Abstract: Systems and methods related to controlling an autonomous vehicle (“AV”) are described herein. Implementations can obtain a plurality of instances that each include input and output. The input can include actor(s) from a given time instance of a past episode of locomotion of a vehicle, and stream(s) in an environment of the vehicle during the past episode. The actor(s) may be associated with an object in the environment of the vehicle at the given time instance, and the stream(s) may each represent candidate navigation paths in the environment of the vehicle. The output may include ground truth label(s) (or reference label(s)). Implementations can train a machine learning (“ML”) model based on the plurality of instances, and subsequently use the ML model in controlling the AV. In training the ML model, the actor(s) and stream(s) can be processed in parallel.

Abstract: Techniques are disclosed for training one or more cost functions of an autonomous vehicle (“AV”) control system based on difference between data generated using the AV control system and manual driving data. In many implementations, manual driving data captures action(s) of a vehicle controlled by a manual driver. Additionally or alternatively, multiple AV control systems can be evaluated by comparing deviations for each AV control system, where the deviations are determined using the same set of manual driving data.

Abstract: Implementations process, using machine learning (ML) layer(s) of ML model(s), actor(s) from a past episode of locomotion of a vehicle and stream(s) in an environment of the vehicle during the past episode to forecast associated trajectories, for the vehicle and for each of the actor(s), with respect to a respective associated stream of the stream(s). Further, implementations process, using a stream connection function, the associated trajectories to forecast a plurality of associated trajectories, for the vehicle and each of the actor(s), with respect to each of the stream(s). Moreover, implementations iterate between using the ML layer(s) and the stream connection function to update the associated trajectories for the vehicle and each of the actor(s). Implementations subsequently use the ML layer(s) in controlling an AV.