Abstract: Techniques are discussed herein for generating trajectories for controlling motion and/or other behaviors of vehicles in complex driving environments. In certain examples, a search algorithm may be used to determine and evaluate a set of possible candidate actions for a vehicle, including candidate actions based on a predetermined exploration policy and additional candidate actions based on machine learned models that output predicted behaviors for the vehicle based on the current driving environment. Costs associated the various candidate actions may be evaluated based on state transition costs and/or future state predictions of the driving environment. Certain examples may include a tree search using a combination of predetermined heuristic candidate actions and adaptive-learning candidate actions at various nodes within a tree structure representing a driving route from a current vehicle state to an intended end state.

Abstract: System, methods, and computer-readable media for determining metrics associated with simulations of proximity events within autonomous vehicle simulations. In some examples, a simulation may be created using driving log data from an autonomous vehicle. Portions of the simulation containing proximity events may be analyzed. Intent times for both the autonomous vehicle and agent vehicles within the simulation may be determined, and the deceleration required for each vehicle to avoid the proximity event may be calculated. These decelerations may be used to determine metrics for the likeliness of the proximity event and the avoidability of the proximity event by the autonomous vehicle. These metrics may be used to prioritize development tasks, assess performance of the simulation, and assess performance of vehicle control systems.

Abstract: Techniques for performing prediction based on relative position data and/or absolute yaw data are described herein. A vehicle may detect an object in an environment. The vehicle may generate an embedding associated with the object and input the embedding into a machine learned model. The machine learned model may add the absolute yaw of the object to the embedding, generate a rotation matrix based on the pose of the object, and apply the rotation matrix to the embedding. Based on modifying the embedding (or generating a modified embedding), the attention layer of the machine learned model may perform attention on the modified embedding and respond by outputting output data. The vehicle may rotate the output data (or generate rotated output data) which may be used by one or more machine learned models to predict object behavior, generate vehicle actions, etc.

Abstract: Techniques for generating a tree structure based on multiple machine-learned trajectories are described herein. A planning component (“ML system”) within a vehicle may receive and encode various types of sensor and/or vehicle data. The ML system can provide the encoded data as input to multiple machine-learning models (“ML models”), each of which may be trained to output a unique candidate trajectory for the vehicle follow. In some examples, each ML model may be trained to output a unique type of learned trajectory that causes the vehicle to perform a certain type of action. Using the learned candidate trajectories, the ML system may generate a tree structure that includes some or all of the candidate trajectories. The vehicle may determine a control trajectory based on the generation and traversal of the tree structure using a tree search algorithm, and may follow the control trajectory within the environment.

Abstract: A machine-learned model that uses sensor and/or perception data to directly determine controls for operating an autonomous vehicle may be trained by identifying a preferred trajectory between a human-driven and vehicle-controlled trajectory, and using a first loss determined between the vehicle-controlled trajectory and the path the autonomous vehicle ultimately ended up taking in a scenario and a second loss determined between the vehicle-controlled trajectory and the human-driven trajectory to refine the machine-learned model. The machine-learned model may additionally or alternatively be refined by a learned reward model constructed by replacing one or more output heads of the machine-learned model with a regression head that is trained using performance metrics determined for the vehicle-controlled trajectory.

Abstract: Techniques for determining a planned trajectory usable to control a vehicle in an environment are discussed herein. A computing device can receive multiple planned trajectories generated by different models, and determine to use one of the planned trajectories to control the vehicle at a future time. The models can represent machine learned models that are independently trained using different training data and one of the models may leverage human driving data during training. The techniques can also include determining a bias value to cause the vehicle to utilize a planned trajectory from a set of available planned trajectories.

Abstract: Techniques for generating a tree structure based on multiple machine-learned trajectories are described herein. A planning component (“ML system”) within a vehicle may receive and encode various types of sensor and/or vehicle data. The ML system can provide the encoded data as input to multiple machine-learning models (“ML models”), each of which may be trained to output a unique candidate trajectory for the vehicle follow. In some examples, each ML model may be trained to output a unique type of learned trajectory that causes the vehicle to perform a certain type of action. Using the learned candidate trajectories, the ML system may generate a tree structure that includes some or all of the candidate trajectories. The vehicle may determine a control trajectory based on the generation and traversal of the tree structure using a tree search algorithm, and may follow the control trajectory within the environment.

Abstract: A machine-learned architecture for estimating the cost determined by a cost function for a prediction node of a tree search for exploring potential paths for controlling a vehicle may include two portions: a set up portion that includes models trained to process static data and a second portion that processes dynamic object data. The respective portions of the architecture may comprise various models that determine intermediate outputs that may be projected into a space associated with estimated cost. That estimated cost may identify an estimate of an output of the cost function for paths that are based on a particular prediction node of the tree search.

Abstract: Techniques for determining a vehicle trajectory that causes a vehicle to navigate in an environment relative to one or more objects are described herein. In some cases, the techniques described herein relate to selectively expanding a tree structure (e.g., a decision tree structure) to efficiently search for simulation data that can be used to evaluate vehicle control trajectories. The tree structure may include state nodes representing observed and/or predicted environment states, and action nodes representing candidate actions the vehicle may take. By selectively and incrementally expanding the tree structure, more optimal trajectories can be determined without exhaustively evaluating every possible outcome.

Abstract: Techniques for improving operational decisions of an autonomous vehicle are discussed herein. In some cases, a system may generate reference graphs associated with a route of the autonomous vehicle. Such reference graphs can comprise precomputed feature vectors based on grid regions and/or lane segments. The feature vectors are usable to determine scene context data associated with static objects to reduce computational expenses and compute time.

Abstract: A machine-learned model that uses sensor and/or perception data to directly determine controls for operating an autonomous vehicle may be trained by identifying a preferred trajectory between a human-driven and vehicle-controlled trajectory, and using a first loss determined between the vehicle-controlled trajectory and the path the autonomous vehicle ultimately ended up taking in a scenario and a second loss determined between the vehicle-controlled trajectory and the human-driven trajectory to refine the machine-learned model. The machine-learned model may additionally or alternatively be refined by a learned reward model constructed by replacing one or more output heads of the machine-learned model with a regression head that is trained using performance metrics determined for the vehicle-controlled trajectory.

Abstract: A machine-learned architecture for estimating the cost determined by a cost function for a prediction node of a tree search for exploring potential paths for controlling a vehicle may include two portions: a set up portion that includes models trained to process static data and a second portion that processes dynamic object data. The respective portions of the architecture may comprise various models that determine intermediate outputs that may be projected into a space associated with estimated cost. That estimated cost may identify an estimate of an output of the cost function for paths that are based on a particular prediction node of the tree search.

Abstract: There is provided a system configured to receive data associated with a vehicle operating within an environment; generate, based at least in part on the data, a graph comprising a plurality of nodes, a node of the plurality associated with one or more of a vehicle operating in the environment, a road feature, an additional vehicle, or a pedestrian; input the graph into a self-supervised machine learned model comprising an encoder, wherein the machine learned model is trained to output a representation associated with the node; receive, from the self-supervised machine learned model, a representation associated with the node; and transmit the representation to a downstream machine learned model trained to output control data based at least in part on the representation, wherein the control data is configured to control the vehicle or another vehicle.

Abstract: A machine-learned architecture for generating a single trajectory or multiple trajectories for controlling a vehicle may comprise an embedding model that generates an embedding of a world state indicating environment, object, and/or other states, one or more machine-learned layers that determine a predicted world state embedding using the world state embedding, and concatenating, as combined data, that predicted world state embedding to a steering angle distribution and a velocity distribution. The combined data is provided as input to a machine-learned model (that may be a single machine-learned model or may comprise two separate machine-learned models) that determines a next steering angle distribution and a next velocity distribution. These distributions may be used as part of generating one or more trajectories for controlling the vehicle. The architecture may be iteratively used to create a series of distributions that are used to create one or more trajectories.

Abstract: Techniques for improving operational decisions of an autonomous vehicle are discussed herein. In some cases, a system may generate reference graphs associated with a route of the autonomous vehicle. Such reference graphs can comprise precomputed feature vectors based on grid regions and/or lane segments. The feature vectors are usable to determine scene context data associated with static objects to reduce computational expenses and compute time.

Abstract: Techniques for generating a tree structure based on multiple machine-learned trajectories are described herein. A planning component (“ML system”) within a vehicle may receive and encode various types of sensor and/or vehicle data. The ML system can provide the encoded data as input to multiple machine-learning models (“ML models”), each of which may be trained to output a unique candidate trajectory for the vehicle follow. In some examples, each ML model may be trained to output a unique type of learned trajectory that causes the vehicle to perform a certain type of action. Using the learned candidate trajectories, the ML system may generate a tree structure that includes some or all of the candidate trajectories. The vehicle may determine a control trajectory based on the generation and traversal of the tree structure using a tree search algorithm, and may follow the control trajectory within the environment.

Abstract: An imitation learning-based machine-learned (ML) model to augment or replace the prediction and/or planner components of an autonomous vehicle may be trained using a two stage and multi-discipline approach. A first stage of training may include training the ML component to output a predicted action associated with a target vehicle and modifying the ML component to reduce a difference between the predicted action and the observed action taken by the target vehicle. A second stage may use reinforcement learning to further tun the ML component. The resultant model may be used on its own, with enough training data, or to rank or weight candidate trajectories generated by a planning component of the vehicle. The ML component may provide embeddings of environment features to first transformer(s) that output to a long short-term memory that outputs to second transformer(s) to determine the predicted action.

Abstract: Trajectory generation and/or execution architecture is described. In an example, a first signal can be determined at a first frequency, wherein the first signal comprises information associated with causing the system to move to a location. Further, a second signal can be determined at a second frequency different from the first frequency and based at least in part on the first signal. A system can be controlled to move to the location, based at least in part on the second signal.

Abstract: Automating reinforcement learning for autonomous vehicles may include assigning a probability with a scenario and varying that probability based at least in part on changes in performance by the autonomous vehicle associated with that scenario. The amount of time and computational bandwidth required to train a machine-learned component of an autonomous vehicle and the accuracy of the machine-learned component may be improved by determining a reward for performance of the autonomous vehicle in a scenario based at least in part on an severity metric. The impact severity metric may be determined based at least in part on a velocity, angle, and/or interaction area associated with the impact.

Abstract: A machine-learned architecture for estimating the cost determined by a cost function for a prediction node of a tree search for exploring potential paths for controlling a vehicle may include two portions: a set up portion that includes models trained to process static data and a second portion that processes dynamic object data. The respective portions of the architecture may comprise various models that determine intermediate outputs that may be projected into a space associated with estimated cost. That estimated cost may identify an estimate of an output of the cost function for paths that are based on a particular prediction node of the tree search.