Abstract: Model-based control of dynamical systems typically requires accurate domain-specific knowledge and specifications system components. Generally, steering actuator dynamics can be difficult to model due to, for example, an integrated power steering control module, proprietary black box controls, etc. Further, it is difficult to capture the complex interplay of non-linear interactions, such as power steering, tire forces, etc. with sufficient accuracy. To overcome this limitation, a recurring neural network can be employed to model the steering dynamics of an autonomous vehicle. The resulting model can be used to generate feedforward steering commands for embedded control. Such a neural network model can be automatically generated with less domain-specific knowledge, can predict steering dynamics more accurately, and perform comparably to a high-fidelity first principle model when used for controlling the steering system of a self-driving vehicle.

Abstract: An autonomous vehicle guidance system that generates a path for controlling an autonomous vehicle based at least in part on a data structure generated based at least in part on sensor data that may indicate occupied space in an environment surrounding an autonomous vehicle. The guidance system may receive a grid and generate a grid associated with the grid and the data structure. The guidance system may additionally or alternatively sub-sample the grid (latterly and/or longitudinally) dynamically based at least in part on characteristics determined from the data structure. The guidance system may identify a path based at least in part on a set of precomputed motion primitives, costs associated therewith, and/or a heuristic cost plot that indicates a cheapest cost to move from one pose to another.

Abstract: Systems and techniques for determining a sideslip vector for a vehicle that may have a direction that is different from that of a heading vector for the vehicle. The sideslip vector in a current vehicle state and sideslip vectors in predicted vehicles states may be used to determine paths for a vehicle through an environment and trajectories for controlling the vehicle through the environment. The sideslip vector may be based on a vehicle position that is the center point of the wheelbase of the vehicle and may include lateral velocity, facilitating the control of four-wheel steered vehicle while maintaining the ability to control two-wheel steered vehicles.

Abstract: Techniques for representing sensor data and predicted behavior of various objects in an environment are described herein. For example, an autonomous vehicle can represent prediction probabilities as an uncertainty model that may be used to detect potential collisions, define a safe operational zone or drivable area, and to make operational decisions in a computationally efficient manner. The uncertainty model may represent a probability that regions within the environment are occupied using a heat map type approach in which various intensities of the heat map represent a likelihood of a corresponding physical region being occupied at a given point in time.

Abstract: Techniques are discussed herein for determining optimal driving trajectories for autonomous vehicles in complex multi-agent driving environments. A baseline trajectory may be perturbed and parameterized into a vector of vehicle states associated with different segments (or portions) of the trajectory. Such a vector may be modified to ensure the resultant perturbed trajectory is kino-dynamically feasible. The vectorized perturbed trajectory may be input, including a representation of the current driving environment and additional agents, into a prediction model trained to output a predicted future driving scene. The predicted future driving scene, including predicted future states for the vehicle and predicted trajectories for the additional agents in the environment, may be evaluated to determine costs associated with each perturbed trajectory.

Abstract: Trajectory generation for controlling motion or other behavior of an autonomous vehicle may include alternately determining a candidate action and predicting a future state based on that candidate action. The technique may include determining a cost associated with the candidate action that may include an estimation of a transition cost from a current or former state to a next state of the vehicle. This cost estimate may be a lower bound cost or an upper bound cost and the tree search may alternately apply the lower bound cost or upper bound cost exclusively or according to a ratio or changing ratio. The prediction of the future state may be based at least in part on a machine-learned model's classification of a dynamic object as being a reactive object or a passive object, which may change how the dynamic object is modeled for the prediction.

Abstract: A reward determined as part of a machine learning technique, such as reinforcement learning, may be used to control an adversarial agent in a simulation such that a component for controlling motion of the adversarial agent is trained to reduce the reward. Training the adversarial agent component may be subject to one or more constraints and/or may be balanced against one or more additional goals. Additionally or alternatively, the reward may be used to alter scenario data so that the scenario data reduces the reward, allowing the discovery of difficult scenarios and/or prospective events.

Abstract: Techniques for accurately predicting and avoiding collisions with objects detected in an environment of a vehicle are discussed herein. A vehicle computing device can implement a model to output data indicating costs for potential intersection points between the object and the vehicle in the future. The model may employ a control policy and a time-step integrator to determine whether an object may intersect with the vehicle, in which case the techniques may include predicting vehicle actions by the vehicle computing device to control the vehicle.

Abstract: Drive envelope determination is described. In an example, a vehicle can capture sensor data while traversing an environment and can provide the sensor data to computing system(s). The sensor data can indicate agent(s) in the environment and the computing system(s) can determine, based on the sensor data, a planned path through the environment relative to the agent(s). The computing system(s) can also determine lateral distance(s) to the agent(s) from the planned path. In an example, the computing system(s) can determine modified distance(s) based at least in part on the lateral distance(s) and information about the agents. The computing system can determine a drive envelope based on the modified distance(s) and can determine a trajectory in the drive envelope.

Abstract: An autonomous vehicle guidance system that generates a path for controlling an autonomous vehicle based at least in part on a data structure generated based at least in part on sensor data that may indicate occupied space in an environment surrounding an autonomous vehicle. The guidance system may receive a grid and generate a grid associated with the grid and the data structure. The guidance system may additionally or alternatively sub-sample the grid (latterly and/or longitudinally) dynamically based at least in part on characteristics determined from the data structure. The guidance system may identify a path based at least in part on a set of precomputed motion primitives, costs associated therewith, and/or a heuristic cost plot that indicates a cheapest cost to move from one pose to another.

Abstract: Techniques for accurately predicting and avoiding collisions with objects detected in an environment of a vehicle are discussed herein. A vehicle computing device can implement a model to output data indicating costs for potential intersection points between the object and the vehicle in the future. The model may employ a control policy and a time-step integrator to determine whether an object may intersect with the vehicle, in which case the techniques may include predicting vehicle actions by the vehicle computing device to control the vehicle.

Abstract: Techniques for compensating for errors in position of a vehicle are discussed herein. In some cases, a discrepancy may exist between a measured state of the vehicle and a desired state as determined by a system of the vehicle. Techniques and methods for a planning architecture of an autonomous vehicle that is able to provide maintain a smooth trajectory as the vehicle follows a planned path or route. In some cases, a planning architecture of the autonomous vehicle may compensate for differences between an estimated state and a planned path without the use of a separate system. In this example process, the planning architecture may include a mission planning system, a decision system, and a tracking system that together output a trajectory for a drive system.

Abstract: A method for autonomously operating a driverless vehicle along a path between a first geographic location and a destination may include receiving communication signals from the driverless vehicle. The communication signals may include sensor data from the driverless vehicle and data indicating occurrence of an event associated with the path. The communication signals may also include data indicating that a confidence level associated with the path is less than a threshold confidence level due to the event. The method may also include determining, via a teleoperations system, a level of guidance to provide the driverless vehicle based on data associated with the communication signals, and transmitting teleoperations signals to the driverless vehicle. The teleoperations signals may include guidance to operate the driverless vehicle according to the determined level of guidance, so that a vehicle controller maneuvers the driverless vehicle to avoid, travel around, or pass through the event.

Abstract: Techniques for determining predictions on a top-down representation of an environment based on object movement are discussed herein. Sensors of a first vehicle (such as an autonomous vehicle) may capture sensor data of an environment, which may include object(s) separate from the first vehicle (e.g., a vehicle, a pedestrian, a bicycle). A multi-channel image representing a top-down view of the object(s) and the environment may be generated based in part on the sensor data. Environmental data (object extents, velocities, lane positions, crosswalks, etc.) may also be encoded in the image. Multiple images may be generated representing the environment over time and input into a prediction system configured to output a trajectory template (e.g., general intent for future movement) and a predicted trajectory (e.g., more accurate predicted movement) associated with each object. The prediction system may include a machine learned model configured to output the trajectory template(s) and the predicted trajector(ies).

Abstract: Techniques for generating more accurate determinations of object proximity by using vectors in data structures based on vehicle sensor data are disclosed. Vectors reflecting a distance and direction to a nearest object edge from a reference point in a data structure are used to determine a distance and direction from a point of interest in an environment to a nearest surface. In some examples, a weighted average query point response vector is determined using the determined distance vectors of cells neighboring the cell in which the point of interest is located and nearest to the same object as the query point, providing a more accurate estimate of the distance to the nearest object from the point of interest.

Abstract: This disclosure is directed to techniques for identifying relevant objects within an environment. For instance, a vehicle may use sensor data to determine a candidate trajectory associated with the vehicle and a predicted trajectory associated with an object. The vehicle may then use the candidate trajectory and the predicted trajectory to determine an interaction between the vehicle and the object. Based on the interaction, the vehicle may determine a time difference between when the vehicle is predicted to arrive at a location and when the object is predicted to arrive at the location. The vehicle may then determine a relevance score associated with the object using the time difference. Additionally, the vehicle may determine whether to input object data associated with the object into a planner component based on the relevance score. The planner component determines one or more actions for the vehicle to perform.

Abstract: Techniques for determining unified futures of objects in an environment are discussed herein. Techniques may include determining a first feature associated with an object in an environment and a second feature associated with the environment and based on a position of the object in the environment, updating a graph neural network (GNN) to encode the first feature and second feature into a graph node representing the object and encode relative positions of additional objects in the environment into one or more edges attached to the node. The GNN may be decoded to determine a predicted position of the object at a subsequent timestep. Further, a predicted trajectory of the object may be determined using predicted positions of the object at various timesteps.

Abstract: Techniques for determining unified futures of objects in an environment are discussed herein. Techniques may include determining a first feature associated with an object in an environment and a second feature associated with the environment and based on a position of the object in the environment, updating a graph neural network (GNN) to encode the first feature and second feature into a graph node representing the object and encode relative positions of additional objects in the environment into one or more edges attached to the node. The GNN may be decoded to determine a distribution of predicted positions for the object in the future that meet a criterion, allowing for more efficient sampling. A predicted position of the object in the future may be determined by sampling from the distribution.

Abstract: Techniques for determining predictions on a top-down representation of an environment based on vehicle action(s) are discussed herein. Sensors of a first vehicle (such as an autonomous vehicle) can capture sensor data of an environment, which may include object(s) separate from the first vehicle (e.g., a vehicle or a pedestrian). A multi-channel image representing a top-down view of the object(s) and the environment can be generated based on the sensor data, map data, and/or action data. Environmental data (object extents, velocities, lane positions, crosswalks, etc.) can be encoded in the image. Action data can represent a target lane, trajectory, etc. of the first vehicle. Multiple images can be generated representing the environment over time and input into a prediction system configured to output prediction probabilities associated with possible locations of the object(s) in the future, which may be based on the actions of the autonomous vehicle.

Abstract: The techniques discussed herein may comprise an autonomous vehicle guidance system that generates a path for controlling an autonomous vehicle based at least in part on a static object map and/or one or more dynamic object maps. The guidance system may identify a path based at least in part on determining set of nodes and a cost map associated with the static and/or dynamic object, among other costs, pruning the set of nodes, and creating further nodes from the remaining nodes until a computational or other limit is reached. The path output by the techniques may be associated with a cheapest node of the sets of nodes that were generated.