Abstract: Techniques are described for clustering scenes based on a scene representation that captures aggregated information related to the scene including objects in the scene, object trajectories, interactions between objects, and map data. Example scene representations may include aggregated labels in spatial bins relative to a driven trajectory of an autonomous vehicle, feature vectors describing a sequence of poses of objects over time and interactions between objects and the autonomous vehicle over time, and scene representations comprising embeddings from trained machine-learned prediction models. The scene may also be assigned a difficulty level based on a prediction accuracy of the prediction models when provided the scene as an input. The clustered scenes may be sampled for generating a dataset meeting specified criteria. For example, the dataset suitable for training a ML model may be generated that maintains a diversity of scenarios while avoiding repetition of common scenarios.

Abstract: Techniques for determining that a first vehicle is associated with a reverse state, and controlling a second vehicle based on the reverse state, are described herein. In some examples, the first vehicle may provide an indication that the first vehicle will be executing a reverse maneuver, such as with reverse lights on the vehicle or by positioning at an angle relative to a road or parking space to allow for the reverse maneuver into a desired location. A planning system of the second vehicle (such as an autonomous vehicle) may receive sensor data and determine a variety of these indications to determine a probability that the vehicle is going to execute a reverse maneuver. The second vehicle can further determine a likely trajectory of the reverse maneuver and can provide appropriate accommodations (e.g., time and/or space) to allow the second vehicle to execute the maneuver safely and efficiently.

Abstract: There is provided methods, systems, and computer-readable media for determining intention of bicycles and other person-wide vehicles. A method comprises receiving, from a first sensor of an autonomous vehicle, first sensor data relating to an external environment of the autonomous vehicle; and receiving, from a second sensor of the autonomous vehicle, second sensor data the second sensor comprising a different sensor type to the first sensor. A person-wide vehicle in proximate to the autonomous vehicle is identified. First object data associated with the person-wide vehicle is determined. Based on the first object data, a future intention of the person-wide vehicle is received from a machine-learned model. The autonomous vehicle is controlled based at least in part on the future intention of the person-wide vehicle.

Abstract: There is provided methods, systems, and computer-readable media for determining intention of bicycles and other person-wide vehicles. A method comprises: receiving, from one or more sensors of an autonomous vehicle, sensor data relating to an external environment of the autonomous vehicle; detecting, based at least in part on the sensor data, a person-wide vehicle in the external environment proximate to the autonomous vehicle; determining, based at least in part on the sensor data and the person-wide vehicle, image data including the person-wide vehicle; inputting the image data to a machine-learned model; receiving, from the machine-learned model based on the image data, a future intention of the person-wide vehicle; and controlling the autonomous vehicle based at least in part on the future intention of the person-wide vehicle.

Abstract: Techniques for determining that a first vehicle is associated with a reverse state, and controlling a second vehicle based on the reverse state, are described herein. In some examples, the first vehicle may provide an indication that the first vehicle will be executing a reverse maneuver, such as with reverse lights on the vehicle or by positioning at an angle relative to a road or parking space to allow for the reverse maneuver into a desired location. A planning system of the second vehicle (such as an autonomous vehicle) may receive sensor data and determine a variety of these indications to determine a probability that the vehicle is going to execute a reverse maneuver. The second vehicle can further determine a likely trajectory of the reverse maneuver and can provide appropriate accommodations (e.g., time and/or space) to allow the second vehicle to execute the maneuver safely and efficiently.

Abstract: Techniques to predict object behavior in an environment are discussed herein. For example, such techniques may include determining a trajectory of the object, determining an intent of the trajectory, and sending the trajectory and the intent to a vehicle computing system to control an autonomous vehicle. The vehicle computing system may implement a machine learned model to process data such as sensor data and map data. The machine learned model can associate different intentions of an object in an environment with different trajectories. A vehicle, such as an autonomous vehicle, can be controlled to traverse an environment based on object's intentions and trajectories.

Abstract: Techniques for determining that a first vehicle is associated with a reverse state, and controlling a second vehicle based on the reverse state, are described herein. In some examples, the first vehicle may provide an indication that the first vehicle will be executing a reverse maneuver, such as with reverse lights on the vehicle or by positioning at an angle relative to a road or parking space to allow for the reverse maneuver into a desired location. A planning system of the second vehicle (such as an autonomous vehicle) may receive sensor data and determine a variety of these indications to determine a probability that the vehicle is going to execute a reverse maneuver. The second vehicle can further determine a likely trajectory of the reverse maneuver and can provide appropriate accommodations (e.g., time and/or space) to allow the second vehicle to execute the maneuver safely and efficiently.

Abstract: Techniques to predict object behavior in an environment are discussed herein. For example, such techniques may include inputting data into a model and receiving an output from the model representing a discretized representation. The discretized representation may be associated with a probability of an object reaching a location in the environment at a future time. A vehicle computing system may determine a trajectory and a weight associated with the trajectory using the discretized representation and the probability. A vehicle, such as an autonomous vehicle, can be controlled to traverse an environment based on the trajectory and the weight output by the vehicle computing system.

Abstract: Techniques to predict object behavior in an environment are discussed herein. For example, such techniques may include inputting data into a model and receiving an output from the model representing a discretized representation. The discretized representation may be associated with a probability of an object reaching a location in the environment at a future time. A vehicle computing system may determine a trajectory and a weight associated with the trajectory using the discretized representation and the probability. A vehicle, such as an autonomous vehicle, can be controlled to traverse an environment based on the trajectory and the weight output by the vehicle computing system.

Abstract: Techniques to predict object behavior in an environment are discussed herein. For example, such techniques may include determining a trajectory of the object, determining an intent of the trajectory, and sending the trajectory and the intent to a vehicle computing system to control an autonomous vehicle. The vehicle computing system may implement a machine learned model to process data such as sensor data and map data. The machine learned model can associate different intentions of an object in an environment with different trajectories. A vehicle, such as an autonomous vehicle, can be controlled to traverse an environment based on object's intentions and trajectories.

Abstract: Techniques for determining that a first vehicle is associated with a reverse state, and controlling a second vehicle based on the reverse state, are described herein. In some examples, the first vehicle may provide an indication that the first vehicle will be executing a reverse maneuver, such as with reverse lights on the vehicle or by positioning at an angle relative to a road or parking space to allow for the reverse maneuver into a desired location. A planning system of the second vehicle (such as an autonomous vehicle) may receive sensor data and determine a variety of these indications to determine a probability that the vehicle is going to execute a reverse maneuver. The second vehicle can further determine a likely trajectory of the reverse maneuver and can provide appropriate accommodations (e.g., time and/or space) to allow the second vehicle to execute the maneuver safely and efficiently.