Abstract: Techniques are discussed for determining prediction probabilities of an object based on a top-down representation of an environment. Data representing objects in an environment can be captured. Aspects of the environment can be represented as map data. A multi-channel image representing a top-down view of object(s) in the environment can be generated based on the data representing the objects and map data. The multi-channel image can be used to train a machine learned model by minimizing an error between predictions from the machine learned model and a captured trajectory associated with the object. Once trained, the machine learned model can be used to generate prediction probabilities of objects in an environment, and the vehicle can be controlled based on such prediction probabilities.

Abstract: Techniques are discussed for determining prediction probabilities of an object based on a top-down representation of an environment. Data representing objects in an environment can be captured. Aspects of the environment can be represented as map data. A multi-channel image representing a top-down view of object(s) in the environment can be generated based on the data representing the objects and map data. The multi-channel image can be used to train a machine learned model by minimizing an error between predictions from the machine learned model and a captured trajectory associated with the object. Once trained, the machine learned model can be used to generate prediction probabilities of objects in an environment, and the vehicle can be controlled based on such prediction probabilities.

Abstract: Techniques are discussed for determining predicted trajectories based on a top-down representation of an environment. Sensors of a first vehicle can capture sensor data of an environment, which may include agent(s) separate from the first vehicle, such as a second vehicle or a pedestrian. A multi-channel image representing a top-down view of the agent(s) and the environment and comprising semantic information can be generated based on the sensor data. Semantic information may include a bounding box and velocity information associated with the agent, map data, and other semantic information. Multiple images can be generated representing the environment over time. The image(s) can be input into a prediction system configured to output a heat map comprising prediction probabilities associated with possible locations of the agent in the future. A predicted trajectory can be generated based on the prediction probabilities and output to control an operation of the first vehicle.

Abstract: Techniques are discussed for determining predicted trajectories based on a top-down representation of an environment. Sensors of a first vehicle can capture sensor data of an environment, which may include agent(s) separate from the first vehicle, such as a second vehicle or a pedestrian. A multi-channel image representing a top-down view of the agent(s) and the environment and comprising semantic information can be generated based on the sensor data. Semantic information may include a bounding box and velocity information associated with the agent, map data, and other semantic information. Multiple images can be generated representing the environment over time. The image(s) can be input into a prediction system configured to output a heat map comprising prediction probabilities associated with possible locations of the agent in the future. A predicted trajectory can be generated based on the prediction probabilities and output to control an operation of the first vehicle.