Abstract: Provided method for training an autonomous driving model including a video prediction model, and the method including: determining, according to at least one of an initial video frame collected by a target vehicle or scenario description metadata of an initial video frame, a scenario context of the initial video frame; determining a vehicle movement instruction of the target vehicle according to at least one of the initial video frame or trajectory data of the target vehicle corresponding to the initial video frame; and training an initial model using the initial video frame and a control text corresponding to the initial video frame, to obtain the video prediction model, where the control text comprises the scenario context and the vehicle movement instruction, and the video prediction model is configured to output a predicted video frame.

Abstract: Provided method for training an autonomous driving model including a video prediction model, and the method including: determining, according to at least one of an initial video frame collected by a target vehicle or scenario description metadata of an initial video frame, a scenario context of the initial video frame; determining a vehicle movement instruction of the target vehicle according to at least one of the initial video frame or trajectory data of the target vehicle corresponding to the initial video frame; and training an initial model using the initial video frame and a control text corresponding to the initial video frame, to obtain the video prediction model, where the control text comprises the scenario context and the vehicle movement instruction, and the video prediction model is configured to output a predicted video frame.

Abstract: The present application provides a method and a unified framework system for full-stack autonomous driving planning. The method comprises: acquiring an image of a scene and converting the image into an image feature, and converting the image feature into a bird's-eye view feature map; detecting agents from the bird's-eye view feature map through track query vectors, and continuously tracking the agents; segmenting different types of map elements from the bird's-eye view feature map through map query vectors, and continuously updating the map elements; predicting a future trajectory of each agent using an interaction between the agents and the different map elements; predicting, according to the predicted future trajectory of each agent, an occupancy grid map over multi-steps into the future; and decoding an ego-vehicle query vector to generate a planned path of an ego-vehicle, and optimizing the planned path using the predicted future multi-step occupancy grid map.

Abstract: The embodiments of the present application discloses an image processing method, an image processing apparatus, an image processing device and a computer-readable storage medium.