Abstract: Systems and methods for providing a framework for predicting future frames using diverse sampling are provided. In one embodiment, a method for predicting future frames includes receiving a video having a first frame from a first time and a second frame from a second time. The first frame and the second frame are represented in image space. The method also includes updating a prediction model based on the video. The method further includes determining whether a stopping condition is satisfied. In response to determining that the stopping condition has been satisfied, the method includes generating a plurality of future frames for a third time after the second time. The plurality of future frames is generated based on a normalized distance metric that preserves distance of samples in the latent space to the image space. The method yet further includes selecting a candidate frame from the plurality of future frames.

Abstract: A system for trajectory prediction with agent prioritization is provided. A motion encoder is configured to determine future directions of agents of a plurality of agents and strengths of the agents based past trajectory information. The past trajectory information includes past trajectories of the agents at a number of time steps over a given time horizon for the agents. An inter-agent encoder is configured to determine the relations between the agents based on the future directions and the strengths. The relations identify motion impacts between the agents. The inter-agent encoder is further configured to calculate a motion prioritization score for each agent of the plurality of agents based on the relations between the agents. The motion prioritization scores define a priority order. The motion decoder is configured to calculate sequential predictions for future trajectories of the agents based on the motion prioritization scores.

Abstract: Systems and methods for uncertainty estimation in vehicle trajectory prediction are provided. In one embodiment, a method includes calculating predicted error scores for each static data point of a set of static data points for a new traffic scenario. A static data point corresponds to a location of the new traffic scenario. The method includes ranking the static data points of the set of static data points based on the predicted error scores. The method further includes selecting a predetermined percentage of the ranked static data points having the highest predicted error scores of the ranked static data points. The method includes identifying locations of the new traffic scenario corresponding to the selected ranked static data points. The method includes collecting additional data points from the identified locations of the new traffic scenario. The method includes training a decoder for a neural network based on the additional data points.

Abstract: Systems and methods for providing a framework for predicting future frames using diverse sampling are provided. In one embodiment, a method for predicting future frames includes receiving a video having a first frame from a first time and a second frame from a second time. The first frame and the second frame are represented in image space. The method also includes updating a prediction model based on the video. The method further includes determining whether a stopping condition is satisfied. In response to determining that the stopping condition has been satisfied, the method includes generating a plurality of future frames for a third time after the second time. The plurality of future frames is generated based on a normalized distance metric that preserves distance of samples in the latent space to the image space. The method yet further includes selecting a candidate frame from the plurality of future frames.

Abstract: Aspects related to accuracy prior and diversity prior based future prediction may include a diversity prior, a concatenator, a decoder, and a processor. The diversity prior may receive a feature extracted history portion of a time series of information and generate a diversity latent representation. The concatenator may concatenate the diversity latent representation and the feature extracted history portion to generate a second decoder input. The decoder may receive a first decoder input and a second decoder input, generate a first output based on the first decoder input, and generate a second output based on the diversity decoder input. The processor may generate an accuracy prior and diversity prior based future prediction based on the first output and the second output. The diversity prior may be trained during a training stage utilizing an accuracy prior distinct from the diversity prior.

Abstract: Object identification may be provided herein. A feature extractor may extract a first set of visual features, extract a second set of visual features, concatenate the first set of visual features, the second set of visual features, and a set of bounding box information, determine a number of object features and a global feature for a scene, and receive ego-vehicle feature information associated with an ego-vehicle. An object classifier may receive the number of object features, the global feature, and the ego-vehicle feature information, generate relational features with respect to relationships between each of the number of objects from the scene, and classify each of the number of objects from the scene based on the number of object features, the relational features, the global feature, the ego-vehicle feature information, and an intention of the ego-vehicle.

Abstract: A system and method for completing continual multi-agent trajectory forecasting with a graph-based conditional generative memory system that include receiving data associated with a surrounding location of an ego agent and inputting the data associated with the surrounding location of the ego agent to at least one episodic memory buffer and processing scene graphs associated with the surrounding location of the ego agent that are associated with the plurality of time steps. The system and method additionally include aggregating the data associated with the surrounding location of the ego agent associated with the plurality of time steps into mixed data and training a generative memory and a predictor with the mixed data. The system and method further include predicting future trajectories associated with traffic agents that are located within the surrounding location of the ego agent based on the training of the generative memory and the predictor.