Abstract: Trajectory value learning for autonomous systems includes generating an environment image from sensor input and processing the environment image through an image neural network to obtain a feature map. Trajectory value learning further includes sampling possible trajectories to obtain a candidate trajectory for an autonomous system, extracting, from the feature map, feature vectors corresponding to the candidate trajectory, combining the feature vectors into the input vector, and processing, by a score neural network model, the input vector to obtain a projected score for the candidate trajectory. Trajectory value learning further includes selecting, from the candidate trajectories, the candidate trajectory as a selected trajectory based on the projected score, and implementing the selected trajectory.