Abstract: A computer-readable medium stores instructions executable by one or more processors to implement a self-driving control architecture for controlling an autonomous vehicle. A perception and prediction component receives sensor data, and generates (1) an observed occupancy grid indicating which cells are currently occupied in a two-dimensional representation of the environment, and (2) predicted occupancy grids indicating which cells are expected to be occupied later. A mapping component provides navigation data for guiding the vehicle toward a destination, and a cost map generation component is configured to generate, based on the observed occupancy grid, the predicted occupancy grid(s), and the navigation data, cost maps that each specify numerical values representing a cost, at a respective instance of time, of occupying certain cells in a two-dimensional representation of the environment.