Abstract: A method includes determining a routing graph based on a map, wherein the routing graph represents locations from the map; determining a routing horizon based on the routing graph, wherein the routing horizon defines a portion of the routing graph that is usable for routing; determining a cost-action structure that includes, for each location of the routing horizon, a cost-to-go value for travel from a respective location from the routing horizon to a destination, wherein the cost-to-go values implicitly describe an optimal route for travel from each of the respective locations from the routing horizon to the destination; receiving, from an autonomous agent, a request that specifies one of the locations from the routing horizon; and providing, to the autonomous agent, information from the cost-action structure based on the specified location from the routing horizon.