Abstract: In various embodiments, methods, systems, and autonomous vehicles are provided. In one exemplary embodiment, a method is provided that includes obtaining first sensor inputs pertaining to one or more actors in proximity to an autonomous vehicle; obtaining second sensor inputs pertaining to operation of the autonomous vehicle; obtaining, via a processor, first neural network outputs via a first neural network, using the first sensor inputs; and obtaining, via the processor, second neural network outputs via a second neural network, using the first network outputs and the second sensor inputs, the second neural network outputs providing one or more recommended actions for controlling the autonomous vehicle.