Abstract: Methods and systems for decision making in an autonomous vehicle (AV) are described. A probabilistic explorer reduces the breadth and depth of the potentially infinite actions being explored allowing for an accurate prediction on a future scene to a defined time horizon and an appropriate selection of a goal state anywhere within that time horizon. The probabilistic explorer uses a neural network (NN) to suggest best (probabilistically speaking) actions for the AV and scene values, and a modified Monte Carlo Tree Search to identify a sequence of actions, where exploration is guided by the NN. The probabilistic explorer processes the suggested actions and driving scene(s) to provide estimated trajectories of all scene actors and an estimated trajectory for the AV at every time step for every action explored. A virtual driving scene is generated, which is iteratively processed to determine a vehicle goal state or vehicle low-level control actions.