Abstract: An online system receives contextual information for a goal-oriented environment at a current time and generates Q-value predictions that indicate likelihoods that one or more participants will reach the desired goal. The Q-value for a current time may also be interpreted as the value of the actions taken at the current time with respect to the desired goal. The online system generates Q-value predictions for a current time by applying an approximator network to the contextual information for the current time. In one instance, the approximator network is a machine learning model neural network model trained by a reinforcement learning process. The reinforcement process allows the approximator network to incrementally update the Q-value predictions given new information throughout time, and results in a more computationally efficient training process compared to other types of supervised or unsupervised machine learning model processes.