Abstract: Provided are a computer program product, system, and method for generating virtual objects for a mixed reality display to render in a field of view of a user operating a vehicle. A view environment image of a user field of view of a user operating the vehicle is received. The view environment image and a user difficulty level for the user are inputted into a generator, comprising a neural network model, to generate an augmented environment image including virtual objects added to the view environment image. The virtual objects are transmitted to a mixed reality display to render in the field of view of the user.