Abstract: Methods for performing inference on a generative model are provided. In one aspect, a method includes receiving a generative model in a probabilistic program form defining variables and probabilistic relationships between variables, and producing a neural network to model the behaviour of the generative model. The input layer includes nodes corresponding to the variables of the generative model, and the output layer includes nodes corresponding to a parameter of the conditional marginal of the variables of the input layer. The method also includes training the neural network using samples from the probabilistic program. A loss function is provided for each node of the output layer. The loss function for each output node is independent of the loss functions for the other nodes of the output layer. The method also includes performing amortised inference on the generative model. Systems and machine-readable media are also provided.