Abstract: A computer-implemented method of processing data comprising a plurality of observations associated with respective ordered input values to train a Gaussian process (GP) to model the data. The method includes initialising an ordered plurality of inducing input locations, and initialising parameters of a multivariate Gaussian distribution over a set of inducing states, each inducing state having components corresponding to a Markovian GP and one or more derivatives of the Markovian GP at a respective one of the inducing inputs. The initialised parameters include a mean vector and a banded Cholesky factor of a precision matrix for the multivariate Gaussian distribution. The method further includes iteratively modifying the parameters of the multivariate Gaussian distribution, to increase or decrease an objective function corresponding to a variational lower bound of a marginal log-likelihood of the observations under the Markovian GP.

Abstract: First parameters of a variational Gaussian process (VGP) (including a positive-definite matrix-valued slack parameter) are initialized and iteratively modified change an objective function comprising an expected log-likelihood for each training data item under a respective Gaussian distribution with a predictive variance depending on the slack parameter. Modifying the first parameters comprises, for each training data item, determining a respective gradient estimator for the expected log-likelihood and modifying the first parameters in dependence on the determined gradient estimators. At an optimal value of the slack parameter, the slack parameter equals an inverse of a covariance matrix for the set of inducing variables, and the objective function corresponds to a variational lower bound of a marginal log-likelihood for a posterior distribution corresponding to the GP prior conditioned on the training data.

Abstract: A computer-implemented method of processing training data comprising a plurality of training data items to determine parameters of a Gaussian process (GP) model comprising a variational Gaussian process (VGP) corresponding to a GP prior conditioned and marginalized with respect to a set of randomly-distributed inducing variables includes initializing first parameters of the VGP including a positive-definite matrix-valued slack parameter, and iteratively modifying the first parameters to increase or decrease an objective function comprising an expected log-likelihood for each training data item under a respective Gaussian distribution with a predictive variance depending on the slack parameter. At each iteration, modifying the first parameters comprises, for each training data item, determining a respective gradient estimator for the expected log-likelihood using a respective one of a plurality of processor cores, and modifying the first parameters in dependence on the determined gradient estimators.