Abstract: A variational hyper recurrent neural network (VHRNN) can be trained by, for each step in sequential training data: determining a prior probability distribution for a latent variable from a prior network of the VHRNN using an initial hidden state; determining a hidden state from a recurrent neural network (RNN) of the VHRNN using an observation state, the latent variable and the initial hidden state; determining an approximate posterior probability distribution for the latent variable from an encoder network of the VHRNN using the observation state and the initial hidden state; determining a generating probability distribution for the observation state from a decoder network of the VHRNN using the latent variable and the initial hidden state; and maximizing a variational lower bound of a marginal log-likelihood of the training data. The trained VHRNN can be used to generate sequential data.

Abstract: A system for machine learning architecture for time series data prediction. The system may be configured to: maintain a data set representing a neural network having a plurality of weights; obtain time series data associated with a data query; generate, using the neural network and based on the time series data, a predicted value based on a sampled realization of the time series data and a normalizing flow model, the normalizing flow model based on a latent continuous-time stochastic process having a stationary marginal distribution and bounded variance; and generate a signal providing an indication of the predicted value associated with the data query.

Abstract: Systems and methods for machine learning architecture for time series data prediction. The system may include a processor and a memory storing processor-executable instructions. The processor-executable instructions, when executed, may configure the processor to: obtain time series data associated with a data query; generate a predicted value based on a sampled realization of the time series data and a continuous time generative model, the continuous time generative model trained to define an invertible mapping to maximize a log-likelihood of a set of predicted values for a time range associated with the time series data; and generate a signal providing an indication of the predicted value associated with the data query.

Abstract: A variational hyper recurrent neural network (VHRNN) can be trained by, for each step in sequential training data: determining a prior probability distribution for a latent variable from a prior network of the VHRNN using an initial hidden state; determining a hidden state from a recurrent neural network (RNN) of the VHRNN using an observation state, the latent variable and the initial hidden state; determining an approximate posterior probability distribution for the latent variable from an encoder network of the VHRNN using the observation state and the initial hidden state; determining a generating probability distribution for the observation state from a decoder network of the VHRNN using the latent variable and the initial hidden state; and maximizing a variational lower bound of a marginal log-likelihood of the training data. The trained VHRNN can be used to generate sequential data.