Abstract: Systems and methods for training a neural network are described. One or more embodiments of the present disclosure include training a neural network based on a first combined gradient of a loss function at a plurality of sampled elements of a dataset; receiving an insertion request that indicates an insertion element to be added to the dataset, or a deletion request that indicates a deletion element to be removed from the dataset, wherein the deletion element is one of the plurality of sampled elements; computing a second combined gradient of the loss function by adding the insertion element to the dataset or by replacing the deletion element with a replacement element from the dataset; determining whether the first combined gradient and the second combined gradient satisfy a stochastic condition; and retraining the neural network to obtain a modified neural network based on the determination.