Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for training a neural network to perform an algorithmic task. According to one aspect, there is provided a method comprising: obtaining an input dataset; generating a first augmented dataset and a second augmented dataset, wherein for both the first augmented dataset and the second augmented dataset: applying the computational algorithm to the augmented dataset causes the same computational operations to be performed at a target computational step as would be performed by applying the computational algorithm to the input dataset; processing the first augmented dataset and the second augmented dataset using the neural network, comprising, for each augmented dataset: generating an intermediate representation of the augmented dataset at an intermediate layer of the neural network; and training the neural network on an objective function, wherein the objective function comprises a self-supervised loss term.