Abstract: A computation node of a neural network training system is described. The node has a memory storing a plurality of gradients of a loss function of the neural network and an encoder. The encoder encodes the plurality of gradients by setting individual ones of the gradients either to zero or to a quantization level according to a probability related to at least the magnitude of the individual gradient. The node has a processor which sends the encoded plurality of gradients to one or more other computation nodes of the neural network training system over a communications network.