Abstract: Each computer of a peer-to-peer (P2P) network performs an iterative computer-based modeling task defined by a set of training data including at least some training data that are not accessible to the other computers of the P2P network, and by a set of parameters including a shared parameter. The modeling task optimizes an objective function comparing a model parameterized by the set of parameters with the training data. Each iteration includes: performing an iterative gradient step update of parameter values stored at the computer based on the objective function; receiving parameter values of the shared parameter from other computers of the P2P network; adjusting the parameter value of the shared parameter stored at the computer by averaging the received parameter values; and sending the parameter value of the shared parameter stored at the computer to other computers of the P2P network.