Abstract: Systems and methods are disclosed to implement a distributed matrix decomposition system using gossip. In embodiments, the matrix decomposition system employs a scalable, parallel, and decentralized approach to divide an input matrix into a grid blocks, and individually decompose the blocks into local decomposed matrices by communicating (gossiping) with a limited set of neighboring blocks. In embodiments, the decomposition may be implemented as an iterative process using Stochastic Gradient Descent, where the decomposed matrices are iteratively updated and kept in approximate agreement for neighboring blocks. The division of the input matrix allows the decomposition operation to be easily parallelized among nodes of a distributed computing system and scaled to suit the size of the input matrix. Moreover, the distributed approach eliminates the need for a central server, which in some systems may represent an operational bottle neck, a single point of failure, or a target for attacks.