Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for parallelizing matrix operations. One of the methods includes implementing a neural network on a parallel processing device, the neural network comprising at least one sparse neural network layer, the sparse neural network layer being configured to receive an input matrix and perform matrix multiplication between the input matrix and a sparse weight matrix to generate an output matrix, the method comprising: for each row of the M rows of the output matrix, determining a plurality of tiles that each include one or more elements from the row; assigning, for each tile of each row, the tile to a respective one of a plurality of thread blocks of the parallel processing device; and computing, for each tile, respective values for each element in the tile using the respective thread block to which the tile was assigned.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for loading a matrix into a circuit having an array having M×N cells. One of the methods includes: receiving a plurality of non-zero input values from a first input matrix; receiving index metadata that indicates, for each non-zero input value in the plurality of input values, which cell of the M×N cells in the array the non-zero input value should be loaded into; sending the non-zero input values and the index metadata to the M×N cells; and at a particular cell of the M×N cells in the array: receiving a particular non-zero input value and corresponding index metadata; and determining from the corresponding index metadata for the particular non-zero input value whether to store the particular non-zero input value at the cell or to shift the particular non-zero input value to another cell.

Abstract: A neural network system includes at least one layer which applies a 1×1 convolution to a dense activation matrix, using a kernel defined by a sparse weight matrix. The layer is implemented by a processor with access to a sparsity dataset which indicates where the null weights are located in the weight matrix. The processor selects the feature values corresponding to the other weights from a memory unit configured to store the activation matrix, and then uses these extracted feature values for calculating the convolved values.