Abstract: With respect to a method of determining a split scheme, the method includes calculating, by one or more processors, data related to data transfer time, for each combination of parallelization axes at respective layers of a hierarchical memory computer, based on data transfer methods, a size of a problem to be calculated, and communication bandwidths between the layers. The data transfer methods are determined by the parallelization axes, and the parallelization axes indicate how to split the problem. The method further includes determining, by the one or more processors, a combination of the parallelization axes based on the data related to the data transfer time calculated for each combination of the parallelization axes.

Abstract: According to one embodiment, a gradient compressing apparatus includes a memory and processing circuitry. The memory stores data. The processing circuitry is configured to calculate statistics of gradients calculated regarding a plurality of parameters being learning targets, with respect to an error function in learning; determine, based on the statistics, whether or not to be a transmission parameter being a parameter which transmits gradients regarding each of the parameters, via a communication network; and quantize a gradient representative value being a representative value of gradients regarding the parameter determined to be a transmission parameter.