Abstract: A method includes providing a plurality of processes interconnected by a network, each of the plurality of processes being configured to hold a block of data destined for others of the plurality of processes. A set of data for all-to-all data exchange is received from one or more of the processes. The set of data is configured as a plurality of blocks of data in a matrix as matrix data, the matrix being distributed among the plurality of processes. The matrix data is transposed by changing the position of selected blocks of data of the plurality of blocks of data relative to the other blocks of data of the plurality of the blocks of data, without changing the structure of each of the blocks of data. The transposed matrix data is over the network and is then received, repacked, and conveyed to destination processes.

Abstract: A method in which a plurality of process are configured to hold a block of data destined for other processes, with data repacking circuitry including receiving circuitry configured to receive at least one block of data from a source process of the plurality of processes, the repacking circuitry configured to repack received data in accordance with at least one destination process of the plurality of processes, and sending circuitry configured to send the repacked data to the at least one destination process of the plurality of processes, receiving a set of data for all-to-all data exchange, the set of data being configured as a matrix, the matrix being distributed among the plurality of processes, and transposing the data by each of the plurality of processes sending matrix data from the process to the repacking circuitry, and the repacking circuitry receiving, repacking, and sending the resulting matrix data to destination processes.

Abstract: A method in which a plurality of process are configured to hold a block of data destined for other processes, with data repacking circuitry including receiving circuitry configured to receive at least one block of data from a source process of the plurality of processes, the repacking circuitry configured to repack received data in accordance with at least one destination process of the plurality of processes, and sending circuitry configured to send the repacked data to the at least one destination process of the plurality of processes, receiving a set of data for all-to-all data exchange, the set of data being configured as a matrix, the matrix being distributed among the plurality of processes, and transposing the data by each of the plurality of processes sending matrix data from the process to the repacking circuitry, and the repacking circuitry receiving, repacking, and sending the resulting matrix data to destination processes.

Abstract: A method in which a plurality of process are configured to hold a block of data destined for other processes, with data repacking circuitry including receiving circuitry configured to receive at least one block of data from a source process of the plurality of processes, the repacking circuitry configured to repack received data in accordance with at least one destination process of the plurality of processes, and sending circuitry configured to send the repacked data to the at least one destination process of the plurality of processes, receiving a set of data for all-to-all data exchange, the set of data being configured as a matrix, the matrix being distributed among the plurality of processes, and transposing the data by each of the plurality of processes sending matrix data from the process to the repacking circuitry, and the repacking circuitry receiving, repacking, and sending the resulting matrix data to destination processes.

Abstract: A method in which a plurality of process are configured to hold a block of data destined for other processes, with data repacking circuitry including receiving circuitry configured to receive at least one block of data from a source process of the plurality of processes, the repacking circuitry configured to repack received data in accordance with at least one destination process of the plurality of processes, and sending circuitry configured to send the repacked data to the at least one destination process of the plurality of processes, receiving a set of data for all-to-all data exchange, the set of data being configured as a matrix, the matrix being distributed among the plurality of processes, and transposing the data by each of the plurality of processes sending matrix data from the process to the repacking circuitry, and the repacking circuitry receiving, repacking, and sending the resulting matrix data to destination processes.